Artificial Economics: Agent-Based Methods in Finance, Game Theory and Their Applications

/HFWXUH1RWHVLQ(FRQRPLFV DQG0DWKHPDWLFDO6\VWHPV 



)RXQGLQJ(GLWRUV 0%HFNPDQQ +3.Q]L 0DQDJLQJ(GLWRUV 3URI'U*)DQGHO )DFKEHUHLFK:LUWVFKDIWVZLVVHQVFKDIWHQ )HUQXQLYHUVLWlW+DJHQ )HLWKVWU$9=,,+DJHQ*HUPDQ\ 3URI'U:7URFNHO ,QVWLWXWIU0DWKHPDWLVFKH:LUWVFKDIWVIRUVFKXQJ,0: 8QLYHUVLWlW%LHOHIHOG 8QLYHUVLWlWVVWU%LHOHIHOG*HUPDQ\ (GLWRULDO%RDUG $%DVLOH$'UH[O+'DZLG.,QGHUIXUWK:.UVWHQ86FKLWWNR



3KLOLSSH0DWKLHX %UXQR%HDXILOV 2OLYLHU%UDQGRX\(GV

$UWLILFLDO(FRQRPLFV   $JHQW%DVHG0HWKRGVLQ)LQDQFH *DPH7KHRU\DQG7KHLU$SSOLFDWLRQV

 

(GLWRUV 3URI3KLOLSSH0DWKLHX /,)/867/&LWp6FLHQWLILTXH 9LOOHQHXYHG·$VFT&HGH[ SKLOLSSHPDWKLHX#OLIOIU

3URI2OLYLHU%UDQGRX\ &/$5((867/ $YHQXHGX3HXSOH%HOJH /LOOH&HGH[)UDQFH ROLYLHUEUDQGRX\#XQLYOLOOHIU

'U%UXQR%HDXILOV /,)/867/&LWp6FLHQWLILTXH 9LOOHQHXYHG·$VFT&HGH[ EUXQREHDXILOV#OLIOIU

/LEUDU\RI&RQJUHVV&RQWURO1XPEHU

,661 ,6%16SULQJHU%HUOLQ+HLGHOEHUJ1HZ
Preface

Agent-based Computational Methods applied to the fields of Economics, Management Sciences, Game Theory or Finance^ have received a great deal of academic interest these past years, in relation with the Complex System approaches. Those fields deal with the computational study of economies (at large), as complex adaptive systems, implying interacting agents with cognitive skills. One of the first use of agent based models has been popularized by Axelrod, [1], in his theory of evolution of cooperation. In this early work, he used extensively computational simulations and methods in order to study strategic behaviour in the Iterated Prisoner's Dilemma. This work is still influencing many researches in various scientific fields. It has for instance been the foundations of a new approach of Game Theory, based on computational ideas. In the mid eighties, under the impulsion of the Santa-Fe Institute, and especially Christopher Langton, [3], a new field of research, called Artificial Life (AL), has emerged. The idea of AL was to mimic real life under its various aspects to understand the basic principles of life. This has lead to encompass wider ideas such as complexity, evolution, auto-organisation and emergence. All concepts induced by these approaches have influenced social scientists among others. Following these initial attempts to mix computational approaches and social sciences, for instance among the pioneering works using Agent-based Computational Economics in finance, one can refer to the Artificial Stock Market, [4]. This model, based on bounded rationality and inductive reasoning, [5], is one of the first allowing correct simulations of real world stock market dynamics. This work has been done by people coming fi-om various scientific fields (Economics, Game Theory, Computer Science and Finance). All these approaches intensively use computer simulation as well as artificial intelligence concepts mostly based on multi-agents systems. In this context, some of the most used models come from Game Theory. Therefore, Agent-based Computational Simulations is more and more an important methodology in many SocialSciences. It becomes now widely used to test theoretical models or to investigate their properties when analytical solutions are not possible. Le. ACE, ABMS, COT, ACF

VI

Preface

Artificial Economics'2005 is one attempt to gather scientists from various horizons that directly contribute to these fields. The book you have in hands reproduces all the papers that have been selected by the programme committee. AE'2005 aims and scopes were to present computer-science based multi-agent methodologies and tools with their applications to social-scientists (mainly people fi'om economics and the management sciences,) as well as to present uses and needs of multi-agent based models and their constraints, as used by these social scientists, to computer scientists. Additionally, it has been a great occasion to favor the meeting of people and ideas of these two communities, in order to be able to construct a much structured multi-disciplinary approach. For its first edition. Artificial Economics has presentend recent scientific advances in the fields of ACE, ABMS, CGT, ACF, but has also been widely open to methodological surveys. Two prestigious invited-speakers have proposed analysis and surveys on major issues related to Artificial Economics topics. Cristiano Castelfranchi, from the Institute of Cognitive Sciences and Technologies, University of Siena, has developed a talk on "The Invisible (Left) Hand" . For Pr. Catsellfanchi, Agent-based Social Simulation will be crucial for the solution of one of the most hard problems of economic theory : the spontaneous organization of a dynamic social order that cannot be planned but emerges out of intentional planning agents guided by their own choices. This is the problem that Hayek assumes to be the real reason for the existence of the Social Sciences. In his talk, Pr Castellfranchi has examined the crucial relationships between the intentional nature of the agents * actions and their explicit goals and preferences, and the possibly unintended finality or function of their behavior [He] argues in favor of cognitive architectures in computer simulations and proposes some solutions about the theoretical and functional relationships between agents' intentions and non-intentional purposes of their actions. [For him,] social order is not necessarily a real order or something good and desirable for the involved agents; nor necessarily the best possible solution. It can be bad for the social actors against their intentions and welfare although emerging from their choices and being stable and self-maintaining. Hayek's theory of spontaneous social order andElster's opposition between intentional explanation and functional [are also] criticized. Robert Axtell, from The Brookings Institution, Washington DC and the Santa Fe Institute, has emphasized a very stimulating reflection on "Very Large-Scale MultiAgent Systems and Emergent Macroeconomics". For Dr. Axtell, the relatively few applications of agent-based computing to macroeconomics retain much of the representative agent character of conventional macro. [Robert Axtell] points out that hardware developments will soon make possible the construction of very large scale (one million to 100 million agent) models that obviate the need for representative agents -either representative consumers, investors or single-agent firms. [He also argues] that the main impediment to creating empirically-relevant artificial agent economies on this scale is our current lack of understanding of realistic behavior of agents and institutions. [He] claims that this software bottleneck-what rules to write for our agents ?-is the primary challenge facing our research community.

Preface

VII

Artificial Economics 2005 has been a two-days symposium. 20 papers have been selected among roughly 40 submitted extended abstracts. The reviewing process has been blind, and each paper has been reviewed by three referees. Space and time limitations are the reasons why no more papers have been accepted, although many of the rejected submissions were really interesting. Nevertheless, the choice of avoiding parallel sessions has been made to favor interactions between participants. The contributions have been gathered in six sessions, each of them devoted to one of the following topics: Artificial Stock Markets, Learning in models, Case-Studies and Applications, Bottom-Up approaches. Methodological issues and Market Dynamics. This book is organized according to the same logic. Artificial Economics'2005 as well as this book is the result of the combinatory efforts of: Frederic AMBLARD - Universite de Toulouse 1, France Gerard BALLOT - ERMES, Universite de Paris 2, France Bruno BEAUFILS - LIFE, USTL, France Paul BOURGINE - CREA, Ecole Polytechnique, France Olivier BRANDOUY - CLAREE, USTL, France Charlotte BRUUN - Aalborg University, Danemark Jose Maria CASTRO CALDAS - ISCTE, DINAMIA, Portugal Christophe DEISSENBERG - GREQAM, France Jean-Paul DELAHAYE - LIFE, USTL, France Jacques FERBER - LIRMM, Universite de Montpellier II, France Bernard FORGUES - CLAREE, USTL, France Wander JAGER - University of Groningen, The Netherlands Marco JANSSEN - CIPEC, Indiana University, USA Alan KIRMAN - GREQAM, France Philippe LAMARRE - LINA, Universite de Nantes, France Luigi MARENGO - DSGSS, Universita di Teramo, Italy Philippe MATHIEU - LIFE, USTL, France Denis PHAN - Universite de Rennes I, France Juliette ROUCHIER - GREQAM, France Elpida TZAFESTAS - National Technical University of Athens, Greece Nicolaas VRIEND - Queen Mary University of London, United Kingdom Bernard WALLISER - CERAS, ENPC, France Murat YILDIZOGLU - IFREDE-E3I, Universite Montesquieu Bordeaux IV, France We also want to thank Rene Mandiau from the Universite of Valenciennes, as he has been a very precious additional referee. Let all of them be thanked for their participation in this scientific event, that surely appeals for fixrther similar manifestations.

Villeneuve d'ascq, June 2005

Philippe Mathieu Bruno Beaufils Olivier Brandouy

VIII

Preface

References 1. R. Axelrod and W.D. Hamilton (1981), The evolution of cooperation. Science, pp. 13901396 2. R. Axelrod (1984), The evolution of cooperation, Basic Books 3. C. Langton (1995), Artificial Life, an overview. The MIT Press 4. R.G. Palmer and W.B.Arthur and J.H. Holland and B. LeBaron and R Tayler (1994), Artificial Economic Life : A Simple Model of a Stockmarket, Physica D, vol 75, pp. 264274 5. B. Arthur (1994), Inductive Reasoning and Bounded Rationality: the El-Farol Problem, American Economic Review, vol 84, pp. 406-417

Contents

Artificial Stock Markets Time Series Properties from an Artificial Stock Market with a Walrasian Auctioneer Thomas Stumpert, Detlef Seese, Make Sunderkotter Market Dynamics and Agents Behaviors: a Computational Approach Julien Derveeuw Traders Imprint Themselves by Adaptively Updating their Own Avatar Gilles Daniel, Lev Muchnik, Sorin Solomon

3 15 27

Learning in Models Learning in Continuous Double Auction Market Marta Posada, Cesdreo Hernandez, Adolfo Lopez-Paredes Firms Adaptation in Dynamic Economic Systems Lilia Rejeb, Zahia Guessoum Firm Size Dynamics in a Cournot Computational Model Francesco Saraceno, Jason Barr

41 53 65

Case-Studies and Applications Emergence of a Self-Organized Dynamic Fishery Sector: Application to Simulation of the Small-Scale Fresh Fish Supply Chain in Senegal Jean Le Fur Multi-Agent Model of Trust in a Human Game Catholijn M. Jonker, Sebastiaan Meijer, Dmytro Tykhonov, Tim Verwaart

79 91

X

Contents

A Counterexample for the Bullwhip Effect in a Supply Chain Toshiji Kawagoe, Shihomi Wada

103

Bottom-Up Approaches Collective Efficiency in Two-Sided Matching Tomoko Fuku, Akira Namatame, Taisei Kaizouji

115

Complex Dynamics, Financial Fragility and Stylized Facts Domenico Delli Gatti, Edoardo Gaffeo, Mauro Gallegati, Gianfranco Giulioni, Alan Kirman, Antonio Palestrini, Alberto Russo

127

Noisy Trading in the Large Market Limit Mikhail Anufriev, Giulio Bottazzi

137

Emergence in Multi-Agent Systems: Cognitive Hierarchy, Detection, and Complexity Reduction part I: Methodological Issues Jean-Louis Dessalles, Denis Phan

147

Methodological Issues The Implications of Case-Based Reasoning in Strategic Contexts Luis R. Izquierdo, Nicholas M. Gotts

163

A Model of Myerson-Nash Equilibria in Networks Paolo Pin

175

Market Dynamics Stock Price Dynamics in Artificial Multi-Agent Stock Markets A. O,I Hoffmann, S.A. Delre, J.H. von Eije, W, Jager

191

Market Failure Caused by Quality Uncertainty Segismundo S. Izquierdo, Luis R. Izquierdo, Jose M. Galdn, Cesdreo IIerndndez203 Learning and the Price Dynamics of a Double-Auction Financial Market with Portfolio Traders Andrea Consiglio, Valerio Lacagnina, Annalisa Russino

215

How Do the Differences Among Order Distributions Affect the Rate of Investment Returns and the Contract Rate Shingo Yamamoto, Shihomi Wada, Toshiji Kawagoe

227

List of Contributors

Mikhail Anufriev S.Anna School for Advanced Studies Italy Jason Barr Rutgers University USA Giulio Bottazzi S. Anna School for Advanced Studies Italy Andrea Consiglio Universita degli Studi di Palermo Italy Gilles Daniel University of Manchester UK

Jean Louis Dessalles ENST France J.H. von Eije University of Groningen The Netherlands Tomoko Fuku National Defense Academy Japan Edoardo Gaffeo University of Trento Italy

Domenico Delli Gatti Catholic University of Milan Universita Politecnica delle Marche Italy

Jose M. Galan University of Burgos Spain

S.A. Delre University of Groningen The Netherlands

Mauro Gallegati Universita Politecnica delle Marche Italy

Julien Derveeuw Universite des Sciences et Technologies de Lille France

Gianfranco Giulioni Universita Politecnica delle Marche Italy

XII

List of Contributors

Nicholas M. Gotts The Macaulay Institute UK

Valerio Lacagnina Universita degli Studi di Palermo Italy

Zahia Guessoum Universite de Reims Universite de Paris VI France

Jean Le Fur Institut de Recherche pour le Developpement France

Cesareo Hernandez University of Valladolid Spain

Adolfo Lopez-Paredes University of Valladolid Spain

A.O.I. Hoffmann University of Groningen The Netherlands

Sebastiaan Meijer Wageningen UR The Netherlands

Luis R. Izquierdo The Macaulay Institute UK

Lev Muchnik Bar Ilan University Israel

Segismundo S. Izquierdo University of Valladolid Spain

Akira Namatame National Defense Academy Japan

W. Jager University of Groningen The Netherlands

Antonio Palestrini Universita di Teramo Universita Politecnica delle Marche Italy

Gatholijn M. Jonker Radboud University Nijmegen The Netherlands

Denis Phan Universite de Rennes I France

Taisei Kaizouji International Christian University Japan

Paolo Pin Universita Ca' Foscari di Venezia Italy

Toshiji Kawagoe Future University-Hakodate Japan

Marta Posada University of Valladolid Spain

Alan Kirman EHESS Universite d'Aix-Marseille III France

Lilia Rejeb Universite de Reims Universite de Paris VI France

List of Contributors Annalisa Russino Universita degli Studi di Palermo Italy

Thomas Stiimpert University of Karlsruhe Germany

Alberto Russo Universita Politecnica delle Marche Italy

Malte Sunderkotter University of Karlsruhe Germany

Francesco Saraceno Observatoire Fran^ais des Conjonctures Economiques France Detlef Seese University of Karlsruhe Germany Sorin Solomon Hebrew University of Jerusalem Israel ISI Foundation Italy

Dmytro Tykhonov Radboud University Nijmegen The Netherlands Tim Verwaart Wageningen UR The Netherlands Shingo Yamamoto Future University-Hakodate Japan Shihomi Wada Future University-Hakodate Japan

XIII

Artificial Stocl< iVIaricets

Time Series Properties from an Artificial Stock IVIarl^et with a Walrasian Auctioneer Thomas Stumpert, Detlef Seese, and Malte Sunderkotter Institute AIFB, University of Karlsruhe, Germany, {stuempert|seese}@aifb.uni-karlsruhe.de Summary. This paper presents the results from an agent-based stock market with a Walrasian auctioneer (Walrasian adaptive simulation market, abbrev.: WASIM) based on the Santa Fe artificial stock market (SF-ASM, see e.g. [1], [2],[3],[4],[5]). The model is purposely simple in order to show that a parsimonious nonlinear framework with an equilibrium model can replicate typical stock market phenomena including phases of speculative bubbles and market crashes. As in the original SF-ASM, agents invest in a risky stock (with price pt and stochastic dividend dt) or in a risk-free asset. One of the properties of SF-ASM is that the microscopic wealth of the agents has no influence on the macroscopic price of the risky asset (see [5]). Moreover, SF-ASM uses trading restrictions which can lead to a deviation from the underlying equilibrium model. Our simulation market uses a Walrasian auctioneer to overcome these shortcomings, i.e. the auctioneer builds a causality between wealth of each agent and the arising price function of the risky asset, and the auctioneer iterates toward the equilibrium. The Santa Fe artificial stock market has been criticized because the mutation operator for producing new trading rules is not bit-neutral (see [6]). That means with the original SFASM mutation operator the trading rules are generalized, which also could be interpreted as a special market design. However, using the original non bit-neutral mutation operator with fast learning agents there is a causality between the used technical trading rules and a deviation from an intrinsic value of the risky asset in SF-ASM. This causality gets lost when using a bit-neutral mutation operator. WASIM uses this bit-neutral mutation operator and presents a model in which highfluctuationsand deviations occur due to extreme wealth concentrations. We introduce a Herfindahl index measuring these wealth concentrations and show reasons for arising of market monopolies. Instabilities diminish with introducing a Tobin tax which avoids that rich and influential agents emerge.

1 Introduction and Model Description One main focus of agent-based financial markets is to find possible reasons to explain phenomena observed on real-world markets v^hich cannot be explained with classical equilibrium models (see e.g. [8]). Among numerous agent-based financial markets, the Santa Fe artificial stock market is one of the pioneering and thus probably most well-knovm market model. The Walrasian adaptive simulation market (WASIM) is

4

Thomas Stiimpert et al.

based on the Santa Fe artificial stock market with the purpose to improve it. The market design of SF-ASM allows no connection between the price function of the risky asset and the wealth of the agents (see [5]), i.e. stock price dynamics are modelled to be independent from the influence of each agent's wealth. In the Walras simulation market we use a Walrasian auctioneer to build a causality between the equilibrium model and the wealth of agents. Operating each period, the Walrasian auctioneer permits interaction of agents on a microscopic level that has influence on the stock price on a macroscopic level. Agents try to forecastfixtureprices and dividends, and then combine these forecasts with their own preferences for risk and return. The market consists of stock shares and a risk-free asset. WASIM uses a bit-neutral mutation operator (see [6]) and presents a model in which highfluctuationsoccur due to extreme wealth concentrations. 1.1 The Market and the Market Structure

The underlying basic model of SF-ASM and WASIM is an equilibrium model, i.e. a model where supply and demand are balanced and the market is cleared each period. The market consists of iV heterogeneous agents. Agents invest in a risky stock (with price pt and dividend dt) or a. risk-fi*ee asset. The demand function of all agents is equal to the sum of all stock shares on the market: N

J2'^i,t = N

(1)

i=l

where xu denotes the number of shares agent i demands to possess in t and N denotes the absolute number of shares on the market (which is equal to the number of agents). In the equilibrium model agents can be short sellers, i.e. they can sell more stock shares than they possess. Furthermore agents can buy more stocks than they can afford, because the equilibrium model is independent of each agent's wealth function. To satisfy the equilibrium condition, it is even possible that some agents must buy a negative amount of shares of the risk-free asset. The risk-free asset is supported infinitely and has afixedreturn r. The stock has a stochastic dividend dt, dt = d + p'{dt-i

-d)-{-£t

(2)

(with J = 10, p = 0.95, €t = A/'(0,cr^)). In each period agents evaluate their portfolio and use a market structure vector to estimate how much stocks they want to buy or sell in the next period. At the beginning of each period the market structure vector Zt is calculated, Zt : {pt,...,Pt-k^dt,...,dt-n)^

{0,1} X ... X {0,1},

with/c > 0, n > 0. (3)

The market structure vector Zt identifies the basic technical and fundamental market state, e.g. Zt signals the relative strength (technical market state) and Zt signals whether a stock is fundamentally overvalued resp. undervalued (fundamental market state), seefigure1. In the following, we will call bits 1-6 of Zt fundamental bits, bits

Time Series Properties from an Artificial Stock Market with a Walrasian Auctioneer Bit

Task

1-6 7 8 9 10 11 12

stock*retum/dividend > 0.25, 0.5,0.75, 0.875,1.0,1.125 pt > 5-period moving average pt > 10-period moving average Pt > 100-period moving average Pt > 500-period moving average control bit control bit Fig. 1. Market structure vector

7-10 technical bits and bits 11-12 test/control bits. Bits 11 and 12 are constantly set on (1) respectively off (0). Bits 7 — 10 show the recent price trend. The market structure vector builds the basis for the buy/sell orders of the agents. In order to determine the attractiveness of the risky asset the return of the stock resulting from the dividend process (dt/pt) is compared to the return of the risk-free asset r. This leads to

i = !:^=p..r-M.

(4)

Pt

If the ratio is greater (less) than 1, the stock is overvalued (undervalued) for a riskneutral investor. Setting equation (4) equal to 1 and solving it for pt leads to the intrinsic or fundamental value of the risky asset pt = p^ = ^ for a risk-neutral investor. The dividend process follovv^s an autoregressive process of order 1, i.e. an AR(l)-process. It can easily be shovm that the constant, d, absorbs the mean, i.e. the expected value of dt+k given the set of information available at time t is d: Et[dt-^i] = E t [ J + p{dt -d)f st^i] = J + pdt Et[dt^2] =Et[d_+ p{dt+i -d)+ et+2] = Et[d + p^dt + pst^i + et+2] = d + p'^dt Et[dt+k] = --^

=d + p^dt''^^

^^^ d.

Equations (4) and (5) imply that in both WASIM and SF-ASM the expected intrinsic value is constant, but not necessarily its second moment. 1.2 The Agents and their Prediction Rules In this section we describe how an agent chooses between different rules to calculate his demand on the basis of a mean-variance maximizer and the market structure vector. For calculation of the demand ftmction Xi^t of an agent i, we introduce forecasting rules. Each agent estimates the expected return of investing in the stock under risk and makes his buy/sell order to a predefined price. The remaining money is invested in the risk-free asset. A forecasting rule is a 3-tuple consisting of condition part, a forecasting part and a fitness measure, the prediction rule r (r = 1,2,3,..., 100) of agent i is defined as follows:

6

Thomas Stumpert et al.

PRi^r =

(rni,m2,...,mi2,a,hj)

where rrin € { 0 ; 1 ; # } with n = 1,2,3, ...,12 denotes the condition part, a G [0.7,1.2] and b G [—10,19] denote the forecasting part and / denotes the fitness measure. The forecasting part consists of m symbols (here 12), m is equal to the number of market situation bits of Zf. Each m^, k G [1,12], has the values 0, 1 or #. The resulting pattern (mi, m2, ms,...., 77112) fi'om the condition part is compared to market situation bits, # represents the don't-care-symbol i.e. both bits 0 and 1 are possible (# generalizes the prediction rule). The market situation bits evaluate both the attractiveness of the stock compared to the risk-free asset and the current stock price compared to its moving averages. A rule is called active in case that the comparison is true, i.e. if the symbols rrik fi*om the condition part match with all market situation bits bitk. Forecasting is a linear combination of two randomly chosen parameters a, b, the current price pt and dividend dt. From a and b the price and dividend is estimated as follows: Et\ptA-i + dt+i] = a{pt + dt) + b

.

(6)

The fitness of a rule should be high, if the rule has many "#". Now define the fitness of a forecasting rule, eli^^ = (1 - ^)e2_i,i,r + 0[{pt+i + dt+i) - Et,i,r{Pw

+ dt+i)]^

.

(7)

This leads to the fitness fixnction ft,i,r' ft,i,r := M - eli^, - Cs,

(8)

where /t,i,r M C s 0 ^t,i,r

= fitness of rule r of agent i in period t, = constant scaling factor, e.g. 0, = weight of s (here: C = 0.005), = number of symbols G {0; 1} in the condition part, (i.e. unequal to # ) , = speed of change of the fitness function, between 0.. 1, "" variance .

Then the expected price is calculated as follows: Ei^t{Pt+l + dt+l) = Ciript + dt) + br

.

(9)

From all active rules the rule with the highest fitness is chosen for forecasting the fixture stock price. After calculating the expected price, the agent puts his buy/sell order: ^ ^ _ %APt^i + dt^i) - (1 + r)pt where xi^t denotes the number of shares the agent wants to possess. 7 is the global risk aversion, which is constant for all agents (7 = 0.5) and df^ denotes the empirical

Time Series Propertiesfroman Artificial Stock Market with a Walrasian Auctioneer

7

variance of the forecast. The agent's new wealth Wi^t+i from investing in stock shares or in the risk-free asset is: Wi^t+i = Xi^t{pt+i + dt+i) + {1 + r){Wi,t -PtXi,t)'

(11)

Then the agent updates the fitness / of all active rules, i.e. agents buy or sell on the basis of those rules who perform best, and confirm or discard rules according to their performance. When some agents are bankrupt, they leave the market and are replaced by new agents, who bring new money into the market. A genetic algorithm (see [9],[2] and [4]) enables agents who started with a rule set containing bad performing rules to produce better performing ones (i.e. new a, b and / are generated). The genetic algorithm of an agent of WASIM is activated each k periods, k e [200, ...,300]. Then 20% of the forecasting rules with the lowest fitness are replaced (for details of the genetic algorithm see [4]). To overcome the limitation of the equilibrium model we introduce the concept of a Walrasian auctioneer which produces a dependency between the basic market equilibrium model and the wealth of the agents. 1.3 The Walrasian Auctioneer The Walrasian auctioneer needs the demand for shares of each agent to calculate the stock price iteratively. The basic underlying model of SF-ASM is the equilibrium model described in the previous section with additional trading restrictions, which may lead to situations when the equilibrium assumption is not fulfilled. Expanding the equilibrium model the auctioneer of WASIM both takes into account the wealth of agents and trading restrictions (see algorithm below). The Walrasian auctioneer is an iteration method towards an equilibrium point (see [8]). Let p be the initial price for the auctioneer and let Cu := Wi^t-i — xi^t-i • Pt-i be the free available cash of agent i, where Cu is a fiinction of the past paid dividends, the risk-free return r and the initial wealth {Wio = 100) of agent i. Then each agent i uses Ei^t{pt-\-i + <^t+i) to calculate his demand xu for all possible prices. The auctioneer executes the following algorithm and iterates the calculation of the new price towards the equilibrium price up to the given resolution £ > 0 (in WASIM s = 10-4): 1. Start at any price p > 0, e.g. the price of the previous period. 2. Calculate to price p the demand for shares Xi of agent i (see equation (10)), where xa satisfies the trading restrictions: • if (xit < 0), then xu = 0, i.e. do not allow short selling, • if (xit > maxown)^ then xu = maxown, i-e. the maximal numbers of shares an agent possesses is restricted to max own3. Calculate the number of shares to be traded.

8

Thomas Stumpert et al. Axit = {xit - Xi^t-i),

4. 5. 6. 7.

where xu denotes the number of shares agent i demands to possess and Xi^t-i is the number of shares the agent currently possesses. The agent does not sell more shares than max trader if Axit < -maxtrade^ then Axu = -maxtrade • The agent does not buy more shares than he can afford: if (Axit • p > Cit), then Axu = ^ . The agent does not buy more shares than max trade, if Axit > maxtrade, then Axu = maxtrade • Excess supply (resp. excess demand) leads to a decreasing price p with the step width Ap (resp. increasing). To improve the convergence speed, we rescaled dynamically,

N

N

where ^ Xi^t is the aggregation of Axu over all agents i and represents the excess demand (resp. supply). The higher the excess is the larger the step width gets. This leads to a new price p = poid + ^P8. If the sum of supply and demand (i.e. negative supply) is under the threshold e, then the equilibrium up to the precision e, else go to step 2. 2 Specification ofprice Testis found Criteria To analyze the influence of heterogeneous agents to the occurrence of interesting market structure, we will define in the next subsection a Herfindahl index that measures the wealth concentration. After that we divide a simulation run into equidistant market phases, in order to analyze the arising of bubbles and market crashes resp. price movements parallel to the intrinsic value of the stock. 2.1 Herfindahl Index for Measuring Market Microscopic Characteristics In the equilibrium model the price results from the aggregated expectations of the agents about the future price and dividend. Additionally, in WASIM the price of the next period is calculated dependent on the wealth of the agents. For measuring wealth concentrations, we define the wealth ratio p-u) as a Herfindahl index:

Pwit) = '-'

. _ 1

,

p^(i)e[0,i]

(13)

N N

where W/^ = 2 Wj^t denotes the cumulative wealth of all agents in t, Wu is the i=l

wealth of a single agent. Remember, that each agent chooses among 100 rules and

Time Series Properties from an Artificial Stock Market with a Walrasian Auctioneer

9

the investigation of microscopic influences on prices without the above ratio is like looking for a needle in a haystack. 2.2 Dividing a Simulation Run into IVIarket Phases A simulation run in WASIM consists of / = 150000 periods. To analyze market phases of overvaluation and undervaluation, we segmented the simulation run in J equidistant market phases of length n = 7500, where j e [1, J]. In each market phase [tj,..., tj +n], we logged the following key numbers and metrics: The average market price ptj = l/n Y^^^^. Pk (similarly the average fundamental price Ptj), the lowest market (fundamental) price Pminj (respectively Pmin,j)> ^^^ highest market (fundamental) price pmax,J (respectively pj^^^^). In order to easily evaluate the order of market price deviations from the intrinsic value of the risky asset, we defined the following relative volatility measure, Pa^j: 2 r<^^,J ~

2 2

'

where cr^(^) . = ;^^ Y^k^t^- (Pk ~Pt])'^ ^^ ^^e market (fundamental) price variance in interval j . Furthermore, we computed the average wealth ratio py^^j for market phase j and the respective maximum py^ rnax j 3^nd minimum pyj rnin j • Ari important dimension to characterize a market situation in terms of the agent's risk preferences and extreme market interactions is the deviation of the market price pt from the known fundamental price p^ = ^ . We capture the degree of over-/ undervaluation of the risky asset in a market phase by logging the relative frequency fij of market price deviation in an interval / in market phase j . For instance, if /(o.85;0.9],j = 0.15, in 15% of all trading periods in market phase j the market price was greater than 0.85 • Pi and less than or equal 0.9 • p^. Classifying market price deviation in 10 different classes, we get a histogram-like logging of the price deviation. Ntbuj, Nfbitj, ^cbitj denote the aggregated number of technical bits, fundamental bits and control (or test) bits which are marked as active in interval j (see section 1.2).

3 Empirical Results We used two different scenarios and ran each scenario with 25 experiments. Each experiment leads to different trajectories of the dividend process and the price process. Parameter configurations included a fast setting of the genetic algorithm (gamin = 200; 9^max — 300, for details see [4]), no trading restrictions (jnaxown — maxtrade = N) and dividend processes with p = 0.95. All other parameters are set as in the original Santa Fee artificial stock market (see [2]). The fast genetic algorithm guarantees that agents can easily adapt their strategy to a change in the market structure. Prohibition of trading restrictions enables single agents to build up monopolies. In the following subsections k denotes the number of the simulation run and j denotes the number of the market phase in a simulation run (j = 1,..., 20).

10

Thomas Stumpert et al.

3.1 Scenario 1: Market Characteristics without Taxes and Trading Restrictions For the first scenario we used the plain market setting without trading restrictions {maxtrade = N^TTiaXown = N) and without taxation. There are three different market conditions which can be observed in different phases (see table 2 with k — 1,2,..., 10): Overvaluation (with /(i.03,oo])» undervaluation (with /(q,o.97])> and a synchronous run of the market price with the fundamental price (with /(o.97,1.03])The sum of all / is chosen to be equal or less than 1, e.g. /(o,o.85] = 0.137 means that at least 13.7% of time periods the price pt of the risky asset deviates from p* by at least -15%. Over all simulations, overvaluation occurs more rarely than undervaluation. The reason for this is that the fundamental price reflects the valuation of a risk-neutral investor, whereas the market price is determined by supply and demand of agents who use a risk-averse utility function. One exception of this market behavior can be observed, if supply and demand would result in a negative market price (which is prohibited and set equals to zero). Figure 2 and figure 3 show the price evolvement and the wealth ratio for the simulation run with k= 1 over 150000 periods. Highfluctuationsoccur with a wealth ratio higher than 0.7. As assumed before a trend is visible that the wealth ratio increases within the simulation run because the parameter setting empowers agents to build up monopolies. Table 1 shows the calculated key ratios for the different market phases in this simulation run. Table 1. Simulation run without taxes and trading restrictions, k = 1 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

pt 99.86 84.31 99.43 105.17 87.92 98.87 98.70 103.28 96.38 99.75 100.43 95.02 94.31 100.22 96.12 98.78 99.89 103.56 102.69 97.61

Pmax

Pmin

Pt

105.92 105.91 266.42 214.35 183.10 148.83 116.98 196.98 144.40 137.38 132.07 139.31 168.51 149.16 140.78 137.46 135.42 155.02 161.37 141.82

66.64 100.15 21.77 100.28 0 100.16 0 100.28 0 99.99 0 100.08 85.65 99.71 0 99.88 0 99.56 0 99.77 72.13 100.27 0 100.00 0 100.08 11.04 100.08 0 100.30 85.16 100.06 80.15 99.93 72.65 99.87 0 100.15 0 99.73

P^ax

P'^in

P
P^

pw,max

108.16 91.06 -1.05 0.041 0.046 108.40 92.31 -70.83 0.141 0.350 108.62 92.81 -125.49 0.185 0.928 110.2193.57 -378.45 0.512 0.974 107.65 92.17 -188.10 0.749 0.998 108.89 92.07 -12.35 0.710 0.952 108.01 91.02 0.03 0.712 0.738 108.70 91.03 -95.85 0.703 0.960 107.76 91.57 -63.12 0.799 0.997 108.25 91.40 -57.52 0.804 0.977 108.43 92.81 -2.67 0.789 0.833 108.00 91.88 -30.98 0.853 0.988 109.27 91.13 -42.76 0.849 0.989 107.93 92.02 -28.93 0.833 0.987 108.73 92.93 -62.82 0.834 0.999 110.38 92.79 -0.99 0.817 0.867 109.37 89.57 -2.09 0.743 0.811 108.05 91.27 -11.10 0.725 0.771 110.41 91.35 -49.55 0.795 0.972 107.76 89.50 -44.84 0.893 0.992

piv,min 0.04 0.046 0.057 0.128 0.490 0.647 0.673 0.629 0.644 0.713 0.777 0.758 0.739 0.797 0.777 0.778 0.700 0.671 0.714 0.799

Ntbit

Nfbn

63650 77211 64999 66113 53634 59703 77173 54080 51209 50515 48603 45531 51851 56867 47887 37808 37322 38855 41407 58167

96524 74296 99280 106583 88247 76371 54297 84591 97735 79980 59933 72232 91738 101758 88656 56635 43796 29315 66268 81653

Ncbit 19700 21619 13894 19655 16595 10980 2786 3653 16928 9720 6375 11240 14967 10480 4295 6249 4542 5222 5034 13238

3.2 Scenario 2: Market Characteristics under Taxation In order to reduce high fluctuations we introduced a Tobin tax (see table 2 with k = 11, ...,20 and table 3), which significantly lead to avoidance of crashes, e.g.

Time Series Properties from an Artificial Stock Market with a Walrasian Auctioneer

11

Fig. 2. [Up] Price pt of the risky asset ioxk — I and j = 1,2,..., 20 without taxes

Fig. 3. [Down] Wealth ratio pw for /c = 1 and j = 1,2,..., 20 without taxes reduction of price volatility and a price movement of the risky asset parallel to its intrinsic value. Each agent has to pay taxes at a rate of 5% on his total wealth, i.e. the sum of the stock value and the value of the risk-free asset. The tax is payable every 100 trading periods into a tax pool. The funds are repayed in equal parts to all agents in the period following after the tax payment. Thus, the tax functions as a simple wealth redistribution system on the market. Prohibition of wealth concentrations reduces longer periods with high spreads between the market price and the fundamental price, see p^2 in table 2. Single price peaks like those observable in the scenario without taxation do not occur any longer, e.g. Pmin ^ 0. In order to verify these assumptions statistically it is necessary to use a significance test. Before using a F-test, we have to do some calculations. First of all, we want to consider the variance of the fundamental price. The conditional variance of the price process under the condition that the process is known up to the previous period is can be computed as follows:

12

Thomas Stiimpert et al. Table 2. 20 runs (without taxes for /cnotaa:=!,...»10 and with taxes for ktax^"^ 1,...,20).

k /(0,0.85] /(0.85;0.9] /(0.9;0.95] /(0.95;0.97] /(0.97;1.0] /(1.0;1.03] /(1.03;1.05] /(1.05;1.1] / ( l . l , o o ] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.137 0.112 0.175 0.141 0.098 0.116 0.089 0.146 0.116 0.093 0.088 0.191 0.157 0.117 0.208 0.217 0.105 0.159 0.227 0.274

0.027 0.061 0.051 0.043 0.030 0.036 0.041 0.045 0.069 0.036 0.166 0.222 0.216 0.235 0.248 0.219 0.195 0.209 0.257 0.275

0.055 0.100 0.071 0.087 0.063 0.078 0.061 0.120 0.103 0.081 0.236 0.231 0.233 0.319 0.202 0.224 0.237 0.225 0.202 0.188

0.056 0.073 0.061 0.076 0.088 0.104 0.057 0.111 0.116 0.085 0.115 0.091 0.102 0.099 0.064 0.072 0.098 0.092 0.062 0.048

0.287 0.188 0.270 0.293 0.337 0.365 0.360 0.273 0.367 0.282 0.241 0.170 0.208 0.151 0.163 0.181 0.218 0.194 0.156 0.130

0.220 0.160 0.191 0.216 0.219 0.175 0.254 0.158 0.143 0.192 0.135 0.083 0.077 0.062 0.106 0.081 0.122 0.094 0.089 0.079

0.048 0.059 0.032 0.028 0.024 0.028 0.035 0.031 0.016 0.028 0.012 0.005 0.003 0.007 0.005 0.003 0.013 0.014 0.003 0.003

0.055 0.084 0.044 0.028 0.037 0.032 0.039 0.042 0.020 0.037 0.005 0.003 0.001 0.005 0.000 0.001 0.007 0.007 0.000 0.000

0.110 0.159 0.100 0.084 0.099 0.061 0.059 0.071 0.045 0.162 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

Pa2

-77.39 -68.30 -52.05 -66.77 -39.59 -44.33 -53.18 -39.31 -31.91 -62.11 -3.90 -4.76 -3.50 -4.33 -3.36 -4.21 -3.85 -5.08 -3.75 -3.08

vart((it+i) = v a r t ( 4 + p{dt - d) f St+i) = erf vart(cZt+2) = vart(d + p{dt+i - d)-{- £^+2) = Y2iYt{d + p^dt + pst^i + et-^2) = (1 + P^)cr'^ = ( l + p 2 ^ ^ . . . + p2(/c-i))a2 ySir t{dt^k) with limfc^oo vart{dt+k) = E ^ o P^^^e = r = ^ ^e = var(cft). Straightforward, the variance of the fundamental price process p^ = dt/r computes to var(p*) = var(dt)/r2 = (14) • p'^r'^ and with the values used in our models we get var(p^) = 0.07429 • 0.1~^ • (1 — O-OS"^)"^ = 76.19 = ap*. Even sufficient for an interval length of n = 7500 trading periods, the empirically observed variances cr^* j match this value and therefore we assume var(p^) ^ var(p^^). Knowing the average relative volatility for each simulation runfi*omthe empirical results, p^^j, we can compute the empirical market price variance as an average value over all intervals j , with j e [ 1 , . . . , J ] , per simulation run k, with he [1,... ,K]. Since we defined 1

^

rr2

_ / T 2

1

1

"^

n-2

P,j,k

2 j=i

^P\3.k

j=zl

(15)

^P*J,k

the average empiric market price variance per simulation run k is ^p,k = (1 - Pa^,k ' J) -CTp^ •

(16)

We statistically test our assumption that market price volatility is significantly higher in simulations runs without taxation compared to a market with Tobin tax with

Time Series Properties from an Artificial Stock Market with a Walrasian Auctioneer

13

the F-test^ The values of a^j^, provided for each simulation run in table 3 with ^«^(^p,tax,/c) = 7817.09, and mm(a2^ notax k) "" 25645.55, generate a significant F-value of F = 25645.55 -r- 7817.09 = 3.28 (critical value at the 99%-confidence level is /(24;24;0.99) = 2.659). Thus, we can reject the h3^othesis that the tax system has no influence on the market price volatility. Comparing the maximum of the average empiric market price variance of all simulation runs with taxation with the minimum empiric market price variance of all simulation runs without taxation, we make sure that the market setting with taxation has significantly lower volatility in all cases compared to the market setting without taxation. Table 3. Average empiric market price variance per simulation run, afp,k k With Tax 1 2 3 4 5 6 7 8 9 10 11 12 13

6019.01 7329.48 5409.49 6674.24 5196.16 6491.39 5942.82 7817.09 5790.44 4769.49 5485.68 7451.38 5759.96

N o Tax 118003.07 104151.73 79389.98 101820.32 60403.43 67626.24 81111.87 59976.77 48700.65 94719.41 152669.52 46369.23 55847.27

k With Tax 14 15 16 17 18 19 20 21 22 23 24 25

4845.68 4739.02 5881.87 4982.83 4601.88 4921.87 6613.29 5836.15 6918.05 5196.16 4739.02 7695.19

N o Tax 132235.36 69911.94 72228.12 117911.64 77576.66 80197.59 50803.49 25645.55 52342.53 59428.20 58833.92 122269.71

4 Conclusion We presented a model based on SF-ASM where the amount of each agent's wealth influences the future price. In this model high fluclxiations occur due to extreme concentrations of wealth and stock shares. We used a parameter setting where agents can easily adapt their strategy to a change in the price function of the risky asset. ^ Commonly used in order to prove that the probability distribution fijnctions of two data sets have significantly different variances is Fisher's "F-tesf. The Test statistic is with (Ji > a i . Hypothesis HQ, i.e. af and al have the same variance a^, can be rejected at confidence level a if F > f(,.^-u2,i-a), where /(^^ ;i/2;i-a) dcnotcs the (1 — a) quantile with Ui = 1 — rii degrees of freedom at m observations.

14

Thomas Stumpert et al.

Prohibition of trading restrictions empowers agents to possess and trade all shares each period. The interaction of the auctioneer with each single agent is reflexive and without any trading restrictions high fluctuations occur in our model. The high fluctuations depend less on the use of technical trading bits but on an arising extreme wealth ratio. By introducing a Tobin tax this effect can be avoided, the tax leads to market stabilization reducing the occurrence of longer periods with high spreads between the market price and the fundamental price.

References 1. LeBaron B, Arthur WB, Palmer R, Holland JH, Tayler P (1997) Asset pricing under endogenous expectations in an artificial stock market. In: Arthur WB, Durlauf S, Lane D (eds) The economy as an evolving complex system II. Addison-Wesley 2. LeBaron B, Arthur WB, Palmer R (1999) Time series properties of an artificial stock market. Journal of Economic Dynamics and Control 23:1487-1516 3. Tesfatsion L (2002) Notes on the Santa Fe artificial stock market model. Econ 308x: Agent-based Computational Economics 4. Seese D, Stumpert T (2003) Influence of heterogeneous agents on market structure in an artificial stock market. Proceedings of Annual Workshop on Economics with Heterogeneous Interacting Agents, WEHIA 2003, Kiel 5. LeBaron B (2002) Building the Santa Fe Artificial Stock Market. Working Paper, Brandeis University 6. Ehrentreich N (2003) A corrected version of the Santa Fe institute artificial stock market model. Complexity 2003: Second Workshop of the Society for Computational Economics. 7. Cont R (2001). Emperical properties of asset returns: Styhzed facts and statistical issues. Quantitative Finance Volume 1:223-236 8. Walras L (1874) Elements d'economie pohtique pure. [English Translation: Elements of Pure Economics of the Theory of Social Wealth. Irwin RD, Homewood, 1926] Corbaz L (eds) Lausanne 9. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley

Market Dynamics and Agents Behaviors: a Computational Approach Julien Derveeuw^ Laboratoire d'lnformatique Fondamentale de Lille - UMR USTL/CNRS 8022 Universite des Sciences et Technologies de Lille Batiment M3, Bureau 14a 59655 Villeneuve d'Ascq Cedex [email protected]://www.lif1.fr/~derveeuw Summary. We explore market dynamics generated by the Santa-Fe Artificial Stock Market model. It allows to study how agents adapt themselves to a market dynamic without knowing its generation process. It was shown by Arthur and LeBaron, with the help of computer experiments, that agents in bounded rationality can make a rational global behavior emerge in this context. In the original model, agents do not ground their decision on an economic logic. Hence, we modify indicators used by agents to watch the market to give them more economic rationality. This leads us to divide agents in two groups: fundamentalists agents, who watch the market with classic economic indicators and speculator agents, who watch the market with technical indicators. This split allows us to study the influence of individual agents behaviors on global price dynamics. In this article, we show with the help of computational simulations that these two types of agents can generate classical market dynamics as well as perturbed ones (bubbles and kraches).

1 Introduction Market simulations with the help of computer agents have become in the last few years a growing field of interest under the impulsion of the Santa-Fe Institute for example. These simulations allow to predict market evolutions, to validate theoretical hypotheses or to test models in perfectly controlled virtual worlds [8]. The most used approach to study these complex systems is the use of agents with bounded rationality who learn and make their behaviors evolve in time. Following the founding works of [1] or [10], who showed that it is possible to make rational global behaviors emerge with simple, bounded individual behaviors, numerous models of markets have been developped. These models aim to reproduce real economic phenomena (for example, bubbles and crashes: [13]) or to study the impact of these phenomena on the agents population [5]. The early version of the Santa-Fe Artificial Stock Market (also known as SF-ASM) by [15] remains a major reference in this field: it shows that, a global, rational economic behavior can emerge from an agents population that build its be-

16

Julien Derveeuw

havior on past events, learning and evolution, which is not a commonly admitted result in standard economic theory. The SF-ASM principle is very simple: agents hold a certain amount of stocks. These assets have a current price and pay at each iteration a dividend. Agents have to take a decision: to invest their cash in new shares or to sell the ones they hold to get their money back. As the price dynamics varies upon thefluctuationsin demand and supply, each decision taken by the agents directly impacts the price motion. It has been modified several times to correct some specific aspects of the model (for example the genetic algorithm which allows the agents to make their behavior evolve in time [4], [6]) or to add more realism to the underlying economic logic of the model [11], [12]. Though, these modifications remain minor technical corrections. We develop our model considering two major modifications of the original SF-ASM: the first improves the agents behavior by putting a stronger economic rationality in their decisions. There, we define two canonical subpopulations: fundamentalists and speculators. Thus, the second modification consists in mixing those subpopulations to observe and characterize some interesting global market dynamics. In this article, we want to address the question of markets dynamics, bubbles and crashes using a bottom-up approach. We show that critical events may be caused by bounded rationality individuals that ground their behaviors on market trends and liquidity signals. Our results are consistant with the general thesis of [9] and the french neo-keynesian school offinanceleaded by [14]. According to those last approaches, critical events are caused by the interaction of rational investors that do not arbitrate prices considering a so-caUQd fundamental value but that try to obtain profits in catching the market mood. If they trust the market will raise despite it is yet overevaluated, they will have a global buying attitude that will therefore push the prices up. The main issue they face is that the market cannot offer enough liquidity if all the agents perform the same decision: thus, if all of them want to sell at the same time, the market breaks down abruptly^. The article is organized as follows: a first part presents the architecture of the original model of the SF-ASM as well as some minor modifications we bring to it. The second part details the learning process used by the agents to make their decisions and the differences between our two subpopulations of agents, the fiindamentalists and the speculators. The last part presents our results and discusses some consequences of this research.

2 Market Model Presentation Our work is directly based on the articles of [15] and [12]. Let us show common parts as well as differences between the original SF-ASM and our modifications.

^ as instance, one can report to the tulipomania bubble which occured in Netherlands

Market Dynamics and Agents Behaviors: a Computational Approach

17

2.1 The Original Model of the SF-ASM The model architecture is reduced to the essential: it is composed of one type of stock (5) and of n heterogeneous agents a i , . . . , a^: they all have a different behavior. Each step of time can be considered as a market day. Agents do not know at which iteration the simulation will stop. At each step of time t, the stock has a current price pt and pays a dividend dt per asset to each stockholder. Each agent a^ owns a certain amount of money rui^t, and a number of shares hi^t- Their goal is to choose between keeping their shares to earn the dividend dt, to sell them to raise their funds or to buy new ones. Another possibility for agents is to invest their cash money in a risk free asset which pays a moderated interest rate r at each step of time t. At the end of each time period, agents are asked their desires: they can either bid to buy new shares (in this case, we have a bid: bi^t = 1 and no stock offered: Oi^t = 0), offer a share (bi^t = 0 and oi^t = 1) or do nothing (bi^t = 0 and Oi^t = 0). We then obtain the cumulated offer (Ot) and supply (Bt) by summing the bi^t and Oi^t' The balance between cumulated offer and supply has a direct influence on the stock price and on the quantities exchanged by the agents. If Bt = Ot, then all offers and demands are satisfied: each agent who asked for a share receives it and each agent who offered a share sells it. For remaining cases, we have to introduce a process to distribute offered shares function of the number of asks: it is a market clearing process. Each agent who asked for a share is given the maximum fraction of share available (offered): iii,t-\-i = i^i,t H

mm{Bt,Ot), , 5 ^i,t H J=>t

mm[Bt,Ot) 7^; Oi^t Ut

One can notice that bi^t = 1 do not mean that agent ai will receive a complete share at t + 1, but that he will receive at most a share. Hence, bi^t must be seen as a proposal to buy the maximum fraction of share available. At the end of each time step, the price is updated function to the offer and supply rule (the more the share is asked, the more its price raise) using the following formula:

Pt+i=Pt{l +

rj{Bt-Ot))

Tj is 3. parameter which controls the impact speed of the offer and supply on the stock price. Another interesting value is the fundamental value of the stock. This value is totally virtual: it has no real existence in the model. Though, it allows to determinate the stock price in an ideal market, which allows to know if the stock is overpriced or underpriced at a given time. It is computed by: fvt = ^ c Zj,ty± t ( l -r + uc) a)" t=0 a is usually considered as equal to r. This equation is hence simplified as: fvt = — r

18

Julien Derveeuw

2.2 Modifications on the Original Model We have seen that at each step of time, agents receive a dividend dt per each share. In the original model of [15], the dividend generation process is relatively complex. We have chosen to simplify it by choosing a well known generation process in economics for the generation of random process poorly evolving in time: the random walk [16]. A random walk is defined as follow: dt+i = dt-{-et

et is a gaussian noise parametered by its mean (null here) and its variance (S'^).

3 Agents Reasoning Process As the agents do not know the generation process of dividend d and of price p, they are forced to elaborate their strategies only with their experiences (the past values of dividend and prices). Their challenge is to maximize their satisfaction (here, maximize the amount of money won at the end of the game). This is seen here as trying to recognize a particular market state to take the best decision function of this state. We will describe in a first part which representation the agents use to observe the market, and in a second part how this representation is used to take a decision. 3.1 Market Representation Each agent has a stock of m rules which describe market states and tell him which decision to take. A rule is composed of three subparts: the first one describes a specific market state (called condition part of the rule). The second part describes the decision to take withing this specific context: bid a share, offer a share or do nothing (called action part. The last part represents the current evaluation of the rule's adequateness in market activity (called^brc^). One have to keep in mind that each agent possess his own stock of rule that is, it is hardly possible that two agents are exactly similar. The /c-th rule of an agent ai is composed of: 1. a condition part that can be viewed as a 7 bit chromosome or a string made of 7 {0,1,#} symbols. Each gene or character contributes to the description of a specific market state that is therefore, completely expressed, with 7 statements. Those statements are said to be true (the value of the gene is 1), false (0) or unrelevant (#). The space of conditions is hence 3"^ size. To give an idea of what a statement is, one can consider the following: Stock price is over 200$ 2. an action ai^k to take if the rule is selected. We have: aik = 1 "^ bid one share, a^fc = — 1 <^ offer one share and aik = 0 ^ do nothing 3. a strength Si^k which tells how good this rule was in making the agent earning money in the past. At t = 0, the rules are generated following those steps:

Market Dynamics and Agents Behaviors: a Computational Approach

19

1. The condition part is randomly built. 2. The corresponding action is determined following a rational process which will be explained further. 3. Initial strengths are 0. When t > 0, the rules are generated, evaluated and updated using a genetic algorithm. The genetic algorithm maintains diversity in the rules population, improves them and allows to easily destroy the worst ones. Let us now focus on the agents' decision process. Since the agents have to perform the best possible choice, they first identify in their stocks of rules which of them correctly describe the current market state. These rules are said to be activated. In other words, the chromosomes are matched against the current market state and a rule is said to be selectable if all of its bits (genes) are non contradictory with this state (they are said to be activated). Hence, a rule is activated if all of its bits are activated too. A bit hi is activated: 1. 6i = 0 and the z-th condition of the market state is f a l s e . 2. hi = 1 and the i-th condition of the market state is t r u e . 3. hi = # and the i-th condition of the market state is either t r u e of f a l s e . Among those activated rules some of them present a positive strength Si^k > 0. He then elects one of these rules with a random process proportional to their strength. The action aik associated with the elected rule gives the agent decision. If there are no rules activated by the current market state, then the agent's decision is to stay unchanged {hi{t) = 0 and Oi{t) = 0). At the end of the time step, the agent updates the previously activated rules according to how much money they would have make him earn, giving: Sik{t + 1) = (1 - c)sik{t) + caik{p{t + ! ) - ( ! + r)p{t) + d{t + 1)) The parameter c controls the speed at which the rules strength is updated. Each time the genetic algorithm is run, the worst rules (e.g. with the smallest strength) are deleted. They are replaced by new rules generated using a classical genetic process: the best rules are selected to be the parents' of the new rules. A new rule can be generated either by mutation (only a bit of the parent's chromosome is changed) or by a crossover process (reproduction between two parents rules). This mechanism permits, on the one hand, to delete the rules that don't make our agent earn money and to build up new rules using good genetical material. This process is aimed to increase the adaptation of the agents to the market activity. There are two types of agents in our simulations: some who try to be as close as possible to the fundamental value of the stock (will be refered as fundamentalist agents in the following) and some who try to make the maximum benefit without taking care of the fundamental value (will be refered as speculators agents in the following). This is a point that largely make our work different from those previously cited. We think that in [3] and [7] one issue is that the decision rules of agents are excessively dominated by randomness: whatever the market statements are, the corresponding action is decided randomly. It is true that along market activity, the evolving process selects best responses to those statements, but nothing grants that

20

Julien Derveeuw

the corresponding actions are relevant with respect to an economic logic. For example, it is very probable that although a stock is mispriced (let's say underevaluated), the agents will never try to arbitrate this spread (here with buying it). The other issue is that technical statements as well as fundamental statements are melted and no typical behavior is clearly observable. We try to improve the agent model by defining a minimum economic logic that leads each subpopulation actions: fundamentalists try to arbitrate any price deviation whereas speculators ground their decisions on subjective, technical informations. As said before, the main characteristic of the fundamentalist agents is that they have appropriate decisions considering the spread between the observed prices and the fundamental value. Let's consider the composition of the chrosome and what kind of statements are coded inside. Bit Market Indicator 1 VtjvU > 0.2 2 Pt/vft > 0.4 3 pt/vft > 0.6 4 Pt/vft >0.S 5 vft 6 ^ > 1.2 vft 7 ^>l-4 8 ^>l-6 9 ^>l-8 10 ^>2.0

Corresponding action Pt/vft e 1 =^ to buy [0.0,0.8] - 1 =^ to sell [1.2,2.0] [0.0,7 ] 7 > 0 . 8 0 =^ stay unchanged [7 , 2.0] 7 < 1.2 0 =^ stay unchanged (ibid.) [0.0,2.0] 0 => stay unchanged (ibid.)

Fig. 1. Fundamentalists' chromosome

Fig. 2. Rules for fundamentalist rationalization

Let's consider the seventh gene; the corresponding statement, depending on its wahxQ {l, 0, ^} is: The price {is, i s n o t , i s o r i s not} at least forty percent above the fundamental value We have added to the original SASM a rationalize procedure. This procedure aims to achieve a minimal economic rationality for the agents. Fundamentalists are assumed to arbitrate significant spreads between fv and p, that is to bid for underpriced shares and to ask for overpriced stock This procedure is based on some rules presented in table 3.1. One has to keep in mind that this procedure is run each time a new rule is generated (consequently, when the genetic algorithm is initialized and run). Let's consider now the second subpopulation: the speculator agents. As said before, those agents do not arbitrate prices but rather try to make profit using trends or subjective knowledges. Therefore, their chromosome is constructed using this kind of market representations as shown in table 1. The chromosome is thought to code general sentiment on the market trend which is very different than the identification of a market state. What we mean here is that

Market Dynamics and Agents Behaviors: a Computational Approach

21

Table 1. Speculators' chromosome Bit

Market Indicator

1 2 3 4 5 6 7 8 9

Pt > Pt-i Pt > Pt-2

pt>i/^xj:iZt-iPi

pt>i/ioxj:iz',\pi pt>i/mxj:triPi p t > 1/250 x E •="!>* Pt > l/2[Minpi Pt > l/2[Minpi Pt > l/2[Minpi

+ + +

Maxpi i e [ t - i , t - i o ] Maxpi i 6 [ t - l , t - 1 0 0 ] Maxpi i e [ t - l , t - 2 5 0 ]

this trend is supposed to constraint the attitudes of the agents that wants to exploit it, not with an arbitrage strategy but rather in following it. Hence, if the general sentiment is bull market a rational behavior for a speculator agent is to buy. (symetrically, if the market is bear, the rational behavior is to sell). We have coded this logic in the speculator rationalization. To have a global sentiment on the market trend, we simply appreciate the dominant trend given by the indicators or groups of indicators. The decision making process for speculator agents is relatively complex and can be divided into two major steps. For bits 1, 2, 7, 8 and 9, we simply consider if the belief of the agent validates the condition or not. Let's consider the example of bit 8: we explicitly test if the price is over or above the median of the interval bounded by the highest and the lowest quotation during the lasts 100 days. If the price is above, it is thought that the price will decrease and alternatively, if it under this median, it is believed that the price will rise. As instance, this last situation pushes the agent to bid new shares. Bits 3 to 6 receive a special treatment: bits 3 and 4 are considered together as well as bits 5 and 6. The first pair allows the estimation of short range trend while the second pair allows the estimation of a long range trend, pairs, hiU is the first one e.g. bit number 3 and bit number 5 while 6^+1 is the second one e.g. 4 and 6. To appreciate the trend, one has to consider the situation of the current price relatively to those bits. As example, let's consider the situation where the chromosome's bits 3 and 4 are respectively 0 and 1. In this case, it is false to assert that the current price is above the mobile average on the past five days whereas it is clearly above the mobile average on the past ten days. We therefore consider that this information is not sufficiently clear to influence the decision and bid and ask positions have to be weighted with the same absolute value scalar: 0.5. When those bits are respectively 1 and 1, the trend is clearly bull and the agents will be temptated to follow it, e.g. to buy. The nine possibilities for each pair are summed up in table 2. Afirststep in the speculators' rationalization process is then achieved: our agent can form an initial belief on the possible tendency of the market summing the values of each indicator. One has to keep in mind that some of them are positive (giving bid signals) negative (ask signals) or null (do nothing). If the number of positive

22

Julien Derveeuw Table 2. Speculators' rationalization when i G {3,5} 0 10# 1 ##0 1 10# 1 0 0 1## Partial rationalization 1 -1 0 {0:5,-"0:5}|{0.5,-0.5} -1 1 -1 1 biti biti+i

signals is dominating, the initial belief will be that the price will probably rise and the corresponding behavior will be to bid. Symmetrically dominating negative signals lead to ask and null signals lead to stay unchanged. One can easily imagine that such a logic may lead to constantly growing or falling markets: bear signals are followed by bid positions that push the price up. Why this tendency should break down ? According to [14], one major indicator observed by the traders is market liquidity. The idea is that operators are very concerned with the possibility of clearing their positions (to sell when they hold stocks or to buy if they are short). This implies that minimum volumes are realized at each time step. When the market becomes illiquid, agents may be stucked with their shares. Therefore, they follow the market only and only if they are confident on the liquidity level of the market. This point has been included in the agents' logic with the following rules: • •

each agent has her own treshold above which she considers that the market is unsufficiently liquid to clear her positions. When this threshold is reached, she adopts a position opposite to the one she would have adopted without considering this treshold. By the way, she decides to reverse her investment strategy to go out of the market.

4 Experimental Schedule and Results As the model contains many numerical parameters, we have chosen to only vary the ones which directly impacts the global price dynamics, that is to say the speculator agents proportion and their liquidity fear parameters. The other ones are considered as constant as they can be seen as more technical model parameters. All of the following experiments are realized on a time range of 10000 iterations. Though, as the genetic algorithm used by the agents to adapt themselves to the market needs a learning period, only iterations between time step 2000 and 10000 are shown. All statistics are conducted on this range unless the opposite is mentionned. As our primarly goal is to study the influence of speculator agents' proportion on price dynamics, we first run an experiment without speculators (i.e. only with fimdamentalists). Thisfirstexperiment allows us to validate our fundamentalist agent model by matching the experimental results with the ones obtained by [15] with the original SF-ASM model. This experiment will also be used as a comparison base with other ones as it represents the baseline price dynamics of our model (i.e. with the less variant price series). Other experiments are realized by gradually increasing the speculator agents proportion in the agents population and by adjusting their liquidity

Market Dynamics and Agents Behaviors: a Computational Approach

23

fear. Many experiments have been run, but we only detail here the ones with the more significant results.

4.1 A Fundamentalist Market The figure 3 represents the price and fundamental value motions when the market is only made of fiindamentalist agents. The two series perfectly overlap.

\yvM Fig. 3. Market dynamics with fundamentalist agents

The first step to test if those motions are somewhat consistent with what happens in the real stock markets consist in testing whether they are driven by non-stationary processes or not. The appropriate test to seek for a random-walk process in market returns is an Augmented Dickey-Fuller unit root test (e.g. ADF). Both fundamental values and prices have to be random walks if we want to qualify the simulations realistic since the immense part of academic researchs attest such motions for modem, real stock market dynamics. In the following tests, the null hypothesis is time seriepresents one unit root (HQ) while the alternative is time serie has no unit root (Hi). Table 3 reports the results of those tests. Interpretation is the following: if t-Statistics is less than the critical value, one can reject the HQ against the one-sided alternative Hi which is not possible in our case. Table 3. ADF Unit Root Tests. time series t-Statistics Prob.* Augmented Dickey-Fuller test fund. val. -2.3594 0.4010 Augmented Dickey-Fuller test price -2.4154 0.3713 Critical values: 1% level: -3.9591, 5% level:-3.4103, 10%: -3.1269 *MacKinnon one-sided p-values.

24

Julien Derveeuw

A Johansen co-integration test shows that prices and ftindamental values coevolve. We also observe that the spread between prices and fundamental values remains very weak (between -3.22% and +2.72% with a 0.02% mean and a 1.11%) standard deviation). This base line experiment exhibits therefore some interesting results if one considers its proximity with real market dynamics. It also shows that bounded rationality agents can make emerge a random walk motion that is characteristic of efficient prices on stock markets. This result is already documented by [15],[2]. Nevertheless, our contribution is to obtain such results with agents following rules that make sense, which was less evident in the original studies. 4.2 A Mixed Market

Figure 4 represents price and fundamental value motions when the market is made of 25% of fundamentalist agents and 75% of speculators.

Fig. 4. Market dynamics with 25% fundamentalist and 75% speculators It appears that the market is more volatile when it is flooded with fundamentalists which is an expected result. If one considers the statistical properties of the price motion globally (on the complete sample), it appears that a Null hypotheses of random walk can be rejected with a very low risk (with p < 3%). This result is understandable as the agents population is composed of a majority of speculators. Though, on smaller samples (for example on time range from 2000 to 3000), the result of the test is inverted: the market is in a period where it behaves as if it follows a random walk. In such periods, the price and the fundamental value motion are co-integrated, which shows that market follows the fundamental value dynamics. In Table 4, we have reported some basic statistics related the spreads between observed prices and fundamental values. It clearly appears that prices are much more volatile in the second regime (with speculators) than in the first one (standard, maximum and minimum deviations). The over-returns mean is also strictly positive. Moreover, returns distribution does not follow a Normal distribution.

Market Dynamics and Agents Behaviors: a Computational Approach

25

Table 4. Prices deviations relatively to the fundamental value Speculators prop. Mean Median Std. Dev. Skewness Kurtosis

Global sample 0% 75% 0.032079 2.135464 0.116219 1.259602 2.067116 3.809237 -0.200899 1.191886 2.421358 4.609048

Critical sub-sample 0% 75% 0.152548 3.180839 0.150780 1.450169 2.114522 5.535200 -0.230643 1.228508 2.236049 3.489872

On a critical period where we can visually identify a bubble, for example during time period 5000-5400, prices are still a random walk. Table 4 reports prices deviations during this critical event. Here the standard deviation is greater than the one observed on the complete sample. A bubble is hence characterized by a great deviation between the stock price and its fundamental value during a long time range. This typical dynamic, obtained with 75% of speculators and 25% of fundamentalists, can be found with other sets of parameters as long as speculator agents proportion is great (> 70%). In the speculative regime (when speculators compose the main part of population), we obtain a highly volatile price dynamic with bubbles and crashes. These phenomena would rather be undetectable if we could not watch the fundamental value. Moreover, as the prices follow most of the time a random walk, nothing can distinguish such a dynamic from the one observed with a fundamentalist population except the comparison between the prices and the fundamental value. Hence, there could be speculative bubbles in real market while the technical efiiency properties would be respected.

5 Conclusion In our simulations, we obtain price dynamics specific to our two agents populations. These behaviors were designed to illustrate two main economic logic: the first follows the classical economic theory which is grounded on agents arbitrating differences between the fundamental values and the current stock prices, whereas the second is mainly based on ideas from the keynesian theory of speculation. The first market dynamics is obtained when the agents population is only composed of fundamentalists. We show that in this case, the price dynamics follows a random walk which co-evolve with the fundamental values. This first result can be related to the ones of [15]: inductive agents in bounded rationality can make efficient prices emerge. The difference here is that fundamentalists only ground their decisions on classic market indicators and that these decisions are made following constitent behavioral rules, which is not the case in many simulated stock markets. When speculator agents compose the main part of the agents population, we obtain another type of dynamics: prices still follow a random walk process, but during

26

Julien Derveeuw

some periods, the system reaches a critical state. This critical state is characterized by the emergence of a new phenomenom: the stock starts to be more and more overpriced (bubble) before falling back violently to its fundamental value (crash). Moreover, these market dynamics are very volatile. Next steps in our research could be to introduce a third agent behavior which will act as a market regulator to arbitrate the market and prevent bubbles from happening. This could for example be realized by introducing a behavior who would ponctually decrease the market liquidity to force the speculators to reverse their decisions. One can also imagine to study the impact of social interaction between agents on market dynamics to see if it would arbitrate the price deviations or amplify them.

References 1. B. Arthur. Inductive reasoning and bounded rationality : the el-farol problem. American Economic Review, S4A06-4n, 1994. 2. B. Arthur. Inductive reasoning and bounded rationality : the el-farol problem. American Economic Review, S4'A06-411, 1994. 3. B.W. Arthur, J.H. Holland, B. LeBaron, R.G. Palmer, and P. Tayler. Asset pricing under endogeneous expectations in an artificial stock market. In D. Lane B.W. Arthur and S.N. Durlauf, editors. The Economy as an Evolving Complex System II, pages 15-44, 1997. 4. N. Ehrentreich. A corrected version of the santa fe institute artificial stock market model. Working Paper, Martin Luther Universitat, Dept of Banking and Finance, HalleWittenberg (Germany), September 2003. 5. S. Focardi, S. Cincotti, and M. Marchesi. Self-organization and market crashes. Journal ofEconomic Behavior and Organization, 49(2):241-267, 2002. 6. L. Gulyas, B. Adamcsek, and A. Kiss. An early agent-based stock market : Replicaton and participation. Proceedings of the NEU 2003, 2003. 7. L. Gulyas, B. Adamcsek, and A. Kiss. An early agent-based stock market : Replicaton and participation. Proceedings of the NEU 2003, 2003. 8. N.F. Johnson, D. Lamper, P. Jeffries, M.L. Hart, and S. Howison. Application of multiagent games to the prediction offinancialtime-series. Oxford Financial Research Centre Working Papers Series N° 2001mf04., 2001. 9. J. M. Keynes. The General Theory of Interest, Employment and Money. MacMillan, London, 1936. 10. B. LeBaron. Experiments in evolutionnary finance. Working Paper, University of Wisconsin - Madison, August 1995. 11. B. LeBaron. Evolution and time horizons in an agent based stock market. Macroeconomic Dynamics, 5(2):225-254, 2001. 12. B. LeBaron. Building the santa fe artificial stock market. Working Paper, Brandeis University, June 2002. 13. H. Levy, M. Levy, and S. Solomon. A microscopic model of the stock market: Cycles, booms, and crashes. Economic Letters, A5{\)'A()2>-\\\, May 1994. 14. A. OrlQan. Le pouvoir de la finance. 1999. 15. R.G. Palmer, W.B. Arthur, J.H. Holland, B. LeBaron, and P. Tayler. Artificial economic life : A simple model of a stockmarket. Physica D, 15:264-21A, 1994. 16. P.A. Samuelson. Proof that properly anticipated prices fluctuate randomly. Industrial Management Review, (6):41-49, 1965.

Traders Imprint Themselves by Adaptively Updating their Own Avatar Gilles Daniel^, Lev Muchnik^, and Sorin Solomon^ ^ School of Computer Science, University of Manchester, UK [email protected] ^ Department of Physics, Bar Ilan University, Ramat Gan, Israel [email protected] ^ Racah Institute of Physics, Hebrew University of Jerusalem and Lagrange Laboratory for Excellence in Complexity, ISI Foundation, Torino [email protected] i.ac.il

Simulations of artificial stock markets were considered as early as 1964 [20] and multi-agent ones were introduced as early as 1989 [10]. Starting the early 90's [18, 13, 21], collaborations of economists and physicists produced increasingly realistic simulation platforms. Currently, the market stylized facts are easily reproduced and one has now to address the realistic details of the Market Microstructure and of the Traders Behaviour. This calls for new methods and tools capable of bridging smoothly between simulations and experiments in economics. We propose here the following Avatar-Based Method (ABM). The subjects implement and maintain their Avatars (programs encoding their personal decision making procedures) on NatLab, a market simulation platform. Once these procedures are fed in a computer edible format, they can be operationally used as such without the need for belabouring, interpreting or conceptualising them. Thus ABM shortcircuits the usual behavioural economics experiments that search for the psychological mechanisms underlying the subjects behaviour. Finally, ABM maintains a level of objectivity close to the classical behaviourism while extending its scope to subjects' decision making mechanisms. We report on experiments where Avatars designed and maintained by humans from different backgrounds (including real traders) compete in a continuous doubleauction market. Instead of viewing this as a collectively authored computer simulation, we consider it rather as a new type of computer aided experiment. Indeed we consider the Avatars as a medium on which the subjects can imprint and refine interactively representations of their internal decision making processes. Avatars can be objectively validated (as carriers of a faithful replica of the subject decision making process) by comparing their actions with the ones that the subjects would take in similar situations. We hope this unbiased way of capturing the adaptive evolution of real subjects behaviour may lead to a new kind of behavioural economics experiments with a high degree of reliability, analysability and reproducibility.

28

Gilles Daniel et al.

1 Introduction In the last decade, generic stylized facts were reproduced with very simple agents by a wide range of models [3, 12, 14, 6, 16, 8]. By the very nature of their generic properties, those models teach us little on real particular effects taking place as result of real particular conditions within the market. In order to understand such specific market phenomena, one may need to go beyond "simple-stupid" traders behaviour [1]. Thus the task of the present generation of models is to describe and explain the observed collective market phenomena in terms of the actual behaviour of the individuals. For a long while, classical economics assumed individuals were homogeneous and behaved rationally. Thus it was not necessary to study real people behaviour since (presumably) there is only one way to be rational. Even after the conditions of rationality and homogeneity were relaxed, many models did it by postulating arbitrary departures not necessarily based on actual experiments. When the connection to the real subjects behaviour was considered [11], an entire host of puzzles and paradoxes appeared even in the simplest artificial (laboratory) conditions. Thus the inclusion of real trader behaviour in the next generation of models and simulations is hampered by the inexistence of comprehensive, systematic, reliable data. Given the present state of the art in psychological experiments, where even the behaviour of single subjects is difficult to assess, we are lead to look for alternative ways to elicit the necessary input for agent-based market modelling. In this paper we propose a way out of this impasse. Rather than considering the computer as a passive receiver of the behavioural information elicited by psychological experiments, we use the computer itself as an instrument to extract some of the missing information. More precisely, we ask the subjects to write and update adaptively, between simulation runs (or virtual trading sessions) their own avatars. By gradual corrections, those avatars converge to satisfactory representations of the subjects' behaviour, in situations created by their own collective co-evolution. The fact that the co-evolution takes place through the intermediary of the avatars interaction provides an objective detailed documentation of the process. More important, the dialogue with the avatars, their actions and their collective consequences assist the subjects in expressing in a more and more precise way their take on the evolving situation and validate the avatar as an expression of the subject internal decision mechanisms. Ultimately, the avatar becomes the objective repository of the subject's decision making process. Thus we extend, with the help of computers, the behaviorist realm of objectivity to a new area of decision making d3mamics. The classical behaviourism limits legitimate research access to external overt behaviour, restraining its scope to the external effects produced by a putative mental dynamics. The method above enables us to study the subjects decision making dynamics without relying on ambiguous records of overt subjects behaviour nor on subjective introspective records of their mental state and motivations. Far from invalidating the psychological experimental framework, the present method offers psychological experiments a wide new source of information in probing humans mind. The competitive ego-engaging character of the realistic NatLab

Traders Imprint Themselves by Adaptively Updating their Own Avatar

29

market platform [17] puts humans in very interesting, authentic and revealing situations in a well controlled and documented environment. Thus standard psychological techniques can exploit it e.g. by interviewing the subjects before and after their updated strategies are applied and succeed (or fail!).

2 Avatars The program sketched in the previous section suggests a Behavioural Finance viewpoint, in a realistic simulation framework. More precisely, the avatars acting in such an environment are able to elicit from the subjects operationally precise and arbitrarily refined descriptions of their decision processes. In particular, by analysing the successive avatar versions that passed the validation of its owner, one can learn how the owner behaves in this market environment, how (s)he designs his/her strategies, how (s)he decides to depart from them, how (s)he updates them iteratively, etc. Thus the new environment acquires a mixed computational and experimental laboratory character. In this respect, the present study owes to previous research that involved simulations / experiments combining human beings and artificial agents, in real-time [5] or off-line [15, 2] - see [7] for a review of computational vs experimental laboratories. The heart of the new simulation-experimentation platform is the co-evolving set of Avatars. They constitute both the interacting actors and the medium for recording the chronicles of the emergent collective dynamics of the subjects. As a medium for capturing cognitive behaviour, the avatars help extend the behaviorist objectivity criteria to processes that until now would be considered as off-limits. We are achieving it by trying to elicit from humans operational instructions for reaching decisions that they want implemented by their market representatives - the avatars. There is an important twist in this procedure: we are not trying to obtain from the subjects reports of their internal state of mind and its evolution; we are just eliciting instructions for objective actions in specific circumstances. They are however formulated in terms of conditional clauses that capture users intentionality, evaluations, preferences and internal logics. 2.1 Principle

At the beginning of a run, every participant designs his own avatar which is used as a basis to generate an entire family of artificial agents whose individuality is expressed by various (may be stochastically generated) values of their parameters. The resulting set of artificial agents compete against each other in our market environment; see Fig. 1. We use many instances, rather than a single instance of the avatar for each subject, for the following reasons: • •

having a realistic number of traders that carry a certain strategy, trading policy or behaviour profile having enough statistics on the performance of each avatar and information on the actual distribution of this performance

30

Gilles Daniel et al.

•

•

!

-

h « « « «

?-

p 7^"^

•

g

P P P P

h^

H n N 1

' Human subject

-

Avatar

• • • •

^

^^"^^rder^^

Population of artificial agents

Fig. 1. The Avatar-Based Method: Human subjects design their avatar in the NatLab environment. From each avatar, a family of artificial agents is generated and included in the market. Once the population of agents is generated, a first simulation run is performed. A typical run lasts about 10 minutes of CPU , which may represent years of trading on the artificial time scale. At the end of each run, the results are processed and presented to the participants. In our experiments until now, both private and public information were made available. In particular, the price (and volume) trajectory, the (relative) individuals wealth in terms of cash holdings, stock holdings, and their evolution were publicly displayed. The avatar codes were also disclosed and the participants were asked to describe publicly their strategy and the design of their avatar. After being presented with the results (whether full or only public information) of the previous run, the participants are allowed to modify their own avatar and submit an upgraded version for the next run, as described in Fig. 2. The goal of this iterative process, co-evolving subjects thinking with computer simulations, is to converge in two respects; the subject understands better and better: • •

the consequences of his/her own strategy how to get the avatars to execute it faithfully

2.2 Comparison Between Approaches In this section, we discuss the relevance of our method in the context of other works in economics. The economics field spans a wide range of fields and approaches. In the table displayed in Fig. 3, the four rows classify the activities in terms of their context and environment, starting with the DESK at the bottom of the table, extending it to the use of computers, then to the laboratory and ultimately to the real unstructured world.

Traders Imprint Themselves by Adaptively Updating their Own Avatar

31

Facilitator

Participant

_1^_ ^ ^ " ^ D e s i g n/l m prove^N ^^^^^^ Avatar ^ ^

^ V^^

Submit Avatar

^ ^ ^ ^

^(^^^^^^^CoW^a V ^ ^ Avatars

i ^ V^^

Generate families

^

^

^ N ^ ^

Learning

& (

Adaptation

Run simulation

)

/ ^ ^ Post-process ^ N V ^ ^ data ^ ^

/^Explain publiclyN^ V o w n avatar d e s i g n ^

^"^^ V ^ ^

Display data

^

^ ^ ^

Fig. 2. Iterative process: participants design and improve their avatar in between every simulation run

CLASSICAL METHODS Wide Wild World

Field Studies Econometrics Behavioral Economics WWW based large scale experiments

AVATAR-BASED METHOD Remote extraction of avatars from special distant subjects: Important bank executives, nationals of various cultures etc.

Interactive subjects experiments LAB

Single Subject cognitive experiments Agent-Based and

COMPUTER

Social Networks Simulations Computational

Intimate dialogue between Subjects and Computer via Avatar Update alternated with NatLab Market Runs Validation

finance

Numerical Model Solving Game Theory and other Analytical work DESK Economic Analysis

Relaxing in a controlled way the assumptions of the models. Expressing operationally qualitative descriptions.

Fig. 3. Positioning the Avatar-Based Method

32

Gilles Daniel et al.

The two columns of the table refere to the usual methods and the "Avatar-Based Method" (ABM from now on). One sees that in our view, the ABM constitutes a rather uniform way to treat economic behaviour and is capable of bridging application areas that were until now disjoint. This is clearly the case for the LAB and COMPUTER rows, where we even erased the separation line, but it has implications for the other rows too. For instance the Avatars, and especially within the NatLab environment, have been used already to extend to more realistic conditions some theoretical models (row 4 of the table) and results [16]. At the other extreme (rows 1-2 in the table), the Avatar-Based Method can help correct a perennial problem of economic studies: the biased and sometimes unrepresentative profile of the involved subjects. Indeed, it is very difficult to involve in those studies decision making officials fromfinancialinstitutions or traders. Substituting them by BA Undergrads is hardly a step towards realistic emulation of the real world. It is much more likely that these important players, rather than coming to a lab, will agree to provide the elements for creating their Avatars. Similar problems can be solved by including in the ABM experiments subjects from far away cultures or environments, without the necessity for distant travels and without separating them from their usual motivations and environment. Moreover, the information provided once by such subjects that are difficult to access can be used now repeatedly by playing their Avatars. Thus ABM has a good chance to bridge the gap between field studies and lab experiments too (rows 1-2 in the table). In fact as opposed to experiments that do not involve a market mechanism with capital gain and loss, in NatLab, incompetent non-representative subjects will naturally be eliminated since their Avatars loose very quickly their capital. Another point on which the ABM procedures are offering new hope is the well known problem of subjects motivation. Within the usual experimental frameworks, it is very difficult to motivate subjects, especially competent important people. From our experience, the NatLab realistic framework and the direct identification of the subjects with their Avatars successes and failures, lead to a very intensive and enthusiastic participation of the subjects even for experiments that last for a few days. In fact, beyond the question of "prestige", even seasoned professionals reported to have gained new insights in their own thinking during the sessions. Another promise that ABM is yet to deliver is that by isolating and documenting the Avatar update at discrete times, one will be able to contribute to neighbouring cognitivefieldssuch as learning.

3 Method Validation A piece of software is not an experimental set-up. With all its power, the value of the platform and of the "Avatar-Based Experiments" method has to be realized in real life and an elaborate technical and procedural set-up has to be created. The basic condition for the very applicability of our method is the humans capability to faithfiilly, precisely and consistently express their decision making in terms of computer feedable procedures. Thus we concentrated our first validation efforts in this

Traders Imprint Themselves by Adaptively Updating their Own Avatar

33

direction, adapting platform and procedural features to accommodate humans. Many other experimental aspects have to be standardised and calibrated, but in those experiments we concentrated on this crucial sine qua non issue. We can conclude at this stage that while there are humans (even economist professionals and successful traders) that could not express their "system" in a computer feedable format ("buy low , sell high"), by-and-large the participants in our experiments were able to confirm at some point that their avatar behaved in agreement with their own behaviour. This happened even with subjects with no particular computer (or market) skills. 3.1 Experimental Setup

Our experiment features a continuous double-auction implemented on the NatLab simulation platform. Every participant received extensive support from a computer scientist to implement his/her avatar in C++ on the platform. NatLab platform The NatLab has the capability to simulate in great detail the continuum time asynchronous real world [15]. Bilateral and multilateral communication between agents outside and in parallel with the market is made possible by NatLab. However, given that this experiment focuses mainly on the participants behaviour, we kept the market mechanism (the rules of the game) as simple as possible, while retaining the concept of continuous double-auction, essential to understand the price formation dynamics. NatLab was initially engineered as a simulation platform but its use is now in three distinct directions: 1. the platform provides a realistic framework for the individuals to act within. Providing this "reality" is independent of whether one is interested in its characteristics; it just allows an interactive continuous extraction of information from each of the participants and thereby refining our understanding on their approach, reactions and decision mechanisms; 2. the platform is part of a recent wide effort to understand the emergence of collective complex dynamics out of interacting agents with well defined, relatively simple individual behaviour; and 3. the platform, due to its realistic features and its asynchronous continuous time microstructure, is a reliable way to reproduce and maybe in the future predict real market behaviour. Market microstructure Our market implements a continuous double-auction mechanism, where agents can submit, asynchronously and at any time, limit or market orders to a single public book. Orders are sorted by price and then by time, as on the NYSE for instance. Every agent acts as a simple trader, and we do not include brokers or market makers at this stage. In this simple setup, agents balance their portfolio between a risky asset (a stock distributing or not a dividend) and a riskless one (a bond yielding or not an interest rate). Agents can communicate with each other through pairwise messages, and react to external news according to an idiosyncratic sensibility.

34

Gilles Daniel et al.

Avatars We organise our experiment as a competition between participants through the intermediary of their avatars. Avatars generate, by assigning values to their parameters, families of agents that act as independent (but possibly interacting) individuals in the market. The subjects' aim in each run is to generate a family of artificial agents that perform well against other families throughout the simulation run. A typical simulation run is exhibited in Fig. 4. Families were compared by their average wealth, but an average utility (given some utility function) or a certain bonus for minimising risk could be used in the future. We give our participants total liberty while implementing their avatar. They can define their own time horizon and design trading strategies as simple or complex as needed, but in the future we may tax agents with heavy data processing by imposing a fine or a specific time lag in the order execution.

3.2 Preliminary Results We have run two sets of experiments so far, with different participants including practitioners (real traders) and academics, either economists, physicists, psychologists or computer scientists. Each experiment included seven participants. The first experiment took place on July 19-31 2004, in Lyon, during the SCSHS Summer School on Models for Complex Systems in Human and Social Sciences organised by the Ecole Normale Superieure de Lyon. The second was organised on January 12-16, in Turin, during the Winter Seminar on Agent-based Models of Financial Markets organised by the ISI Foundation. A typical run, with a preliminary analysis of the price time series and relative evolution of populations, is presented on Fig. 4. We report here on some of the non trivial aspects of the participants behaviour during the experiments, while creating and updating their avatars. Imprinting oneself We noticed, specially at the beginning of the process, that some of our participants encountered some difficulties to express themselves in terms of computer feedable strategies. However, this improved dramatically during the iterative process itself This is clearly linked to the learning process that one has to face while performing any experiment, especially computerised ones. Conscious / Unconscious decisions The very nature of our method barely allows such things as intuition, improvisation or unconscious decisions to be operationally expressed in the avatar. In fact, after a few runs, avatars capture exclusively the conscious part of our subjects decision making process. Since we we do not know to what extent markets dynamics are driven by unconscious choices, it would be interesting to design a double experiment, comparing subjects and their own avatar in the same market micro structure.

Traders Imprint Themselves by Adaptively Updating their Own Avatar

35

Returns Volatility

/ -0.04

Time (au)

-0.03

-0.02

-0.01

0.00

0.01

0.02

0.03

0.04

Log Return

WlrMrf^^^'^^^'^T'V

e

.....,._,._,. ^ .

Time(au)

,,--,,,„,

-•/•"v-

Time (au)

Fig. 4. Typical run with 7 avatars, 1000 agents each, for above 350 000 transaction ticks, (a) Autocorrelation functions - absence of raw returns autocorrelation and long-term autocorrelation of volatility, as defined as absolute returns, as observed in empirical data [12];(b) Normality Plot - fat tailed distribution of returns; (c) Price trajectory; (d) Relative wealth of agents populations - measure the relative success of competing avatars; (e) Stock holdings some strategies are clearly buy-and-hold, others interact with each other; and (f) Cash holdings Convergence There are two different but related convergence processes that took place during the successive iterations: the first was the convergence of the avatar's behaviour to its creator's intended strategy, while the second involved the evolution of subjects strategy itself to beat other participants. While it appeared relatively easy after a couple of runs to get an avatar successfiiUy reproducing their initial intended behaviour, subjects, driven by competition, kept refining and complexifying their strategy.

36

Gilles Daniel et al.

Strategies An interesting panel of strategies was proposed and grown by the participants, that could loosely be termed random trader, momentum trader, oscillatory trader, diversified Bollinger Bands trader, volume seeker. Neural Network based trader and evolutionary trader. Practitioners clearly outmarked themselves by their ability to think out of the box, the creativity of their strategies, their high analysis power and ability to quickly understand what was going on and spot opportunities to arbitrage other participants' strategies. We also observed the emergence of cooperation between participants to hunt for the leader, trying to bring down the winning strategy by copying and modifying it or even custom-designing new strategies for this specific purpose. Fundamental Value In the two experiments we ran, our computer simulation figured a closed artificial market, with no creation of stocks, no distribution of dividends and no interest rate associated with the riskless asset, cash. In those conditions, we observed that after a transition period, characterised by high volumes, during which assets were heavily reallocated between agents, the price kept fluctuating around a steady state equilibrium price. This price, emergingfi*omthe interactions between heterogeneous, relative risk aversion agents, was generally different from the fundamental value we could have expected from rational agents with homogeneous preferences.

4 Conclusion The rapidly growing field of Agent-based Computational Finance comes naturally as a complementary approach to the other Finance subfields: Behavioural Finance, Laboratory experiments. Econometrics, Game Theory, etc. Thefieldis definitely out of his infancy and a rather wide range of choices is available to academics and practitioners that wish to define and test concrete real and realistic systems or new models of individual and market behaviour. The next step is to set common standards for the platforms that propose to represent and simulate artificial financial markets [9, 4, 19, 2]. One possible goal is to transform them in virtual or even real laboratories capable to implement and test in realistic conditions arbitrarily sophisticated experiments. One way to solve the problems of realistic trader behaviour is the Avatar-Based Method introduced in the present paper. Even though there are many obstacles not even yet uncovered in realizing its ambitions, the method is already providing new insight and definitely even if its main ambitions are going to remain unfulfilled, it is guaranteed to provide fresh unexpected and valuable material to the existing methods. Among the fundamental issues which the ABM can address is the mystery of price formation by providing in great detail, reliability and reproducibility, the traders decision making mechanisms. Occasionally the Avatars are going to be caught unprepared and inadequate to deal with some instances that were not previewed by their

Traders Imprint Themselves by Adaptively Updating their Own Avatar

37

owners. By the virtue of this very instance, they will become effective labels for the emergence of novelty in the market. Thus in such instances, even in its failure, the ABM will provide precious behavioural and conceptual information. ABM can serve as a design tool for practitioners in the development of new trading strategies and the design of trading automata. Moreover, we hope that this approach will provide new ways to address some of the fundamental problems underlying the economics field: • • •

how people depart from rationality how out-of-equilibrium markets achieve or not efficiency how extreme events due to a shifting composition of markets participants could be anticipated

The experiments we ran, beyond eliciting information, provided a very special and novel framework of interaction between practitioners and academics. Thus NatLab and ABM might have an impact on the community by providing a common language and vocabulary to bring together academics and much needed practitioners. As a consequence, it appears necessary to gather interdisciplinary projects that would house within the same team the psychologists that run experiments on people's behaviour, computer scientists that canonise this behaviour into artificial agents, practitioners that relate those experiments to real markets and economists that assess the consequences in terms of policy making.

Acknowledgements We would like to thank the participants of our experiments for their time and commitment, together with the participants of the Seminar on (Un)Realistic Simulations of Financial Markets at ISI Foundation, Turin, Italy, on April 1-5 2005, for their enlightening comments, from which this paper largely benefitted. Finally, we are largely indebted to Alessandro Cappellini and Pietro Tema for their views and experience on online laboratory experiments of stock markets, as well as Martin Hosnisch, Diana Mangalagiu and Tom Erez. The research of SS was supported in part by a grant from the Israeli Academy of Science, and the research of LM was supported by a grant from the Centre for Complexity Science. All errors are our own responsibilities.

References 1. R. Axelrod. The Complexity of Cooperation: Agent's Based Models of Competition and Collaboration. Princeton University Press, 1997. 2. K. Boer, M. Polman, A. Bruin, and U. Kaymak. An agent-based framework for artificial stock markets. In 16th Belgian-Dutch Conference on Artificial Intelligence (BNAIC), 2004. 3. P. Bak, M. Paczuski, and M. Shubik. Price variations in a stock market with many agents. Physica A, 246:430-453, 1997.

38

Gilles Daniel et al. 4. A. N. Cappellini. Esperimenti su mercati finanziari con agenti natural! ed artificiali. Master's thesis, Dipartimento di Scienze Economiche e Finanziarie, Facolta di Economia, Universita di Torino, Italy, 2003. 5. A. Cappellini. Avatar e simulazioni. Sistemi intelligenti, 1:45-58, 2005. 6. R. Cont and J.-P. Bouchaud. Herd behaviour and aggregate fluctuations in financial markets. Macroeconomic Dynamics, 4:170-196, 2000. 7. J. Duffy. Agent-based models and human subject experiments. Computational Economics 0412001, Economics Working Paper Archive at WUSTL, December 2004. available at http://ideas.repec.Org/p/wpa/wuwpco/0412001.html. 8. I. Giardina and J.-P. Bouchaud. Bubbles, crashes and intermittency in agent based market models. The European PhysicalJournal B, 31:421-537, 2003. 9. B. I. Jacobs, K. N. Levy, and H. Markowitz. Financial market simulations. Journal of Portfolio Management, 30th Anniversary, 2004. 10. G. Kim and H. Markowitz. Investment rules, margin, and market volatility. Journal of Portfolio Management, 16(l):45-52, 1989. 11. D. Kahneman and A. Tversky. Prospect theory: An analysis of decision under risk. Econometrica, 47(2):263-292, 1979. 12. Y Liu, P. Gopikrishnan, P. Cizeau, M. Meyer, C. Peng, and H. E. Stanley. Statistical properties of the volatility of price fluctuations. Physical Review E, 60:1390-1400, 1999. 13. H. Levy, M. Levy, and S. Solomon. Microscopic Simulation of Financial Markets: From Investor Behavior to Market Phenomena. Berkeley, CA: Academic Press, 2000. 14. T. Lux and M. Marchesi. Scaling and criticality in a stochastic multi-agent model of a financial market. Nature, 397:498-500, 1999. 15. L. Muchnik and S. Solomon. Statistical mechanics of conventional traders may lead to non-conventional market behavior. Physica Scripta, 7106:41-47, 2003. 16. L. Muchnik, F, Slanina, and S. Solomon. The interacting gaps model: reconciling theoretical and numerical approaches to limit-order models. Physica A, 330:232239, 2003. 17. Lev Muchnik. Simulating emergence of complex collective dynamics in the stock markets.

http://shum.huji.ac.il/~sorin/ccs/Lev-Thesis.pdf. 18. R. G. Palmer, W. B. Arthur, J. H. Holland, B. LeBaron, and R Tayler. Artificial economic life: a simple model of a stock market. Physica D, IS'.ld^llA, 1994. 19. M. Shatner, L. Muchnik, M. Leshno, and S. Solomon. A continuous time asynchronous model of the stock market; beyond the lis model. In Economic Dynamics from the Physics Point of View. Physikzentrum Bad Honnef, Germany, 2000. 20. G. J. Stigler. Public regulation of the securities market. Journal of Business, 37(2): 117-142, 1964. 21. P. Tema. Sum: A surprising (un)realistic market - building a simple stock market structure with swarm. In Computing in Economics and Finance. Society for Computational Economics, 2000.

Learning in l\/lodels

Learning in Continuous Double Auction IVIarket Marta Posada, Cesareo Hernandez, and Adolfo Lopez-Paredes University of Valladolid, E.T.S. de Ingenieros Industriales, Paseo del Cauce s/n, 47011 Valladolid, Spain posada@eis . u v a . es

Summary. We start from the fact, that individual behaviour is always mediated by social relations. A heuristic is not good or bad, rational or irrational, but only relative to an institutional environment. Thus for a given environment, the Continuous Double Action (CDA) market, we examine the performance of alternative intelligent agents, in terms of market efficiency and individual surplus. In CDA markets traders face three non-trivial decisions: How much should they bid or ask for their own tokens? When should they place a bid or an ask? And when should they accept an outstanding order of some other trader? Artificially intelligent traders have been used to explore the properties of the CDA market. But, in all previous works, agents have afixedbidding strategy during the auction. In our simulations we allow the soft-agents to learn not only about how much they should bid or ask, but also about possible switching between the alternative strategies. We examine the emergence or not of Nash equilibriums, with a bottom-up approach. Our results confirm that although market efficiency is an ecological property, an it is robust against intelligence agents, convergence and volatility depend on the learning strategy. Furthermore our results are at odds with the results obtained from a top-down approach, which claimed the existence of Nash equilibriums.

1 CDA Market and Artificial Agents CDA is the dominant institution for the real-world trading of equities, CO2 emissions permits, derivatives, etc. In the CDA, a buyer can submit a price at which he is willing to buy (make a bid), and a seller can submit a price at which he is willing to sell (make an ask). If another buyer bids a higher price, it becomes the market bid; if another seller asks a lower price, it becomes the market ask. A buyer is free to accept the market ask; a seller is free to accept the market bid. Such acceptances consummate a binding transaction. Trades takes place as new bid-ask arrive. The auction continues for a specified period of time. The convergence and efficiency properties of the CDA have been the subject of interest among experimental economists, beginning with the seminal work of Smith [10]. In experimental economics (EE) the experiments directly reveal agent's

42

Marta Posada et al.

aggregated behaviour, they do not provide information about their individual decision rules and the impact of these rules on individual and aggregate performance. Simulation and game-theoretic analyses partially overcome this limitation. Gametheoretic studies on double auction generally adopt the framework of static games with incomplete in-formation, for which the equilibrium solution is Bayesian Nash equilibrium. How-ever, CDA is a complex dynamic system and we need a framework that captures the basic dynamics of the system. As Kirman and Vriend [7] argued, if we want to understand the dynamics of interactive market processes and the emergent properties of the evolving market structures and outcomes, it might pay to analyze explicitly how agents interact with each other and how information spreads through the market. A natural way to do this is following an agent-based computational Economics (ACE) approach (for reasons see Vriend [14]). The first shocking result from ACE approach to CDA markets was that ZeroIntelligent (ZI) agents with random offers, may lead to high market efficiency, thus proving that the institutional design is robust against individual learning (Sunder [11]). This result is in agreement with the eighteenth century classical philosophers, Adam Smith and David Hume, and later on with the Austrians, that claimed that spontaneous order may be an outcome of the individual interactions. Ecological learning and social learning beat individual intelligent learning.

2 intelligent Agents in the CDA Market Although the Zero Intelligence agents may achieve high market efficiency, ZI agents perform poorly when they are competing with agents with learning capacity (Tesauro and Das [12]). It seems intuitively reasonable that one should endorse the agents in a CDA with both, intelligence and adaptation (social learning). In a series of subsequent papers non zero intelligent agents have been proposed to analyze the performance of the CDA market: ZIP agents, GD agents, etc. A GD agent (Gjerstad and Dickhaut [4]) chooses the offer that maximizes his surplus, defined as the product of the gain from trade and the belief that some agent will accept the offer. GD agents use the history HM of the recent market activity (the bids and asks leading to the last M traders: ABL accepted bid less than b, AL accepted bid and ask less than 6, RBG rejected bid greater than 6, etc.) to calculate a belieffianction.Interpolation is used for prices at which no orders or traders are registered in HM. For example, the belief function of a buyer is: ^^^

ABL{b) + AL{b) + RBG{b)

^^

ZIP agents (Cliff and Bruten [2]) have a mark up that determines the price at which it is willing to buy or sell. The agents learn to modify this profit margin with adaptive learning. For example, the profit margin of a buyer is: M= 1

howMuchBidt-i -\- At ;;^ ;7^ ? ReservePnce

(2)

Learning in Continuous Double Auction Market

43

where At is calculated using the individual trader's learning rate (/?), the momentum learning coefficient (7) and the difference between the target bid and the bid in the last round (howMuchBidt_i) in the following way: At = jAt-i

+ (1 - j)P (targetBid - howMuchBidt_i)

(3)

In our analysis we consider one bidding strategy more, the Kaplan strategy (K) that was the winner of the Santa Fe tournament (Rust et al [9]). The basic idea behind this strategy can be summarized as follows: wait in the background and let others negotiate. Kaplan buyers will bid at the best ask only when one of the following three conditions are met: 1 The fraction of time remaining in the period is less than 10%. 2 The best ask is less than the minimum trade price in the previous trade. 3 The best ask is less than the maximum trade price in the previous period and the ratio of the bid-ask spread and the best ask is less than 10%, and the expected profit is more than 2%. We have reproduced these results for homogeneous population of agents with fixed learning strategies during the auction. Figure 1 presents the time series of transaction prices during three sessions (100 rounds per session). Learning and intelligence play an important role in the convergence of the transaction prices to equilibrium competitive price. The GD agents learn very soon to trade at a price very close to the competitive equilibrium price. The transactions are made in the first rounds of each period. The ZIP agents take more time than GD agents both to exchange and to learn. K agents must be parasitic on the intelligent agents to trade and to obtain profit. If all traders in the market are K agents no trade will take place. Although learning and the convergence to the Nash Equilibrium have been widely studied (Kirman [6]), there are few applications to the analysis of learning strategies in a CDA market (Walsh et al [15]). Walsh et al [15] found two Nash equilibria points when these three types agents (GD, K and ZIP) rival each other in a CDA market and their agents strategies are fixed by the modeller. But the question we put forward in this work is: If the agents

r 1% • *

m

1

Zl

flk

;V^^'^/r^»:>y

i-XA s 1

-J*

>--

...a...... -e-

# •

\

T

100

200

300 0

100

—1

200

300 0

100

200

300

Fig. 1. Transaction price time series for different homogeneous populations of agents during three sessions (100 rounds per session)

44

Malta Posada et al.

can adjust their strategies in an adaptative way, closer to the behaviour in the real world, will the Nash equilibrium be achieved?

3 Agents that Learn how to Choose Between Alternative Strategies Our goal is to examine the emergence of Nash efficient solutions with a bottom-up approach. The agents in our model can achieve high profits. They are robust with respect to various opponent strategies. Each agent chooses a strategyfi-oma set of three alternatives (GD, K and ZIP) at the start of each period. The initial strategy is chosen randomly. On subsequent periods, each agent learns to change his strategy looking for the best bidding strategy in the following way. To take this decision each trader only knows their own reservation prices and the information generated in the market, but he does not know the bidding strategy or the profit achieved in a market session by other agents. An agent will consider to change his strategy if the profit is less than the profit from the previous period. The agent considers whether he could have reached higher profits following an alternative strategy. To this end he assesses the transactions he made in the past. He considers both the transactions in which he took an active askbid strategy and those where he passively accepted the bid-ask (Fig. 2). In the first case, there are three possible alternatives for the buyer: 1 The bid of the alternative strategy was lower than the minimum price of exchange for this period. In this case the buyer will assume that no seller would have accepted it. 2 The bid of the alternative strategy was lower than the realized bid, but greater than the minimum transaction price for that period. Then the buyer will consider that the bid would have been accepted and he could have obtained greater profits. The profit is represented in thefigureby a blue bar. 3 The bid of the alternative strategy was greater than the realized bid. Then, he could have obtained lower profits. In the second case, above, there are only two possibilities: 1 The bid of the alternative strategy was lower than the seller's ask. The buyer would have rejected the ask with no profit. 2 The bid of the alternative strategy was greater than the seller's ask. Then he could have obtained the same profits whatever the value of the bid was. If an agent has not traded yet, he will consider the transactions made by the other agents and he will proceed with the same criteria discussed above. If an agent has no information at all, and there are no open orders, he will change his strategies in a random way.

Learning in Continuous Double Auction Market

45

4 Results and Discussion We have found that no matter what the initial population of agents is, the final composition of agents remains in a central region, where no strategy seems to dominate. The results of the simulation show that not only the proportion of strategies is relevant. It is also important the proportion of strategies of buyers and sellers. Let us comment in detail about these results. 4.1 Results when the Agents Have Different but Fixed Strategies We have simulated two scenarios. One in which the agents are not allowed to change their strategies during the auction. The aim is to reveal the learning patterns corresponding to each strategy. In the other one, the agents can change the strategy to increase their profits. This individual goal diversity should lead to an increase in market efficiency. We represent the strategy space by a two dimensional simplex grid with vertices corresponding to the pure strategies (Fig. 3): all ZIP (point a), all GD (point d) and all K (point f). We draw three regions to represent the populations that have a dominant strategy when more than 33% of the agents use the strategy. As it happens, ZIP agents are a majority in the red region (abc), GD agents settle in the blue region (bde), K agents choice is the yellow region (cef).

NO CHANGE

L

AcTive

pp Ptissive

CHANGE based price (self) ;i) (ii CHANGE based price (others)

7) min_|

:3)

©

CHANGE randoiti

i^Fig. 2. Buyers algorithm to change the type of strategy for each trading period

46

Marta Posada et al.

(a) ZIP GD{f}

K(f}

first period

(a) ZIP GD(f) last period Fig. 3. Efficiency market of the strategy space thefirstand the last session of experimental run when the agents havefixedstrategies during the auction Figure 3 presents the average efficiency market of all the populations in the strategy space in the first and the last session of experimental run (100 rounds per session) when the agents have fixed strategies during the auction. Note, that in the first period the CDA market efficiency is near 100% in almost populations of the strategy space. On the other hand, the topography has not changed substantially after fifteen auction sessions. Our results confirm that market efficiency is an ecological property, thus robust against alternative intelligent agents. But, the CDA market efficiency is very low when most of the agents are of the K t3rpe (yellow region). The volatility of the market efficiency is low but increases when the proportion of K agents is very high. The reason is not only that the proportion of strategies matters. The distribution of this proportion of strategies between the two market sides, buyers and sellers, is important as well. Let us analyze with further detail this fact. In figure 4 we show the price evolution and the market efficiency when 50% are K agents and 50%) are GD agents and they have fixed strategies during the auction. When no side is silent there is high efficiency and the transaction prices approach the equilibrium better.

Learning in Continuous Double Auction Market

47

buyer: 50%GD&50%K

- ^ - ^ - - : ^ j ^ - ^ - * n - '9F - ' ^ -•> - -fei/ " t ' - h ^ - - ^

500

1000

•I5D0;

buyer: 100%K competitive price

•

•

I

I

I

comjjetitive puhe

seller: 100%K

Fig. 4. Price dynamics and efficiency market when 50% are K agents and 50% are GD agents in a CDA When this 50% of K agents are all buyers, the transaction prices are below the competitive equilibrium price and the market efficiency is low. When they are all sellers the transaction prices are over the competitive equilibrium price and the market efficiency is low as well. We find equivalent results in experimental economics: contracts tend to be executed to the disadvantage of the side having the price initiative. But in this case it is no possible to control the parasitic behaviour of the real agents, and this control has to be done by the conductor of the auction, through the auction rules. The institutions in which one side of the market is free to accept but is not permitted to make an offer are labelled offer auction (if buyers are not permitted to make bids) and bid auction (if sellers are not permitted to make asks). This is a good example of the equivalence between an institution with the agents free to choose their strategies and another institution with individual agents forced to maintain a given strategy.

48

Marta Posada et al.

4.2 Results when the Agents Have Different Strategies and Can Change them We start the simulation with a proportion of 50% of K agents but we allow them to change their strategies to increase his profit so that they can move to ZIP or GD all along the experimental run. As it can be seen from the Figure 5, the transaction prices are very near the competitive equilibrium price and there is an increase in market efficiency (near 100%) in all cases. Figure 6 presents the evolution of transaction prices, market efficiency and proportion of strategies when we start the simulation with a proportion of 50%) of K agents and 50%) of ZIP agents. The coloured bars indicate the percentage of the alternative strategies in the population (blue for GD, yellow for K and red for ZIP) and we can observe the dynamics of the change in strategies along the auction sessions. In fact, comparing figures 5 and 6, it seems that the final proportions of strategies outcome strongly depend upon: the path, the initial strategy composition of the population and the proportion of strategies of buyers and sellers. We have found that the final proportion of strategies' agents remains in a central region, where no strategy seems to dominate. Some GD agents and ZIP agents consider whether they could have reached higher profit following a K strategy. But they are well aware that if the number of parasitic agents increases too much, they will

^—1-^

• IJ

7Un

• GD

1 K 1

15'

• ZIP

10'

5'

H1tiJ:':':'^iVrr[1 i

2

3

4

5

6

\ : llt/i

7

8

9

10 11 12 13 14 15

J

. ..

^;rrrrrrrrrrri 1

2 3 4 5 6 7 8 9 10 11 12 13 14 15

I nmrrir.r.r.l'.tl 1

2 3 4 5 -e- 7 8 9 10 11 12 13 14 15

Fig. 5. Price dynamics and efficiency market when initially K agents (50%) and GD agents (50%) change their strategy in a CDA market

Learning in Continuous Double Auction Market

49

iiiHidlbUy^l^^^K&Sa^ZIP

[7171

si

•••^V^'VVvVVVVVV*d :;:

mTir.r.rrl.r.rJ i

Initfal buyers ftW^K

J

J

5

«.

7

12

U

U

15

iV-r

ty

[-

if

^

M I flilitnnhi J

^

5

t

T

?

^

10

11

1i

U

U

15

kdnflJi i i i i r J.

2

.!

J,

5

(-.J

4

a

10

11

1£

1!

W

15.

Fig. 6. Price dynamics and efficiency market when initially K agents (50%) and ZIP agents (50%) change their strategy in a CDA market

decrease their profit and the market efficiency, thus limiting theirfi*eeriding opportunity. This behaviour resembles passive investment in the stock market. If there are too many passive investors, there will not be enough arbitrage and the stock market will lose efficiency. Our results are at odds with those by Walsh et al [15], from a top-down approach that claimed the existence of two Nash equilibriums. We do not find such results in our simulations. We want to remark that in Walsh's work the agents have fixed strategies during the auction for each population of the space strategy. They calculate the Nash equilibrium from the following rule that is fixed externally once the auctions have finished. The new percentage of a specific type agent is the last percentage plus an amount proportional to the strategy's profit and the average market's profit. This is a major drawback of the top-down approach, since there is not emergence in the proportion of the strategies, but it is forced by the modellers. We think that the agents should have freedom to change their strategies and it is not realistic to assume that the agents know the group average profit for each strategy. We would like to interpret our results as a contribution to clarify a crucial issue in economics: the relationship between methodological individualism and social knowledge (Arrow [1]), as one of the reviewers pointed out.

50

Marta Posada et al.

With this focus our results and previous works, mainly the seminal contribution of Sunder confirm that ZI traders are only an important but the first historical step toward using computer simulations with artificially intelligent traders to explore the robustness and structural properties of markets. These simulations that are the wind tunnels of economics, can give us interesting clues and discoveries on the relationship between individual and social learning. Individual and social learning are interrelated (Vriend [14] and Gigerenzer [3]) and consequently, a learning process is not good or bad, rational or irrational, but only relative to an institutional environment. In general terms a distributed system with autonomous agents, that are not very instructed but are socially aware, can exhibit high intelligence as a system. The whole purpose of market design is to achieve this market efficiency even with traders that may not be much instructed. Our instruments for bounded rational agents explicitly should make methodological individualism and social knowledge compatible views (Lopez et al. [8]). To this end, a bottom-up approach, such as agent based simulation can be very useful. Thus the opportunity of this symposium and hopefully of this paper as well.

Acknowledgements We wish to thank three anonymous referees for their helpfiil comments.

References 1. Arrow K (1994) Methodological Individualism and Social Knowledge. American Economic Review 84 (2): 1-9. 2. Cliff D, Bruten J (1997) Zero is not enough: On the lower limit of agent intelligence for continuous double auction markets. HP-1997-141, Hewlett Packard Laboratories, Bristol, England. 3. Gigerenzer G (2005) Striking a Blow for Sanity in Theories of Rationality. In: Augier M, March JG (eds) Models of a Man: Essays in Memory of Herbert A. Simon, 389-410. MIT, Cambridge. 4. Gjerstad S, Dickhaut J (1998) Price formation in double auctions. Games and Economic Behavior 22: 1-29. 5. Gode D, Sunder S (1993) AUocative efficiency of market with zero-intelligent traders: Market as a partial substitute for individual rationality. Journal of Political Economy 101: 119-137. 6. Kirman A, Salmon A (eds) (1993) Learning and rationality in economics. Blackwell, Oxford. 7. Kirman A, Vriend N (2001) Evolving market structure: an ACE model of price dispersion and loyalty. Journal of Economic Dynamics and Control 25: 459-502. 8. Lopez A, Hernandez C, Pajares J (2002) Towards a new experimental socio-economics. Complex behaviour in bargaining. Journal of Socio-Economics 31: 423-429.

Learning in Continuous Double Auction Market

51

9. Rust, J, Miller J, and Palmer R (1993) Behavior of trading automata in computerized double auctions. In: Friedman and Rust (eds) The double auction markets: Institutions, theories and evidence. Addison-Wesley, New York. 10. Smith V (1962) An experimental study of competitive market behavior. Journal of Political Economy 70: 111-137. 11. Sunder S (2005) Markets as Artifacts: Aggregate Efficiency from Zero-Intelligence Traders. In: Augier M, March JG (eds) Models of a Man: Essays in Memory of Herbert A. Simon, 501-519. MIT, Cambridge. 12. Tesauro G, Das R. (2001) High performance bidding agents for the continuous double auction. In: Proceedings of the ACM Conference on Electronic Commerce (EC-01), 206209. Tampa, Florida. 13. Vriend N (1996) Rational behavior and economic theory. Journal of Economic Behavior and Organization 29: 263-285 14. Vriend N (2000) An illustration of the essential difference between individual and social learning, and its consequences for computational analyses. Journal of Economic Dynamics and Control 24: 1-19. 15. Walsh W, Das G, Tesauro G, Kephart J (2002) Analyzing complex strategic interactions in multi-agent systems. In: Game theoretic and decision theoretic agents, 109-118. AAAI Press, Menlo Park.

Firms Adaptation in Dynamic Economic Systems Lilia Rejeb^ and Zahia Guessoum^ ^ MODECO-CRESTIC University of Reims, L i l i a . Re j e b @ p o l e i a . I i p 6 . f r ^ 0ASIS-LIP6 university of Paris VI, Z a h i a . G u e s s o u m @ l i p 6 . f r

1 Introduction Evolutionary economic systems are often large-scale, dynamic, open and characterized by a strong competition. The survival offirmsin such complex systems relies on their decision process and also on the behavior of the otherfirms.Pajares [9] shows that in evolutionary economics, learning is the central issue of every model. Firms have thus to continuously revise their strategies according to their experience [5]. They have to select the most suited actions according to their local data and a partial perception of their dynamic environment. It is important then to find a learning technique allowing firms to construct a model of their environment, to update it according to their experience and to foresee the possible consequences of their decision before using it. Learning classifier systems (LCS) provide a good solution to model thefirmdecision. They combine reinforcement learning and evolutionary computing to produce adaptive behavior. They allow afirmto : • • •

build a model of its environment through a set of rules, use this model to anticipate the value of each action before adopting it, and evaluate its actions.

ThefirstappHcations of LCS have been introduced by Holland [8] [12,13]. However, recent research have proposed new improvements of LCS such as ZCS [16] and XCS [17]. We propose to use XCS to model afirmdecision process. Unlike LCS and ZCS, XCS can construct a complete and accurate model of the environment of the firm through efficient generalizations. Moreover, XCS can develop a readable set of rules which help to explain the evolution of the environment. The purpose of this paper is to build an XCS-based multi-agent system representing adaptive firms and their interactions. Each firm uses XCS for its learning. This XCS-based multi-agent system allows to explain thefirmand the global system behavior. The study of adaptive firms allows displaying the advantages and drawbacks of using XCS to model learning in multi-agent systems. We show that the

54

Lilia Rejeb and Zahia Guessoum

performances offirmscan be improved by using a more precise representation of the parameters of the firm. The paper is organized as follows. Section 2 presents the firm model. Section 3 describes the adaptivefirms.It gives an overview of XCS and presents the modeling of firms by the XCS-based agents. Section 4 presents an overview of the realized experiments and Section 5 gives a short overview of the related work.

2 The Firms Model Our economic system is characterized by a set of firms in indirect interaction via a competitive market. To model the firms, we use a resource-based approach [11]. This approach regards a firm as a collection of physical and human resources. It stipulates that the survival of afirmdepends on its use of its resources. The firm is thus defined by the parameters X, Y, K, B and S where: • • • • •

X is a set of resources, Y is a set of performances (profitability Yt[l\ and market performance ^^[2]). The performance of afirmis measured using the statistical Lisrel Model, K is a capital, B is a budget. It is used to update the resources, S is a set of strategies. Each strategy defines a method for distributing the budget among the different resources according to their priorities. The behavior of thefirmis described for each period by the following steps:

• • • •

Perception of the environment and update of the competition model. Update of the internal parameters such as the capital and the budget. Decision making, which corresponds to the choice of a strategy according to the current context. Update of the performances of the firm.

The context of a firm is determined by the firm's internal parameters (K, B^ X, and Yt), and its perception of the environment which is strongly competitive and non-stationary. At each time period,firmscan either join or leave the market. A firm leaves the market either when its performance decreases over a number of successive periods or when its capital decreases and reaches an exit threshold. However, it is impossible for the firm to dispose of all the information about its rivals. Thus, the firm has to use its previous experience to disambiguate the current state of the environment and foresee the consequences of its possible strategies to choose the adequate one. Eachfirmis represented by an agent. The perception component of an agent uses the internal parameters of the firm and its environment to build a representation of its current context. The decision process allows to choose the most suited strategy in a given context.

Firms Adaptation in Dynamic Economic Systems

55

3 Adaptive Firms The decision process of the firm is represented by XCS. The first section gives an overview of the learning classifier system XCS and the second section presents the XCS-based firm. 3.1 XCS XCS is a learning classifier system defined by Wilson [17]. Knowledge is represented by standard "condition-action" rules called classifiers. Each classifier is also characterized by three parameters: • • •

the prediction p, which corresponds to the average estimated reward when the classifier is used; the prediction error e, which estimates the error in the prediction of the classifier, the fitness F, which evaluates the prediction quality. It is the average prediction accuracy given by p. This quality distinguishes XCS from its predecessors.

A genetic algorithm is used to update this set of classifiers, whereas a Q-leaming technique is used to assign a value to the prediction p. In XCS p corresponds to the accuracy of the prediction of the reward. Moreover, XCS includes the generalization which allows representing knowledge in more compact manner by generalizing similar situations. XCS allows to choose an action either by exploration (random choice) or by exploitation (choice of the action having the greater value of PSi). The execution of the chosen action by the environment, generates a reward which is used by the reinforcement learning component of XCS to evaluate the classifiers. The reward could be immediate (single step XCS) or obtained at the end of a chain of actions (multiple step XCS). This reward is used to update the parameters characterizing the classifiers in XCS (p, e and F). The update of these parameters is done by the reinforcement learning component of XCS. The following formula are used in the presented order to update these parameters: Pclj = Pclj + P{PSi - Pel J ) (1) where (3 is the learning coefficient of XCS, clj is the classifier j using the chosen action and PSi is the average prediction of the chosen action a^. ecij = eel J + P{\PSi - Pclj I - edj)

(2)

edj is the prediction error of the classifier j . This prediction error is then used to update the fitness and the accuracy k :

*'-{r and

(ej/eo)

'',ej> eo; otherwise.

.^.

56

Lilia Rejeb and Zahia Guessoum

kj * num F,=F,+p{^^::::z^-F^

(4)

where • • •

clj is the classifier j using the chosen action a^, eo and v are parameters specific to XCS allowing to determine A;, nurrij is the numerosity of the classifier j . One step of XCS is described in the algorithm defined in table 1 ^.

do { 1 2

Table 1. An experiment of XCS

perception of the state of the environment, determination of the set [M] of the matching classifiers to the environment state; if [M] is empty, covering takes place; 3 determination of the system prediction array [SP]; 4 choice of the action "a" according to [SP] either by exploration(random choice)or by exploitation (choice of the action having the best value SP); 5 execution of the action "a" by the environment and determination of the reward R; 6 generation of the action set [A] gathering the classifiers in [M]having "a" as action; 7 reception of the reward R; 8 evaluation of the classifiers and application of the genetic algorithm if possible either in [A] if the reward is immediate or in [A]_1 if the reward is not immediate; } while the end of the problem is not reached.

3.2 XCS-Based Firms XCS-based firms are obtained by the integration of XCS and the agent representing a firm. XCS receives the perception of the context of the firm. It applies steps 1-4 of the algorithm in table 1 to determine the adequate action. This action is applied by the firm and gives a reward which is sent back to XCS to update the parameters of the classifiers. The reward is immediate. Thus, we use one-step XCS. This section presents the perception coding of the firm context and the used reward fiinctions.

' for more details see "An algorithmic description of XCS" [4]

Firms Adaptation in Dynamic Economic Systems

57

Perception of the environment

Fig. 1. XCS-based firm The Context

The diversity of the firm context parameters and their types (real, fuzzy, integer) and the continuous and non-stationary character of the system, makes it difficult to delimit the definition domains of the parameters. So, the recent representation methods for learning classifier systems such as real intervals and S-expressions [18] are not suitable to model the firm context. A unification method to homogenize the representation of thefirmparameters is then required. We propose therefore a unification method based on the decomposition of the definition domains of the parameters into m intervals . Each interval is characterized by a fuzzy value giving the rough estimate of the corresponding parameter. We opt for fiizzy granulation as it mimics the human reasoning and manipulation of information resulting from perception [21]. The domain definition of each parameter is described by symbolic values such as (very small, small, medium, large). Thesefiizzyvalues are then translated in binary coding to obtain a homogeneous representation of thefirmparameters (the method will be presented in details in the full paper). This method is general and independent of the application. The action in XCS corresponds to the firm's strategy. The number of strategies is fixed at the beginning by the economist. Table 2 represents a firm classifier and its translation into a classifier usable by XCS. This classifier associates the strategy 1 to the defined context. The use of strategy 1 in this context gives an estimated reward (prediction) of 0.5, a prediction error of 0.1 and afitnessof 100. 3.3 Decision Process

Usual rewardfiinctionsare discrete. They correspond to the allocation of a positive value when the action gives good results and 0 otherwise [18]. However, in real-life applications and in the firm problem, an action that results in a great improvement of the performances must not be recompensed as an action that results in a small improvement. We propose then to use a fiinction varying according to the context. Thisfiinctioncould be defined, either by considering the individual performances of thefirm,or by collective ones taking into account the other firms.

58

Lilia Rejeb and Zahia Guessoum Table 2. Classifier representing thefirmcontext Classifier representing the firm's context

Classifiers in XSC

K is small 0001 B is medium 0010 X = { xi is very small, X2 is small, X3 is medium, 0000,0010,10001 X4 is very small, X5 is very small,: >C6 is verysmall. 0010, 0001,0000 X7 is very small, xs is very small} 0000,0000, Y = {yi is small, y2 is small} 0001,0001, Average_K is large 0011 Average_B is very large 0111 , NbFirms is very small 0000 Average.Y={Aver_yi is medium ,.Aver_y2 is small} 0010, 0001 Action = Strategy! , parameters

1, (P)=0.5,(e>=0.01,(F)=100

The Indivual Reward Function We model this reward according to the performance variation. It is defined by an aggregation of the variation of the performances of the firm:

where Yt [1] corresponds to the profitability, Yt [2] corresponds to the market performance and aggreg is an aggregation operator. The average operator is used for this work. The Collective Reward Function Peres-Uribe [10] notes that it may be profitable to consider the effect of an agent on the other agents to measure its performance improvement. We propose then to model this collective performance by the relative performance of the firm proposed by Durand [6]. The relative performance considers the past performances of the firm and the competition state. It evaluates the position of the firm according to its rivals in the market. It is defined by: RelPerft = ^ ( ^ , B, C, D).

(6)

where •

A corresponds to the variation rate of profitability

Y,[l]-Y,_,[l] ^n-i[i]

^^^

Firms Adaptation in Dynamic Economic Systems •

59

B corresponds to thefirm'sprofitability in comparison to the average profit in the market Yt[l] -averageYi J=> =

77

•

averageYi and averageYi is the average profitability in the market. C is the evolution of the market performance

•

D is an index that gives a premium for the best market performer at time t Yt[2]-Min{Yt[2]i) Min{Yt[2]i)

(o)

^ ""^

where i e[l,k] and k is the total number of firms. The reward function corresponds then to the variation of this relative performance. It is expressed by : reward = RelPerft — RelPerft-i

(11)

4 Experiments For the implementation of the XCS-based firms, we use the XCSfi'ameworkproposed by Butz and Wilson [4]. We integrate it in the agent-based framework DIMA [7]. The first series of experiments compares the rule-based and XCS-based firms. The second series of experiments performs a sensibility analysis of the performances and learning of XCS to the coding precision and the reward ftinction. The XCS parameters are: •

the population size N is set to 6000 to allow the system to represent all the possible classifiers when the generalization is not used, the generalization probability #_probability = 0.5, the learning rate /3 = 0.001, the crossover Rate = 0.8, the mutation rate = 0.02, the minimum error = 0.01, the genetic algorithm frequency Ogen = 10, the exploration probability = 0.5, the exploration/exploitation selection mechanism is the same for all the firms.

The simulations were replicated 20 times. The obtained results are the average values. We compare the learning time or convergence of the classifiers and their performance improvement.

60

Lilia Rejeb and Zahia Guessoum

4.1 XCS-Based Firms Versus Rules-Based Firms To validate the XCS-based firm architecture, we compare the evolution of a population of 300 XCS-based firms and a population of 300 rule-based firms. Rule-based firms are endowed with a set of static rules. This rules choose the strategy according to the comparison of the performances of the firm to those of the market. To focus on the influence of the decision process, we endow these two kinds of firms with the same initial parameters. XCS-based firms use the individual reward function (see Section 3.3).

XCS-bfised furns Rule-based fims

0

?0Q

40D

eOD

80Q

1000

12O0

1400

16

Time

Fig. 2. Rule-based firms versus XCS-based firms

Table 3 Comparison of firms resistance Age

XCS-based firm Rule-based firm

Maximum 142 Average 23 Median 22

123 19 17

Figure 2 compares the average capital of the two populations of firms. Table 3 compares the firms age. Based upon Figure 2 and Table 3, we can conclude that XCS-based firms have better performances and are more resistant. Their capital is often stabilized well above that of the rule-based firms. This is due to their ability to improve their classifiers. However, they have a first difficult phase. Their initial classifier base is empty, their difficulties can be therefore explained by the random choice of strategies. 4.2 Coding Influence We focus in these experiments on the coding influence. We compare two populations of 300 firms, having the same initial parameters and differing by their representation

Firms Adaptation in Dynamic Economic Systems

61

precision. The first population uses a decomposition of the definition domain into 8 intervals. However, the second uses a decomposition in 16 intervals. Table 4. Comparison of different representations firms resistance Age

8 intervals representation 16 intervals representation 230 27

Maximum 209 Average 25

Table 4 shows that the use of a more precise representation improves the firm's resistance. However, less precise representation can lead to classifiers overgeneralization. Classifiers can thus become too general to be explained by economists.

0

10Q0 2000

3000 4000

5000 6000 7000 5000

9000

Time

Fig. 3. Influence of coding precision on the evolution of classifiers populations

Figure 3 compares the number and evolution of the classifiers of the two kinds of firms using respectively 8 intervals and 16 intervals. The classifiers number of firms using 16 intervals stabilizes later than that's offirmsusing 8 intervals representation. However, the classifiers population is richer for 16 intervals. Thus, more environment states are considered. So, a more precise decomposition of the definition domains gives a better representation of the environment and consequently better performances and resistance. However, it is costly in terms of learning time. Nevertheless, using more precise representations is an open problem when the environment states space is large. This is due to limited length of classifiers in XCS. 4.3 Individual vs Collective Reward Function We compare in these experiments two populations of 300firms.The first reinforces the classifiers according to its performances enhancement without considering the otherfirms.The second reinforces the classifiers according to the firm relative position in the market. These two populations have the same initial parameters and the

62

Lilia Rejeb and Zahia Guessoum

same representation. They differ only by their reward function. As we are in a multiagent context, the aim of these experiments is to verify if it is sufficient for firms to consider the other firms only in their perception or it is necessary to consider them also in the definition of the reward. 400 -5 E -o

300 200

B1

100

Collective-lndivjdual J

Jk

i .u 1

iA ,

.

1 [iL MTilfl'n/nffl T i l W\iid V \wu mP III ^ r^LiuMiKiNiU " Unp !ftJ 4oU yp W!^ T so^il L P vfvr ^\ f f r

»

te -200 a .S fi B*" -100

1 ^

-300

S

-400

V 1 1

'500 Time

Fig. 4. Collective versus Individual Reward Figure 4 shows that the difference between the performances of the firms using the two reward functions is not important. The figure shows that on average the collective reward function does not greatly improve the firm performance. The percent average improvement is of 1%. The comparison of their resistance confirms this result. We can conclude then that considering the other firms, only in its perception, is sufficient for the firm to make good decisions. These experiments respond to the question of Peres-Uribe about the utility of a collective performance improvement. A complete perception of the environment is more important than a collective performance improvement function in learning multi-agent systems.

5 Related Work Many techniques were used to model learning economic agents such as genetic algorithms [3,19], neural networks [2], reinforcement learning techniques (classifier systems [12, 13, 19] and Q-Leaming [15]). Neural networks work as black boxes. They cannot explain the firm behavior as it is needed by economic researchers. Moreover, their complexity relies on the importance of the environment states [14]. Q-leaming is also unsuited to model the firm's decision since it suffers from a problem of convergence in non- stationary environments and large state-actions spaces. It is also inefficient in capturing the environments regularities and consequently cannot avoid the exponential explosion in time and space [18]. Genetic algorithms do not fit our firm model as they do not allow the firm to take advantage of its previous experience to construct expectations about the environment [19]. In fact, the actions are evaluated after their actual use. However, this can result in bad consequences for firms. Learning classifier systems were widely used in learning economic systems [12, 13, 19, 1]. The first works [12, 13] used the LCS model of Holland. These

Firms Adaptation in Dynamic Economic Systems

63

systems are characterized by a set of internal messages constituting a kind of memory and by using forces to evaluate the classifiers. The learning speed of these systems are however very slow. More recent works such as those of [19, 1] use XCS. Yildizoglu[19] used XCS for the Nelson and Winter model based firm's learning. The parameters representation of these firms is a binary string indicating if the corresponding parameters improved or not. The reward function is represented by the capital productivity and profit. However, this system was only tried in homogeneous environments. Bagnall [1] used it to make offers for electricity generation. This system was only tried in stationary environments.

6 Conclusion In this paper an example of application of adaptive multi-agent systems to the simulation of economic models was presented. XCS was used to model adaptive firm's decision process. It provides the firm with the capacity to construct a complete model of its environment although a limited access to the information about its rivals. Moreover, XCS allows the firm to reason about the environment to choose the most suited action. The use of XCS meets also the needs of economic researchers as it can help in understanding the adaptation to a changing environment. The aim of our fiiture work is then to improve the XCS learning in very dynamic environments. The first perspective of this work is to control the explorationexploitation activation in XCS. The second perspective however is to enlarge the perception of the firm to take into account the organization structure of the market (organizational forms).

References 1. Bagnall A.J. (2004) A Multi-Agent Model of the UK Market in Electricity Generation. In: Bull (eds) Applications of Learning Classifier Systems, Studies in Fuzziness and Soft Computing 15. 2. Barr J and Saraceno F (2002) A computational theory of the firm. Journal of Economic Behavior and Organization 49:345-361 3. Bruderer E, Singh J.V. (1996) Organizational Evolution, Learning, and Selection: A Genetic Algorithm-Based Model. Journal of Economic Behavior and Organization 39:1216-1231 4. Butz MV and Wilson SW (2001) An algorithmic description of XCS. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Advances in learning classifier systems. Lecture Notes in Artificial Intelligence 2321:253-272 Springer Verlag 5. Chen Y, Su XM, Rao JH and Xiong H, (2000) Agent-based microsimulation of economy fi-om a complexity perspective. In: Gan R (eds) The ITBM2000 Conference - Information Technlology for Business Management. Beijing, China 6. Durand R, Guessoum Z (2001) Competence systemics and survival, simulation and emperical analysis. Competence 2000 Conference. Helsinki, Finland 7. Guessoum Z, Briot JP (1999) From active objects to autonomous agents. IEEE Concurrency 7:68-76

64

Lilia Rejeb and Zahia Guessoum

8. Holland JH (1991) Artificially adaptive agents in economic theory. American Economic Review 81: 265-370 9. Pajares J, Hermandez-Iglesias C, Lopez-Paredes L (2004) Modeling learning and RD in innovative environments: a cognitive multi-agent approach. Journal of Artificial Societies and Social Simulation 7: 10. Peres-Uribe A, Hirsbrunner B, (2000) The risk of Exploration in multi-agent learning systems: a case study. Proc. Agents-00 Joint workshop on learning agents, Barcelona, June 2000. 11. Penrose ET (1959) The theory of the growth of thefirm.Basil Blackwell 12. Rivero SLDM, Storb BH, Wazlawick RS (1999) Economic Theory, Anticipatory Systems and artificial Adaptative Agents. BEJE (Brazilian electronic Journal of Economics) 2: 13. Schulenburg S, Ross P (2000) An Adaptive Agent Based Economic Model. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning Classifier Systems: From Foundations to Applications. Springer Verlag. Lecture Notes in Artificial Intelligence 1813: 265-284 . 14. Sutton RT, Barto AG (1998) Reinforcement Learning: An Introduction. IT Press, Cambridge, MA 15. Tesauro G, Kephart JO (2002) Pricing in Agent Economies Using Multi-Agent QLeaming. Autonomous Agents and Multi-Agent Systems 5: 289-304 16. Wilson SW, (1994) ZCS: A Zeroth Level Classifier System. Evolutionary computation 2: 1-18 17. Wilson SW, (1995) Classifier fitness based on accuracy. Evolutionary computation 3: 149-175 18. Wilson SW, (2000) State of XCS classifier system research. LNAI1813: 63-83 19. Yildizoglu M. (2001) Modeling Adaptive Learning: RD Strategies in the Model of Nelson and Winter. In: DRUID's Nelson and Winter Conference. Aalborg, Danemark 20. Yildizoglu M, (2000) Connecting adaptive behaviour and expectations in models of innovation: The Potential Role of Artificial Neural Networks. European Journal of Economics and Social Systems 15: 21. Zadeh L, (2001) A new direction in AI: Toward a computational theory of perceptions. AI Magazine 22: 73-84

Firm Size Dynamics in a Coumot Computational IVIodel Francesco Saraceno^ and Jason Barr^ ^ Corresponding Author. Observatoire Frangais des Conjonctures Economiques. 69 Quai d'Orsay, 75007 Paris. Tel: +33 1 44 18 54 93. Fax +33 1 44 18 54 88 [email protected] ^ Rutgers University, Newark. jmbarr@rutgers . edu

1 Introduction This paper exploresfirmsize dynamics, with thefirmmodelled as a type of artificial neural network (ANN). Twofirms/networkscompete at two different levels. The first level, which has been explored in detail in other work (Barr and Saraceno (BS), 2004; 2005), looks at Coumot competition between two neural networks. In this paper, this level of competition is essentially in the background, while the main form of strategic interaction is in regards to firm size dynamics. Thefirm,while playing the repeated Coumot game, has to make long mn decisions about its size, which affects not only its own profits, but those of its rival as well. Our previous research showed that firm size, which was left exogenous, is an important determinant of performance in an uncertain environment. Here we reverse the perspective, taking as given both the leaming process and the dependence of firm profit on size and environmental complexity, and endogenize firm size in order to investigate whether simple adjustment mles succeed in yielding the optimal size (defined as the result of a best response dynamics). The computational requirements needed to discover the optimal network size may be quite expensive for the firm, thus we explore two simpler adjustment mles. Thefirst("the isolationist") has thefirmadjusting its size simply based on past profit. An "imitationist" firm, instead, adjusts its size if the rival has larger profits. We also study thefirmdynamics resulting from a combination of the two. To our knowledge, no other paper has developed the issue of long mnfirmgrowth in an adaptive setting. Our mainfindingsare: •

In the isolationist case firm's long mn size is a nonlinear function of both environmental complexity, initial size, rival's initial size and the adjustment rate parameter. Application of this simple mle yields an inverse relationship between long mn size and environmental complexity.

66 • •

•

Francesco Saraceno and Jason Barr In the imitationist case, the "drive to imitation" has to be large enough for the two firms to converge to the same size, higher complexity is associated with more instability and higher long runfirmsizes. Via regression analysis we measure the effects initial conditions on long run dynamics. Wefindthat own initial size is positively related to long run size, rival's initial size has a negative effect for small initial size, but positive effect for larger initial sizes. The adjustment parameters, are positively related with long run size. Finally, we show that our simple dynamics very rarely converge to the 'best response' outcome. The isolationist parameter seems to be the only one that plays a role in guaranteeing such a convergence.

Our paperfitswithin the agent-based literature on information processing (Chang and Harrington, forthcoming), that models organizations as collections of agents that process data. As no single agent can process all the data necessary for modem corporations to function, this creates the need for managers with the task of information processing (Chandler, 1977; Radner, 1993). Typical models are concerned with the relationship between the structure of the network and the corresponding performance or cost (DeCanio and Watkins, 1998; Van Zandt, 1998). In our paper the network tries to map signals about the economic environment to both demand and its rival's output decision. Unlike other information processing models, we explicitly include strategic interaction: a firm's ability to learn affects the competitor pay-off Thus a firm must adapt its structure to maximize efficiency in learning, and to respond to its rival's actions. Firms in our paper employ simple adjustment rules, routines, that are satisfactory rules-of-thumb alternatives to computationally expensive optimizing behaviors (Simon, 1982; Nelson and Winter, 1982).

2 Neural Networks as a Model of the Firm In BS (2002) we argued that when focussing on the information processing task of firms, computational learning theory may give usefiil insights and modelling techniques. Among the many possible learning machines, we focussed on Artificial Neural Networks as models of the firm, because of the intuitive mapping between their parallel processing structure andfirmorganization. Neural networks, like other learning machines, can generalize from experience to unseen problems, i.e., they recognize patterns. Firms do the same: the know-how acquired over time is used in tackling new, related problems (learning by doing). A specific feature of ANNs is the parallel and decentralized processing. ANNs are composed of multiple units that process relatively simple tasks in parallel, resulting in the ability to process very complex tasks^. In the same way firms are often composed of different units working autonomously on specific assignments, and coordinated by a management that merges the results of these simple operations in order to design complex strategies. ^ We discussed the details of ANNs in BS (2002, 2004, 2005); for an extensive treatment of the subject the reader is referred to Skapura (1996)

Coumot Firm Size Dynamics in a Coumot Computational Model

67

The parallel between firms and learning algorithms also shows in the trade-off linked to complexity. Small firms are likely to attain a rather imprecise understanding of the environment they face; but they act quickly and are able to design decent strategies with small amounts of experience. Larger and more complex firms, on the other hand, produce more sophisticated analyses, but they need time to implement their strategies. Thus, the optimal firm structure may only be determined in relation with the environment, and it is likely to change with it. In BS (2002, 2004, 2005) we showed, by means of simulations, that the tradeoff between speed and accuracy generates a hump-shaped profit curve in firm size. We also showed that environmental complexity and optimal firm size are positively related. These results reappeared when we applied the model to Coumot competition. Here we leave the learning process in the background, and focus on endogenous changes of network size.

3 Setup of the Model The background neural network model is taken fi*om BS (2004, 2005). Two firms competing in quantities face a linear demand function whose intercept is unobserved. They observe a set of environmental variables that are related to demand, and have to learn the mapping between the two. Furthermore, firms have to learn their rival's output choice. BS (2005) shows that in general firms/neural networks are able to learn how to map environmental factors to demand, which allows them to converge to the Nash equilibrium. We fixrther showed that the main determinants of firm profitability are, on one hand, firm sizes (i.e., the number of processing units of the two firms, mi and 777-2); and on the other environmental complexity, modelled as the number of environmental factors affecting demand (n). These facts may be captured by a polynomial in the three variables: TTi = / ( m i , 777,2, n)

z = 1,2.

(1)

To obtain a specific numerical form for equation 1, we simulated the Coumot learning process with different randomly drawn firm sizes (777,1,777-2 ^ [2,20]) and complexity (n e [5,50]), recording each time the profit of the two firms. With this data set we ran a regression, that allows to obtain the specific polynomial relating profits to size and complexity:"* TTi = 271 + 5.93mi - O.SSmj + O.OOTmf + 0.497712

(2)

- 0.377117712 - 2.2n + 0.0033ri^ + 0.007mim2n - 0.016m2n. Figure 1 shows the hump shape of profit with respect to own size discussed above. We report three curves, corresponding to small, medium and large opponent's size. ^ In previous work other variables affected profit; here we hold them constant. The appendix details the process yielding the profit equation (table 3); notice that the setup is symmetric, so that either firm could be used.

68

Francesco Saraceno and Jason Barr

240 profl

10 ml 12

14

16

18

20

Fig. 1. Firm 1 profit vs. size (1712 = 2, solid line; m2 = 10, crosses; m2 = 20, diamonds). Complexity isfixedat n = 10

4 The Best Response Function and Network Size Equilibrium In this section we discuss the firm's best response functions and the corresponding equilibria. Given the functional form for profit of equation (2), we can derive the best response function in size by setting the derivative of profit with respect to size equal to zero; this yields the following solution for firm i: m^''{m-i,n)

= 16.9 ± 2.26^2.6m_i - 0.058nm_i + 3.9

The 'best response' function is polynomial in m_j and generally has more than one solution (real and/or complex). Nevertheless, for values of m-i and n in the admissible range (rrii G [2,20], n G [5,50]), the solution is unique and decreasing. The Network Size Equilibrium (NSE) is given by the intersection of two firms' best responses (figure 2); these equilibria are stable as, in spite of the complexity of the profit function, the best response is quasi-linear. Notice that increasing complexity shifts the best response functions and the NSE upwards. Optimal firm size, m | , is a fiinction of environmental complexity. Exploiting symmetry, we have (figure 3): ml=m*2

= m* = 23.5 - 0.15n - 4.5x/(14.1 - 0.34n + 0.001n2)

(3)

If we substitute the optimal value given by equation (3) into the profit equation (2), we obtain a decreasing relationship between profits and equilibrium firms size (also plotted in figure 3). To conclude, the best response dynamics yields a unique and stable equilibrium. Furthermore, we were able to show that firm size in equilibrium is increasing in complexity, while profit is decreasing. In the next section we turn to simpler firm size adjustment rules, more realistic in that they require lower computational capacity for the firm.

Coumot Firm Size Dynamics in a Couraot Computational Model

7

rife

9

10

11

69

12

Fig. 2. Best response functions (n = 10, diamonds; n = 25, crosses; n = 40, solid lines) for firms 1 and 2. The Network Size Equilibria are given by the intersection of the lines

12

)tf 11 10

Fig. 3. Profit (solid line, left axis) at equilibrium is decreasing in environmental complexity. Equilibrium size (crosses, right axis) is increasing

5 Adaptive Adjustment Dynamics As discussed above, firms often face a large amount of information to process. The amount of information firms are assumed to possess in standard economic theory, is remarkable: cost functions, profit functions, and the effect of a rival's decisions on profits. If we depart from the full information case, the cost to obtain such knowledge may be significant. This is why firms engage in routines and adaptive behavior. The best response function in section 4 is quite complex, and assumed that the firm knows it; more specifically, it is assumed to know the expected maximal profit obtainable for the entire range of a rival's choice of network size. In addition, in a world in which production and market conditions constantly change, past information may quickly become irrelevant. Meaning that even if a firm has perfect knowledge of its

70

Francesco Saraceno and Jason Barr

best response function at a certain point in time, that function may quickly become outdated. Even when the firm is in possession of the computational capabilities necessary to compute the best response, it may be not efficient to actually do so. In this section, we explore relatively simple dynamics, that only assume the knowledge of firm's profits and their opponent's. We explore adjustment dynamics following a general rule-of-thumb rule, using the profit function generated in section 3. rui^t = rrii^t-i+P {'7^i,t-i - 7Ti,t-2)-\-OiIi [(m_i,t-i - m^,t-l)(7^_^,t_l - 7Ti^t-i)] • (4) P represents the sensitivity of firm size to own profit growth. A firm experiencing profit growth will increase its size by P {T^i,t-i — '^1,1-2) units. The parameter a captures the "drive to imitation"; U is an indicator function taking the value of 1 if the opponent's profit is larger than the firm's, and a value of 0 otherwise:

'

^ ri<^(7r_i,t_i-7ri,t-i) > 0 \ 0 4^(7r_,,t-i-7r,,t-i)<0

If /i = 1, then firm size will be adjusted in the direction of the opponents'. Thus, the firm will adjust towards the opponent's size, whenever it observes a better performance of the latter. To sum up, our adjustment rule only uses basic routines: first, the firm expands if it sees its profit increasing; second it adapts towards the opponent's size whenever the latter is doing better. These routines require very little observation and computation on the part of the firm. In short, we can think of the Coumot game as happening on a short term basis, while the adjustment dynamics occurs over longer periods of time. In the next sections we investigate what kind of firm dynamics can these simple adjustment rules yield, and under what parameter values will firms reach the equilibrium level presented in figure 3.

6 Results Scenario 1: The Isolationist Firm Suppose that a = 0. Then each firm will only look at its own past performance when deciding whether to add or to remove nodes: rrii^t = mi^t-i + P[jTi^t-i{rni^t-i,rn-i^t-i,n)

- 7ri,t-2(^i,t-2,m_i,t_2,n)]

This does not mean that firm dynamics are independent of each other, as TVi^t depends on both sizes. Figure 4 shows the size dynamics of two isolationist firms for different complexity levels. The dynamics are relatively simple and show a few things. First, the long-run level is reached fairly quickly. Then, this level seems to depend on initial own size. Both the opponent's initial size and complexity seem to have a very limited effect, if

Coumot Firm Size Dynamics in a Couraot Computational Model

71

30 -

/ a; 25 -

a fe 20 N*«s 15 -

^^

..y^

•^^^-^^_^^^^-^^* - — - « _ - 2 ^ _ _ n=5 n=20 - - - • n=40

/

10 -

Fig. 4. Firm dynamics when thefirmsstart at different sizes (mi(0) = 10; m2(0) = 20), for three different levels of complexity. /3 = 0.05

any. These qualitative features do not change for different initial sizes (other figures available upon request). The result nevertheless is not robust. The statistical analysis carried on below shows that m_i(0), n and ^ have significant effects on long run size, that in figure 4 are hidden by the predominant effect of initial size. Scenario 2: The Imitationist In this case the contrary of the isolationist firm holds: firms do not care about their own situation, but rather about the comparison with the opponent: /? = 0. Thus, rrii^t = rrii^t-i + ce^z [(m_i,t-i

M,t-i )(7r. i,t-l

^i,t-l

)].

The firm will adjust towards the opponent if it has a smaller profit. Thus at each period only one firm moves, and at the final equilibrium the two profits must be equal, which happens when firm sizes are equal but not necessarily only in this situation. For n = 5, suppose one firm begins small and the other large (mi = 4 and 7712 = 15). The resulting dynamics depend on a, as shown in figure 5. For low levels of a the drive to imitation is not important enough, and the two firms do not converge to the same size. For intermediate values (around a = 0.125), instead, firms converge to an intermediate size. When a is too large, on the other hand, the initial adjustment is excessive, and may overshoot. Complexity has a role as well (figure 6, where n = 40). In fact, as greater complexity yields larger firm size, we see that the adjustment is faster (for each given a), and that for large enough values of a, the system explodes. Both in the imitationist and in the isolationist cases, the dynamics show a very strong dependence on initial conditions. This feature of the time series calls for a systematic analysis of the parameter space and we present below regression results for the combined scenario.

72

Francesco Saraceno and Jason Barr

r^^ - - - m2

5

10

15

20

10

15

20

25

30

5

10

15

20

25

30

10

15

20

25

30

1 - - - m2

25

30

Fig. 5. Firm dynamics for different values of a n = 5

Scenario 3: Combined Dynamics. Regression Analysis In this section we combine the two t5^es of dynamics, (a,/3 ^ 0. To do this we generate a dataset with random parameter values, mi{0) G {2,20}, a e [0.025,0.075], P G [0.025,0.075] and n e {5,10, ....,40}. Then we record the long run steady state firm size for firm 1, which is our dependent variable. The re-

10

15

20

25

30

a=0.175

/T ml - - - m2

f 10

15

20

25

30

10

15

20

25

Fig. 6. Firm dynamics for different values of a n = 40

30

Coumot Firm Size Dynamics in a Coumot Computational Model

73

gression equation (table 1) is non-linear, and includes several interaction terms. We also included a dummy variable, S [mi (0) > m2 (0)] = 1 if mi (0) > m2 (0) and 0 otherwise, since we found it to be statistically significant. Table 1. Regression results for adjustment dynamics. All variables stat. sig. at 99% or greater confidence level Dependent Variable: mi (100) Variable mi(0) [mi (0)]^ m2(0)

Coef. 0.517 0.009 -0.289 0.031 K(0)]^ m i (0) • m2 (0) -0.020 Nobs. 5000

Variable Coef. (3 159.342 Q:-mi(0) -3.110 S [mi (0) > ms (0)] 0.447 a 58.159 Constant 3.157 R^ 0.878

Variable p • m i (0) /?-m2(0) n n-f3 n-mi(O)

Coef. -2.042 3.907 -0.040 1.401 0.006

From the regression table we can draw the following conclusions: Increasing initial own size increases long run size, whereas the opponent has different effects (negative for low values of m2 (0), and positive for large values). a and P have positive effects onfirmsize, but interact negatively with initial size. Relative initial size (the dummy variable) has a positive effect. If firm 1 starts larger thanfirmtwo, it will have a larger long run size. Increasing environmental complexity has a negative effect. The reason is that it reduces profits and thus long run size. Several interaction effects capture the non-linear relationship between the independent variables and long run firm size. For example, both initial own size and the adjustment parameters have positive effects, but there are also off-setting interaction effects: ami(O) and f3mi{0) both have negative coefficients.

7 Convergence to Nash Equilibrium The previous section shows that the long run size of thefirmis determined by several variables; the convergence to the Nash equilibrium is not guaranteed by the simple adaptive dynamics that we study in this paper.This section investigates the conditions under which it takes place. We made random draws of the relevant parameters (a, (3 G [0,0.2], mi{0) e [2,20]), and we ran the dynamics. Then, we retained only the runs that ended with bothfirmswithin one node of the Nash value (i.e., mi(50) G [m* — 0.5, m* + 0.5].) That was done one million times for each complexity value nG {5,10,...,45}. Table 2 reports the success rate, the average a and P, the initial mi, and its mode. The first result that emerges is that only a very small number of runs converged close to the NSE. The success ratio does not even attain half of a percentage point. The value was particularly low for intermediate complexity values. Thus, we find

74

Francesco Saraceno and Jason Ban-

Table 2. Convergence to Nash equilibrium for different complexity levels. Standard deviation in parentheses n

(m*)

Succ.

5

(6.85)

0.492%

10

(7.19) 0.413%

15

(7.58) 0.296%

20

(8.01) 0.190%

25

(8.53) 0.207%

30

(9.15) 0.247%

35

(9.91) 0.293%

40

(10.93) 0.354%

45

(12.50) 0.356%

a 0.122

/3 0.082

mi(0) 8.758

(0.052)

(0.042)

(5.380)

0.115

0.087

7.897

(0.057)

(0.056)

(5.267)

0.093

0.082

4.137

(0.056)

(0.075)

(1.476)

0.106

0.009

4.845

(0.053)

(0.007)

(1.792)

0.106

0.011

4.972

(0.054)

(0.007)

(1.896)

0.105

0.012

5.408

(0.053)

(0.008)

(2.137)

0.105

0.014

5.700

(0.053)

(0.010)

(2.347)

0.110

0.016

6.304

(0.050)

(0.011)

(2.583)

0.119

0.021

6.967

(0.048)

(0.014)

(2.948)

mod[mi(0)]

that even extremely simple and commonsensical adjustment rules, while allowing for convergence to a steady state value, do not yield the equilibrium 'full information' outcome. This result calls for a careful assessment of the conditions under which a full information outcome m a y be seen as plausible (the 'as i f hypothesis). The mode and mean of initial size do not change significantly with complexity. Thus the increase in complexity, and in the associated Nash size, does not require larger initial firms for convergence to happen. In fact, the only variable that seems to vary significantly is (3. Increasing complexity requires increasing reactivity to own profit to yield convergence to NSE.

8 Discussion and Conclusion This paper has presented a model of the firm as an artificial neural network,and explored the long run size dynamics of firms/neural networks playing a C o u m o t game in an uncertain environment. We looked at long-run firm size resulting from two types of simple adaptive rules: the 'isolationist' and the 'imitationist.' These dynamics were compared to a benchmark 'best response' case. First we found that when using simple adjustment rules, long run firm size is a function of initial firm size, initial rival's size, environmental complexity and the adjustment rate parameters. These variables interact in a non-linear way. We also found that only under very precise initial conditions and parameter values does the firm converge to the N S E . The reason is that the dynamics we consider tend to settle rapidly on a path that depends on initial conditions. The simple rules generally yield suboptimal long run equilibria, thus suggesting cautiousness in taking the Nash equilibrium as a focal point of simple dynamics. We further find that when firms use simple adjustment rules, environmental complexity has a negative effect on size.

Coumot Firm Size Dynamics in a Coumot Computational Model

75

This is explained by the negative correlation between profits and complexity. More efficient information processing and more complex adjustment rules, would play a positive role in the long run profitability of the firm, and would deserve investment of resources.

Appendix. Derivation of the Profit Function This appendix briefly describes the process leading to equation 2, that in the present paper is left in the shadow. Cournot Competition in an Uncertain Environment Two Coumot duopolists face the demand function p^ = jf — (qn + q2t). Assume that production costs are zero. Then, the Nash equilibrium is q"^^ = 7t/3. Firms do not observe 7^, but know that it depends on a set of observable environmental variables x G {0,1}^:

^

k=l

Each period, thefirm/neuralnetwork views an environmental vector x and uses this information to estimate the value of 7 (x). Note that 7(xt) can be interpreted as a weighted sum of the presence or absence of environmental features. To measure the complexity of the information processing problem, we define environmental complexity as the number of bits in the vector, n e [5,50]. Thus, in each period: 1. Firms/networks observe a randomly chosen environmental state vector x, and use it to estimate a value of the intercept parameter, 7^. They also estimate the rival's choice of output, g!_^,. 2. Firms then observe the true value of 7, and q-i^ and use this information to determine the errors, su = {'ji — 7)^ and 621 = {q'Li — q-i) • 3. Based on these errors,firmsupdate the weight in the network. This process repeats for a number T = 250 of iterations, and the average profit IS

1 ^

j;Z^Qit{7t

- {qit + q2t))'

t=i

Regression Results for Profit Equation (2) was derived by using the model described in the preceding appendix. We built a data set by making random draws of n G [5,50], rrii e [2,20]. We ran the Coumot competition process for T = 250 iterations. We recorded average profit for the twofirms,and the values of mi, 7712, and n. This was repeated 10,000 times, in order to obtain a large data set. We then ran a regression to obtain a precise polynomial form for profit as a function of sizes and environmental complexity. Table 3 gives the complete results of the regression, which is reflected in equation (1).

76

Francesco Saraceno and Jason Barr

Table 3. Profit function for firm 1. All coefficients stat. sig. at 99% or greater confidence level Dependent Variable: 10,000 • TTI. Variable

Coef.

constant 270.6 nil

m\

ml Nobs.

5.932 -0.375 0.007 10,000

Variable Coef. m2 0.490 mi • 7712 -0.304 n -2.201 r B? 0.864

Variable

Coef. 0.003 mi •ni2 • n 0.007 -0.016 77l2 • n n

References 1. Barr, J. and Saraceno, F. (2005). "Coumot Competition, Organization and Learning." Journal of Economic Dynamics and Control, 29(1-2), 277-295. 2. Barr, J. and Saraceno. F. (2004). "Organization, Learning and Cooperation." Rutgers University, Newark Working Paper #2004-001. 3. Barr, J. and F. Saraceno (2002), "A Computational Theory of the Firm," Journal of Economic Behavior and Organization, 49, 345-361. 4. Chandler, Jr., A. D. (1977). The Visible Hand: The Managerial Revolution in American Business. Harvard University Press: Boston. 5. Chang, M-H, and Harrington, J.E. (forthcoming). "Agent-Based Models of Organizations." Handbook of Computational Economics, Vol 2. Eds K.L. Judd and L. Tesfatsion. 6. DeCanio, S. J. and W. E. Watkins (1998), "Information Processing and Organizational Structure," Journal ofEconomic Behavior and Organization, 36, 275-294. 7. Nelson, R. R. and Winter, S. G. (1982). An Evolutionary Theory of Economic Change. Belknap Press of Harvard University Press: Cambridge. 8. Radner, R. (1993), "The Organization of Decentralized Information Processing," Econometrica,6\, 1109-1146. 9. Simon, H. A. (1982). "Rational Choice and the Structure of the Environment." Psychology Review. 63(2): 129-138. Reprinted in: Behavioral Economics and Business Organizations. The MIT Press: Cambridge. 10. Skapura, D. M. (1996), Building Neural Networks, Addison-Wesley, New York. 11. Van Zandt, T. (1998). "Organizations with an Endogenous Number of Information Processing Agents," In: M. Majumdar (Ed.) Organizations with Incomplete Information. Cambridge University Press: Cambridge.

Case-Studies and Applications

Emergence of a Self-Organized Dynamic Fishery Sector: Application to Simulation of the SmallScale Fresh Fish Supply Chain in Senegal. Jean Le Fur Institut de Recherche pour le Developpement Centre de Recherche Halieutique Mediterraneenne et Tropicale (CRH) Av. Jean Monnet, BP171, 34203 Sete Cedex, France [email protected]

1

Introduction

The artisanal fishery sector in Senegal is a complex system of fishermen and fish traders acting in close interaction. Indeed, in the overall marine fishery sector, several ethnic groups with different behaviors target more than a hundred fish species using nineteen types of fishing gear (Laloe and Samba 1990). Once fishermen land their catches, another complex set of fish traders is in charge of the food product distribution (Chaboud 1985). Seafood can then be sold on the beach, processed, transported to the various markets of the country or brought to Dakar, the capital, for export. The overall dynamics observed are the result of composite biological, technological and socio-economic interactions. A historical review of this fishery sub-sector (Chauveau and Samba 1990) pointed out that management changes introduced in the small-scale fishery more often than not led to unexpected effects. Indeed, some management measures introduced for a given purpose often led to undesirable consequences on other parts of the exploitation that were not concerned by the given measure. The system appears to be an archetype of complexity. From a management perspective, explanations may be sought for the conditions under which such a complex sector achieves organization and stability, despite the multiple dependencies existing between the different components. Answering such a question could help to better depict the possible derived consequences of a given management measure introduced into this sector. Studies and models on trade/market/price systems of self-organization and equilibrium are usually detailed insights which consider one or a few aspects of the market or chain dynamics such as price fixing (Gale 1987, Donangelo et al. 2000, Calvo-Armengol 2003, Zhu 2004), trade-offs and negotiation mechanisms (Faratin et al. 2002, Lemke 2004), interaction schemes and the network configuration of markets (Guriev and Shakhova 1996, lori 2002, Vriend 2004), decentrali-

80

Jean Le Fur

zation effects (Calvo-Armengol 2003), multiple levels of exchange such as in supply chain models (Kaihara 2001, Dangelmaier et al. 2002), and the interaction between prices and consumption (Nagel et al. 2004). In a world like the fishery sector, a given market can be considered as a local phenomenon embedded and connected in a variety of others, all of which together constitute the overall sector dynamics. This means that all of the cited aspects of the dynamics may be of importance and should simultaneously play a role in the overall dynamics. Moreover, the studies presented are usually described using abstract situations where localization and distances are not considered. In the real world of the fishery trade sector, transport costs are clearly a significant part of the fish price fixing as well as choices for a given place or another constitute key factors of the supply and demand dynamics. Furthermore, the resource-harvesting dynamics should clearly influence the dynamics of supply and fish market prices. Again, these multiple factors all intervene simultaneously at different levels and scales. The question then remains of the ability and means for a complex trading sector to converge, in such a context, towards self-organization and exhibit any equilibrium or steady-state. A multi-agent computer model restricted to the fresh fish supply chain in Senegal has been developed to study this question. The model is based on the representation of the agents' choices, actions and interactions. The multi-agent formalism has been used since it easily permits modeling of decentralized processes as well as studying adaptive or emerging phenomena (Railsback 2001). These latter are indeed felt to be the possible keys for the emergence of self-organization or equilibrium. Moreover, multi-agent formalism looks very suitable for studying systems where negotiation and multi-criteria functions play central roles (Oliveira et al. 1999).

2

Model Presentation

2.1

Outline

The model is based on a cybernetic (Ashby 1964) perception of the domain. In this approach, the fishery sector is considered as a set of four interconnected networks within which circulate money, fish, human agents and information. These flows may be interconnected at some points where matter can be exchanged (e.g., money exchanged for fish, activity for money). From this viewpoint, looking for a sustainable exploitation of Senegalese resources may consist in maintaining these flows. These overall dynamics are formalized at a finer scale using a diversified combination of local actions and interactions: The human groups in charge of the exploitation (fishermen, fish traders) constitute the concrete link between biological.

Emergence of a Self Organized Dynamic fishery Sector

81

technical, economic and social dynamics. The agents are endowed with various behaviors allowing them to obtain information from their environment, make choices about it and produce several actions. For these agents, the ability to respond to changes in their environment hinges on their ability to choose from one alternative to the other, and their ability to negotiate with other agents. The combination of the different actions produces flows of fish, currencies, human agents and activity and, finally, the overall dynamics (production, richness) of the fresh fish channel.

2.2

Class Structure

To investigate this composite issue, the object-oriented design of the multi-agent model leads to a class hierarchy where each sub-class 'is-a-kind-of its superclass. The classes that have been retained in the model of the Senegalese exploitation are presented in Fig. 1. FISHING COMMUNITY:

I

Community -|

Act ive — (agents) Passive-

Fislitrader Fislier'^ Consumer Factory

memory community size species (sp) money/sp quantity/sp cost/sp consumption/sp equipment current place known places confidence/place

artisanal fishe-ycomponent

• Living Fisli StGcl< LIVING FISH STOCK: memory species site biomass

MARINE PLACE: memory communities stocks species substrate type temperature

FISHING GEAR: loading capacity species harvested Icatchability/sp

Legend: CLASS NAME \ pointer field valuated field

Fig. 1: Computer constituents (classes) selected to formalize the domain. Grey boxes show examples of the object characterization for each corresponding class (note: the "memory" field refers to a separate storage class not figured in the hierarchy) The overall fishery exploitation is composed of four main classes: the communities conducting the various flows, the knowledge they can access for this purpose, the places in which they operate and the living fish stocks they harvest. These major classes are divided into more specialized ones to obtain a sufficient (parsimonious) description of what composes the fishery exploitation. In each of the eleven terminal classes, several objects are differentiated. Each object in a given class is given a set of attributes that are defined by the class to which it be-

82

Jean Le Fur

longs (four examples are presented in Fig. 1). The value of these objects' attributes document either the relationships with other objects (pointer fields) or specific characteristics (valuated fields). 2.3

Functional Representation

Upon this structure a multi-agent formalism, as described in Ferber (1999), has been developed. The Active-Community class contains the active agents (fishermen and fish traders) of the food supply chain. These objects can elaborate information and produce actions through the nesting of messages sending and replies : In this model for example, if a fishing community needs to know the traders' demand for a given species in a given port, it sends the corresponding message to the port's agent. The port sends the message "species' needs" to each of the fishtraders currently in this place. Each fish trader then conducts an internal evaluation of its requirements for this species. It replies by sending a message back to the port. The port cumulates the answers and after a compilation can reply back to the fishing community. At a higher level of combination some basic activity of the various agents in the exploitation can be formalized. The example in Fig. 2 represents fishermen's actions once they have gone fishing and come back to land their catches:

1

Go fishing

Go to port

t

1^

inaease confidence for current place

change place (spend moving costs) Look for buyers sell fishes

memorize richness estimate consumption species 2u (demandj. price j)

i

Fig. 2: Flow chart of the 'go to port' action performed by a fishing community agent (e.g. after fishing). Each box corresponds to an 'order' (set of messages) sent by the fishing community to itself Each fisherman agent arriving in a port tries to sell its fish. If it succeeds, it memorizes the results of its action and then stands by. The next time it will have to act, it will choose to go to sea. If the transaction does not succeed, because there are no fish traders for its fish or the negotiation did not succeed, fishermen look for another port (action 'changeport') using a decision process sub-model derived from

Emergence of a Self Organized Dynamic fishery Sector

83

Le Fur (1995, 1998): according to the information it can gather from other objects and agents, the community elaborates a set of alternatives to where it can go. For each of these places, it evaluates the opportunity related to several criteria. In this case, the opportunity to go to one port or the other will depend on traders' demand, species prices, transport costs, confidence for one or the other place. After comparing the opportunity for each of the alternatives, whether it finds a better place to sell its fish and then goes to the new port where the whole process starts again, or it does not find a better place, and stands by for the next fishing trip. If, during its standby, fish trader agents happen to arrive in this port, selling may occur. To account for the whole exploitation activity, four similar processes have been formalized: two for the fishermen agents: 'go fishing' and the 'go to port' just described, and two for the fish traders' agents: 'go to port' and 'go to market'. At each time step, each active agent, fisherman or fish-trader has to choose from one of its two moves. Going to the same place is allowed as a stand-by. Depending on its stocks (fish, money) and the result of its preceding choices, it goes to a given place whose characteristics conduct it to a specific 'action'. For example, a fisherman arriving at a fishing zone fishes, a fish-trader in a port tries to buy fish, etc. Any action conducted leads to a subsequent 'evaluation' of its result. This evaluation may lead to change or no change in the next action aimed at by the active agent. Interaction and Transaction

When two communities, a buyer's and a seller's, happen to meet in a given place, negotiation may occur, followed or not by a transaction. A sub-model has been developed to formalize this mechanism. Transaction sub-model: Since in Senegal, bargaining is the rule for exchange, the transaction sub-model represents selling by private contract between the different communities: at the beginning of the transaction, the selling community (fisherman, fish trader) obtains information from its surroundings, evaluates the cost caused by the previous activity (moving, buying, fishing) and proposes its price. The buyer (fish trader, customer) considers its previous costs or needs and puts forth its own proposition. The final price of the transaction will be a value between the seller's lowest price and the buyer's highest price. In decision theory, given A a set of acts, E the possible states of an environment, the possible consequences (Q can be described through a probability distribution. A rule of thumb (e.g., Charreton and Bourdaire, 1985) establishes the possible mean of this distribution (i.e., final price) as: Vs • [maximum of the distribution + minimum of the distribution + mode]. Following this scheme, the transaction price will be, for example in a port: Vs • [fishermen's price + traders' price -^finalprice in the port during the last transaction concerning this species]. If the final price is acceptable to both partners, the transaction occurs and the price changes in the port. In a given time step, the evolution of the traders' arrival in the port and their successive transactions generates the port's fish prices dynamics. These fluctuations will

84

Jean Le Fur

again intervene in the agents' choices. This procedure is duplicated in the market places where transactions occur between fish traders and consumers. Since the agents may own fish and currencies, their moves lead to the activation of the various flows constituting the exploitation. Moreover, depending on the place where an agent operates, it can come into interaction with other agents and decide whether or not to exchange matter (work into fish, fish into currency, etc.). In this way, the accuracy of the moves and of the interactions will be a condition for an accurate subsequent action. 2.4

Simulation Process

Simulation scenarios are built using data available in the literature (Chaboud 1985, Chaboud and Kebe 1990, CRODT 1989, 1990). The whole Senegalese exploitation has first been instantiated and led to a scenario with a system composed of 126 fishing community agents accounting for 3193 fishing teams, 1421 traders agents, 14 markets, 9 ports, 13 fishing zones, 5 fishing gears types, 6 vehicle types and 21 types of fish species. For practical reasons, simulations have been conducted with subsets of this configuration. In this study, the scenario accounts for the North coast of Senegal with only gillnets and lines, 2 ports and 8 markets. At the beginning of the simulation, active fish traders (trucks) and fishermen (canoes) are positioned on their current places. Each community agent is given 45.000 CU for 15 days (with CU: currency unit such as 1 CU approximates to 1000 Senegalese CFA Francs). The simulation then proceeds step by step with one time step equivalent to a fortnight. At each time step the external sources of fluctuation are documented. This consists in making the natural resources produce fish on one side of the system and providing consumers with money and consumption needs on the other side. Each active community is then allowed to produce an action. Depending on their environment, their preceding choices and results, fishermen and fish traders move to one or another type of place, port or market, and try to fish, sell or buy. At the beginning of a simulation and depending on the initial scenario, the communities introduced into the "virtual exploitation" may not fit with the particular environment simulated. For example, a fisherman with bad information will not go where fish traders are waiting, another may look for unavailable species, a fish trader may go to distant markets and incur high transport costs, etc. Depending on their action some communities may thus lose money. If a community, through its activity happens to lose more than 10.000 CU in the 15 preceding steps, it leaves the fishery. The most indebted agents leave the 'virtual exploitation' first and, from time to time, only the fittest communities remain in the exploitation.

Emergence of a Self Organized Dynamic fishery Sector

3

85

Simulation Results

Owing to the structure of the multi-agent system, it is possible to obtain an insight into the various levels of the fishery system dynamics. • At the finest scale, activities, decision processes, negotiations and transactions between agents can be traced precisely. Traces show diversified choices and actions through time and from one agent to the other. The listing in Fig. 3 presents a snapshot of a fishing community during the transaction phase of their activity. 5 pml-ky-2 leave the Kayar_sea and [transport: 2CU] arrive at Kayar Price given for rays by pml-ky-2: .220 Price given for sea bream by pml-ky-2: .13 0 by Kayar-4 0: .043 after negotiation: .097 5 pnil-ky-2 sell (each) 200kg/day of sea bream to 3 kayar-40 and earn 19.4 CU/pers./day Price given for red bream by pml-ky-2 .177 Price given for grouper by pml-ky-2 .488 by Kayar-75 .014 after negotiation: .315 , Kayar-75 refuses transaction. by Kayar-4 0: .3 99 after negotiation: .444 5 pml-ky-2 sell (each) 41kg/day of groupers to 3 kayar-40 and earn 18.2 CU/pers./day pml-ky-2 has no cuttlefish to sell. 5 pml-ky-3 leave the Kayar_sea and at Kayar

[transport: 2CU] arrive

Fig. 3: An example of the simulated transactions. At the local scale, the actions performed by the agents are traced by the computer. In this example, information has been filtered to keep the selling task trace only (explanation given in text) In this example, the fishing community agent named 'pml-ky-2' comes back to 'Kayar' port object after having fished in 'Kayar_sea' marine place object. It tries to sell its 'rays' but no fish trader is interested in this species. Nothing thus happens. The fishermen community then tries to sell its 'sea bream'. Taking into account its previous costs, the quantity owned and the port's price, it proposes a price (.130). The fish traders' agent named 'kayar-40' is interested in buying and makes an offer. After "bargaining" the price is negotiated to .097 and the two communities proceed to the transaction. For the 'red bream' species, fishermen do not find any traders. For the 'grouper' species, the 'kayar-75' traders' agent proposes a very low price (.014). The negotiated price remains too high and the trader refuses the exchange. The former trader's community 'kayar-40' is also interested in the 'grouper' species and here, the negotiation succeeds. At the end of negotia-

86

Jean Le Fur

tion, fish traders ask for 'cuttlefish' species but the fishermen community did not target this species. Nevertheless, this may conduct the community to later choose a fishing place to catch this species. Thereafter another fishing community 'pml-ky3' arrives at 'Kayar' and the process goes on. • At a higher scale, local indicators can be studied. For example, from time to time and depending on the moves of the various communities, supply and demand fluctuates from one site to the other and, through negotiations, the species prices change. The example in Fig. 4 presents an emerging co-evolution of the grouper species price dynamics in a port and its nearest market.

3.5 consumer's prbe at Louga market

O^ CD O

ex-vessel price at port Saint-Louis

2.5

QCD Q-

1.5

O

&.

0.5

simulation time step (fortnights) Fig. 4: Price changes of the grouper species in two related places (a port and a nearby market). Results from a simulation Dependencies appear between the two places. The comparison of temporal changes in prices shows higher prices in the market than in the port. Moreover, fluctuations in the port propagate with a delay in the market. The response curve shows a sharp increase followed by a slow price decay. This corresponds to a period of shortage when the prices increase followed by period of over-supply when price falls as new providers/sellers arrive. Some unexplained fluctuations also appear such as the amplification of the prices observed between t2o-t5o followed by a decrease around tgo- In the model, since the whole set of ports and markets are interconnected, changes occurring in a given place object (e.g., disaffection for a port or a market) can lead to changes in other places thus causing these indirect, and often difficult to figure out, variations. • Finally, at the overall scale, various flows can be monitored in the simulation. The example in Fig. 5 shows a possible evolution of the simulated fishery.

Emergence of a Self Organized Dynamic fishery Sector

87

population size

CU or tons 15000

\ 1200 fish traders pop. size

1000

10000 4

1! i'r^i%

ports' richness (CU)

800

600

5000

400

200

-5000

50

100

150

time step (fortnigiit) Fig. 5: Self-organization of the virtual exploitation (North coast and markets of Senegal with gillnets and lines only). Results from a simulation At the beginning of the simulation, fish landings are greater than the buying capacity of the fish traders. The unsold quantities are too high and fishermen lose money. The exploitation is indebted overall (port's richness). Through time, fishermen who are not able to support the loss leave the fishery (fishermen population size). The fish traders' number remains stable. In the middle of the simulation, the fisherman number reaches a low level until it is fitted to the trading capacity of the fish traders. The exploitation is, from that time, composed of many fish traders with small buying capacities and a few fishermen communities providing the exact demand. The four dynamics become stationary. The unsold quantities tend to zero; the exploitation richness is positive and stable.

Discussion Following the classification by Straskraba (2001), the simulated fishery sector exhibits both self-adaptation (tuning parameters without modifying structure) and self-organization (connecting or disconnecting a diversified set of relationships). Some simulations demonstrated the model's sensitivity to the agents' choice criteria, the order in which the agents act or the initial conditions of the simulations. As these factors modify the dynamics, the results of the simulation cannot

88

Jean Le Fur

be closely related to observed real events; they just display a possible situation arising from an approximation of the real context. The validation of the model is conducted by comparing the activity of the virtual exploitation at each functional level with observed patterns: fishermen go fishing, fish traders arrive in the right market at the right time, the negotiation process is rational and reliable compared to what is known of the bargain process, prices are realistic compared to those observed, related places co-evolve, etc. The simultaneous consistency of the dynamics observed at various levels is a factor that reinforces confidence in the emergent dynamics observed.

5

Conclusion

In the course of a simulation, the communities' objectives change depending on their various activities (fishing, selling, moving, buying, consuming). From one objective to another, from one type of community to another and from one environment to another, the decision processes lead to different choices. The resulting sum of the activities modifies the context (i.e., environments) through time and, by feed-back, influences the various evaluations processed by the agents. Even in such a complex multivariate system there exist some combinations for which the system is sustainable. The agents' diversity and the multiplicity of their local action provide a large degree of freedom to the multi-agent system. This feature can contribute to make sustainable combinations available. Moreover, when associated with a simple process of agent fitness selection, this 'distributed-diversity' feature also provides the ability for the simulated system to converge autonomously towards a correct parameter combination.

References Ashby WR (1964) Introduction to Cybernetics. Methuen, New York Calvo-Armengol A (2003) A decentralized market with trading links. Mathematical Social Sciences 45:83-103 Chaboud C (1985) Le mareyage au Senegal. CRODT-ISRA, doc Sci 87, 112p Chaboud C, Kebe M (1990) Commercialisation du poisson de mer dans les regions interieures du Senegal (donnees statistiques). CRODT-ISRA, contrat FAO 695 TCP/SEN/6653(t), septembre 1990, 300p. Charreton R, Bourdaire J M (1985) La decision economique. Presse universitaire de France ed, Que sais-je?, 1985, ISBN 2-13-039042-0, 125p Chauveau JP, Samba A (1990) Un developpement sans developpeurs ? Historique de la peche artisanale maritime et des politiques de developpement de la peche au Senegal. Doc ISRA, serie Reflexions et Perspectives, 20p CRODT (1989) Statistiques de la peche artisanale maritime senegalaise en 1987. Arch Centr Rech Oceanogr Dakar-Thiaroye, no 175, juillet 1989, 85p

Emergence of a Self Organized Dynamic fishery Sector

89

CRODT (1990) Recensements de la peche artisanale maritime senegalaise entre Djifere et Saint-Louis mai et septembre 1987. Arch Centr Rech Oceanogr Dakar-Thiaroye, no 181,juilletl990,49p Dangelmaier W, Franke H, Pape U (2002) A Multi-Agent-Concept in Supply Chain Management. In: International Manufacturing Leaders Forum. Adelaide, 8 - 1 0 February 2002 Donangelo R, Hansen A, Sneppen K, Souza S (2000) Modelling an imperfect market. Physica A 283:469-478 Faratin P, Sierra C, Jennings NR (2002) Using similarity criteria to make issue trade-offs in automated negociations. Artificial Intelhgence 142:205-237 Ferber J (1999) Multi-agent systems - An introduction to distributed artificial intelligence. Addison-Wesley ed, Arlow, Great Britain, 509 p Gale D (1987) Limit theorems for markets with sequential bargaining. J Economic Theory 43(l):20-54 Guriev S, Shakhova M (1996) Self-Organization of Trade Networks in an Economy with Imperfect Infrastructure. In Schweitzer F (ed) Self-Organization of Complex Structures: From Individual to Collective Dynamics. Gordon and Breach Scientific Publishers, London lori G (2002) A microsimulation of traders activity in the stock market: the role of heterogeneity, agents' interactions and trade fictions. J Econ Behavior & Organisation 49:269-285 Kaihara T (2001) Supply chain management with market economics. Int J Production Economics 73:5-14 Laloe F, Samba A (1990) La peche artisanale au Senegal: ressource et strategies de peche. Etude et Theses, Paris, Orstom ed, 395p Le Fur J (1995) Modeling adaptive fishery activities facing fluctuating environments: an artificial intelligence approach. AI Appl Agric Nat Res Environ Sci, 9(1): 85-97 Le Fur J (1998) Modeling fishery activity facing change: Apphcation to the Senegalese artisanal exploitation system. In : Durand MH, Cury P, Mendelssohn R, Roy C, Bakun A and D Pauly (sci eds) Global vs local changes. Orstom coll CoUoques et seminaires, Paris, pp 481-502 Lemke RJ (2004) Dynamic bargaining with action-dependent valuations. J Econom Dynamics & Control 28:1847-1875 Nagel K, Shubik M, Strauss M (2004) The importance of timescales: simple models for economic markets. Physica A 340:668-677 Oliveira E, Fisher K, Stepankova O (1999) Multi-agent systems: which research for which applications. Robotics and Autonomous Systems 27:91-106 Railsback SF (2001) Concepts from complex adaptive systems as a framework for individual-based modeling. Ecol Modelling 139:47-62 Straskraba M (2001) Natural control mechanisms in models of aquatic ecosystems. Ecol Modelling 140:195-205 Vriend N (2004) ACE models of market organization. Rev Econ Industrielle 107:63-74 Zhu J (2004) A buyer-seller game model for selection and negotiation of purchasing bids: extension and new models. European J Operational Research 154:150-156.

Multi-Agent Model of Trust in a Human Game Catholijn M. Jonker^ Sebastiaan Meijer^, Dmytro Tykhonov^ and Tim Verwaart^ ^Radboud University Nijmegen, Montessorilaan 3, Nijmegen, The Netherlands {c.jonker, d.tykhonov}@nici.ru.nl ^Wageningen UR, Burg. Patijnlaan 19, Den Haag, The Netherlands {sebastiaan.meijer, tim.verwaart}@wur.nl

Summary. Individual-level trust is formalized within the context of a multi-agent system that models human behaviour with respect to trust in the Trust and Tracing game. This is a trade game on commodity supply chains and networks, designed as a research tool and to be played by human players. The model of trust is characterised by its learning ability, its probabilistic nature, and how experience influences trust. The validity of the trust model is tested by comparing simulation results with aggregated results of human players. More specifically the simulations show the same effects as human plays on selected parameters like confidence, tracing cost, and the trust update coefficient on observable game statistics like number of cheats, traces, certificates, and guarantees.

1

Introduction

People from different cultures differ significantly with respect to uncertainty avoidance, individualism, mutual caretaking and other traits [1]. Personal traits and human relations affect the forming and performance of institutional frameworks in society. Important economic institutional forms are supply chains and networks [2]. The Trust and Tracing game [3] is a research tool designed to study human behaviour with respect to trust in commodity supply chains and networks in different institutional and cultural settings. The game played by human participants is used both as a tool for data gathering and as a tool to make participants feed back on their daily experiences. Although played numerous times, the number of sessions that can be played with humans is limited. It is expensive and timeconsuming to acquire participants [4]. Furthermore one needs many sessions to control for variances between groups [5]. Multi-agent simulation can to some extent overcome these disadvantages in two ways. It can validate models of behaviour induced from game observations and it can be a tool in the selection of useful configurations for games with humans (test design). Validation of the models we designed was done on the aggregated level using computer simulations. Simulation results were compared to a set of hypotheses based on human games observations and conventional economical rationality.

92

C. M. Jonker et al.

This paper presents a multi-agent model of the Trust and Tracing game. It is an instrument in the research method presented in Section 2. Section 3 provides a brief description of the game and results from human sessions. Section 4 describes the agent architecture and models for buyer's behaviour and trust. In section 5 we illustrate the validity of the approach by experimental results from multi-agent simulations. Section 6 presents the main conclusions of the paper.

2

Method

Our research uses a methodological cycle as described in figure 1. It started in the upper left comer with the human game environment. The first series of sessions led to a number of observed individual and aggregated tendencies in the human game. On the basis of observed tendencies and conventional economical theories a multi-agent model was designed and implemented in a simulated environment. In this environment sessions were simulated using the same settings as the initial human sessions. Through verification of aggregated tendencies we have been able to prove gross vaUdity of our model, and the fruitfulness of our approach.

Multi agent modei

Agent tendencies

I \/nrinhla_ Variable Vnrinhig Variable Setting PI

V

I

O K - > generate variations

Sim. Aggr. Model tendencies

Simulated sessions "

Test for PI.... Pn tendencies, and select interesting P settings to play.

Fig. 1. Methodological cycle In current and future work more variations of the setting (including the current one) will be tested in both the human and simulated environment. This will lead either to further adjustments of the multi-agent model or to more variations to test. By testing large numbers of settings quickly in the simulated environment we can select more interesting settings for the human sessions, and thus save research time. The long-term result will, hopefully, be a fiilly validated model of trust with respect to situations comparable to the Trust and Tracing game, where validation is reached for the agent- and the aggregated level.

Multi-Agent Model of Trust in a Human Game

3

93

The Trust and Tracing Game

This section provides a brief description of the Trust and Tracing game; an extensive description is available in [3]. Observations from sessions played are discussed at the end of this section. The focus of study is on trust in a business partner when acquiring or selling commodities with invisible quality. There are five roles: traders (producers, middlemen and retailers), consumers and a tracing agency. . Typically there are 4 producers, 4 middlemen, 4 retailers and 8 consumers, to reflect the multiple steps and oligopoly character of most supply networks. The real quality of a commodity is known by producers only. Sellers may deceive buyers with respect to quality, to gain profits. Buyers have either to rely on information provided by sellers (Trust) or to request a formal quality assessment at the Tracing Agency (Trace). This costs a tracing fee for the buyer if the product is what the seller stated (honest). The agency will punish untruthful sellers by a fine. Results of tracing are reported to the requestor only or by public disgrace depending on the game configuration. A strategy to be a truthful seller is to ask for a trace before selling the product. Sellers use the tracing report as a quality certificate. Middleman and Retailers have an added value for the network by their ability to trace a product cheaper than a consumer can. The game is played in a group of 12 up to 25 persons Commodities usually flow from producers to middlemen, from middlemen to retailers and from retailers to consumers. Players receive 'monopoly' money upfront. Producers receive sealed envelopes representing lots of commodities. Each lot is of a certain commodity type (represented by the colour of the envelope) and of either low or high quality (represented by a ticket covered in the envelope). The envelopes may only be opened by the tracing agency, or at the end of the game to count points collected by the consumers (table 1). The player who has collected most points is the winner in the consumer category. In the other categories the player with maximal profit wins. Table 1. Consumer satisfaction points by commodity type and quality Quality Low High

Blue 1 2

Type Red 2 6

Yellow 3 12

Sessions played until 2005 provided many insights. ([3] and unpublished) We mention three applicable here: 1. Dutch groups (with a high uncertainty tolerant culture [1]) tend to forget about tracing and bypass the middlemen and retailers as they don't add value. This gives the producers a large chance to be opportunistic. Few traces lead to more deceits. 2. American groups tend to prefer guaranteed products. They quickly find out that the most economic way to do this is by purchasing a traced product and to let

94

CM. Jonkeretal.

the middlemen do the trace, as this is the cheapest step. After initial tracing of any lot middlemen start to take samples when relationships establish. 3. Participants who know and trust each other beforehand tend to start trading faster and trace less. The afterwards indignation about deceits that had not been found out during the game is higher in these groups than it is when participants do not know each other.

4

Agent Architecture and Buyer's Model

The agent architecture for simulation of the Trust and Tracing game has been described in [6]. The models for cheating are discussed in [7]. Types of agents acting in the simulated game are trading agents (producers, middlemen, retailers, and consumers) and the tracing agent. The architecture of the tracing agent is straightforward: it reports the real quality of a product lot to the requestor, informs the sellers that a trace has been requested and penalizes untruthful sellers. In this paper we focus on the trading agents and in particular on their behaviour as buyers, entailing the trust-or-trace decision. agent

trading buying parameter management buyer utility evaluation negotiation termination trade proposal detemriination trust or trace selling parameter management seller utility evaluation negotiation termination trust management

trade proposal determination

goal determination

cheating decision

stock control

Fig. 2. Agent process composition Trading agents have processes for initialization, goal determination, trading, which entails the cheating decision in case of selling and the trust-or-trace decision in case of buying, trust management and stock control. In the goal determination process agents decide to buy or to sell, depending on their role and stock position, and selects a partner at random, weighted by success or failure of previous negotiations with particular partners.

Multi-Agent Model of Trust in a Human Game

95

The trading process is based on the algorithm presented in [8]. This approach to multi-attribute simultaneous negotiations is based on utility functions theory. Negotiation partners send complete bids (a set of negotiation object attributes with assigned values) to each other. Once an agent has received a bid it can accept, or respond with an alternative bid, or cancel the negotiation. Agents evaluate their own and their partner's bid using a generalized utility function that is a weighted linear combination of particular attribute evaluation functions. Weights represent preferences of each agent. In this case the utility function uses normalized values of income and risk (which are calculated from the negotiation object attributes) . The buyer's utility function involves individual experience-based trust in the seller as an argument to estimate the risk of being deceived. Modeling of trust for this purpose and experience-based updating of trust - as part of the trust management process - is the subject of subsection 4.1. Subsection 4.2 explains the utility function and the way it can be used to represent agent's preferences or buying strategies. Subsection 4.3 treats the tracing decision. 4.1. Trust Models In literature a variety of definitions of trust phenomena can be found. The common factor in these definitions is that trust is a complex issue relating belief in honesty, trustfulness, competence, reliability of the trusted system actors, see e.g., [9, 10, 11, 12]. Furthermore, the definitions indicate that trust depends on the context in which interaction occurs or on the observer's point of view. According to Ramchum et al. [10] trust can be conceptualized in two directions when designing agents and multi-agent systems: • Individual-level trust - poses agents beliefs over honesty of his interaction partner(s); • System-level trust - system regulation protocols and mechanisms that enforce agents to be trustworthy in interactions. In this paper we address problems and models for individual-level trust as our simulation environment already has system-level trust mechanisms such as the tracing agency that encourage trading agents to be trustworthy. Defining trust as a probability allows relating it with risk. Josang, and Presti [12] analyse the relation between trust and risk and define reliability trust as "trusting party's probability estimate of success of the transaction". This allows for considering economic aspects; agents may decide to trade with low-trust partners if loss in case of deceit is low. An important subprocess of the agent's trust management process is trust update based on tracing results. The current model uses the trust update schema proposed in [13]: g{ev,tv) = dtv + {\-d)ev

(1)

where tv is the current trust value, ev is the experience value, and d is the ratio that introduces the memory effect of the trust update function. This function poses the following properties: monotonicity, positive and negative trust extension, and

96

CM. Jonkeretal.

strict positive and negative progression. This model is suitable, because (1) models learning, necessary, because experience is the only source of information; (2) it has a probabilistic nature, usefull in the calculation of risk; (3) it has a memory effect and allows inflation of experience. Each agent maintains the level of trust he has in the other agents with respect to their role as a supplier and uses tracing results to update its trust. A trace revealing deceit has a negative effect on trust in the partner as a supplier. If a supplier is found truthful by tracing this will strengthen trust. The tracing agent has two different modes, to be set by the game leader: (1) with confidential reports of deceit to the requesting agent only and (2) with public disgrace of deceivers, where all agents are informed if a deceiver has been punished. Experience values were assigned taking into account empirical data that conclude that "it appears easier to destroy trust than to build trust" [14]. This means that negative experience has stronger impact on trust than positive experience has. This assumption is reflected in appropriate experience evaluation values: ev(pos) = 0.5 and ev(neg)=-\. The value of d and the initial value of tv are agent parameters set by the game leader. Usually (i= 0.4 and tv = 0.5. 4.2.

Buyer's Model

Negotiation skill is an important capability of agents since it enables them to efficiently achieve their goals. In the T&T game the trading process is used to achieve trade deals. The negotiation system employed in the simulation is based on utility functions. The utility function for buyers involves the risk of being deceived when buying (stated) high quality commodities. Depending on trust in seller (belief about the opponent) and risk-attitude (personal trait of buyer), the buyer can try to reduce risk. Risk can be eliminated by demanding a quality certificate or reduced by a money-back guarantee. The attributes of a transaction are product type, stated quality, price, and certificate or money-back guarantee. The buyer's utility function is a weighted sum of normalized functions of price, satisfaction difference between high and low quality (for consumers) or expected turnover (for others), and risk (estimate based on trust in seller, guarantee and prices):

+ >^2/exp ected _ turnover {expccted _ tumover(bid')) + wj^.^, {risk^eiier (bid') The weight factors implement buyer's strategies. For quality-minded buyers that are willing to pay to ensure high quality, both W2 and W3 are high relative to Wi, for instance <0.2, 0.4, 0.4>. The opportunistic buyer prefers high quality for low price but is prepared to accept uncertainty, for instance <0.4, 0.4, 0.2>. The suspicious buyer follows an what-you-see-is-what-you-get strategy, represented for instance by <0.4,0.2,0.4>. Effective price is the total amount of money that the buyer has to pay:

Multi-Agent Model of Trust in a Human Game

97

P^^^^ effective (pid') = pHce ^^^^,^^^ + cost ^^^^^^^^,^^

(3)

where cost^^^^^^^^.^^ represents some extra cost for the buyer that depends on the type of partner and is taken from the transaction cost matrix to be defined by the game leader. (Table 2 gives an example) Table 2. Example of a transaction cost matrix Buyer Producer Middleman Retailer Consumer

Producer 10 2 4 8

Seller Middleman 100 10 2 4

Retailer 100 100 10 2

Consumer 100 100 100 10

Expected turnover is the average of the agent's beliefs about minimal and maximal future selhng price of the commodity to be bought. For consumers the expected turnover is changed to satisfaction level. Buyer's risk represents the estimation of probable losses for a given trade partner and trade conditions. It is calculated as product of probability of deceit and cost in case of deceit. ^isk,^^^^ ibid) = p,^^^, • cost,^^^,

(4)

Probability of deceit is grater than zero only if commodity quality is high and it is not certified. If these conditions are satisfied than the probability of deceit is estimated as the complement of buyer's trust in the seller. P deceit {bid') = q(bid') • c(bid') • (1 - trust {seller))

(5)

Costs in case of deceit are estimated for middlemen and retailers as the sum of the fine for untruthfully reselling a product and, only if no guarantee is provided, the loss of value that is assumed to be proportional to the loss of consumer satisfaction value taken from table I. The formula for middlemen and retailers is: cost,^^^, {bid) =fine^^^^,.^^+ loss^^^^,.^^ {bid)

(6)

where loss^^^^,.^^ {bid) = g{bid) • price^ff^^,^^ • (l - ratio,^^^,.^, {bid)) and g represents the guarantee function (5): g(bid)=l if the bid involves a guarantee; g(bid)=0 otherwise. For consumers the cost in case of deceit is also assumed to be proportional with the loss of satisfaction value, but they do not risk a fine, so for consumers: ^ost,^^^, {bid) = g{bid) - price^ff,^,^^ • (l - ratio^^^^,.^, {bid))

(7)

(8)

98

CM. Jonkeretal.

This subsection presented the buyer's model. The seller's utility function partially reflects the buyer's model. It considers effective price and risk as attributes, see [7]. 4.3. Tracing Decision

For buyers, trading entails the trust-or-trace decision. In human interaction this decision depends on factors that are not sufficiently well understood to incorporate in a multi-agent system. Hearing a person speak and visual contact significantly influence the estimate of the partner's truthfulness [15]. To not completely disregard these intractable factors the trust-or-trace decision is modeled as a random process instead of as a deterministic process. In our model the agglomerate of all these intractable factors is called the confidence factor. The distribution involves experience-based trust in the seller, the value ratio of high versus low quality, the cost of tracing, and the buyer's confidence factor. Tracing reveals the real quality of a commodity. The tracing agent executes the tracing and punishes cheaters as well as traders reselling bad commodities in good faith. The tracing agent only operates on request and requires some tracing fee. Agents may request a trace for two different reasons. First, they may want to assess the real quality of a commodity they bought. Second, they may provide the tracing result as a quality certificate when reselling the commodity. The decision to request a trace for the second reason originates from the negotiation process. This subsection focuses on the tracing decision for the first reason. Several factors shown in figure 3 influence the tracing decision to be made after buying a commodity. First of all the tracing decision is based on the buyer's trust in seller. Trust is modelled as a subjective evaluation of the probability that the seller would not cheat on the buyer. It is updated using tracing results: positive tracing results increase the trust in seller, negative ones decrease it. Then satisfaction ratio (see Table 1) of the commodity is considered. The buyer would trace more valuable products rather than products with small satisfaction ratio, because damage would be greater. My trust in seller

Good/bad customer satisfaction ratio

Tracing cost/ (effective price+tracing cost)

'' Trace factor: alpha*

Trace if trace factor>random

Fig. 3. Tracing decision model

Confidence

Multi-Agent Model of Trust in a Human Game

99

Tracing costs also influences the decision, so a middleman is more likely to trace than a consumer. The tracing fee depends on the depth to be traced, so for middlemen tracing is cheaper then for consumers. Confidence is an internal characteristic that determines the preference of a particular player to trust rather than trace, represented as a value on the interval [0,1]. The following expressions are used to make tracing decision: tracing _ level (bid) = (1 - trust{seller(bid)) * (l - ratio^^^,;, .^^ (pid)) * * tracing _ cos t _ ratio{bid) * (1 - confidence) where tracing_level(bid) - is a value on the interval [0,1] that represents an evaluation of tracing preference of a given bid and . . 7 . ,x effective priceibid) ,,^, tracing_cost_ratio {bid) = =-^^ (10) tracingjcost + effective _ price(bid) The tracing decision depends on the following rule: //

tracing _ level (bid) >rnd

then

trace

(11)

where rndis a random number in [0,1]. If an agent has decided to trace the product it sends a tracing request message to tracing agent. Once the tracing result has been received the agent updates its trust belief about the seller and adds the product to the stock.

5

Experimental Results

A group of experts possessing empirical knowledge of the game formulated the conceptual model of the Trust and Tracing game's system dynamics on an aggregated level using Vennix' group model building [15], and used it to formulate hypotheses about the effect of selected parameters {confidence, tracing cost, and trust update coefficient^trust, represented by din equation 1) on observable game statistics (number of cheats, traces, certificates, Sind guarantees). The experts used experiences from over 40 sessions with the game (during its development, testing and real world application phase) and knowledge from case studies from literature to express the following hypotheses. As an example we present the hypotheses about confidance: Hypotheses about the effects of confidence. 1. Increasing confidence decreases tracing. A highly confident buyer makes fewer traces as he thinks that his buying mechanism is taking care of risks. Confidence is present in our agent's tracing model and defines threshold for tracing. 2. Increasing confidence increases cheating, because honesty will not be corrected. High confidence means that players perform fewer traces. This means

100

C M . Jonkeretal. that sellers experience low numbers of fines that should decrease their level of honesty. Low level of honesty makes cheating more probable. Increasing confidence decreases certificates. Confidence has reverse impact on the tracing rate. High confidence decreases tracing rate and consequently decreases number of found cheats. This keeps average trust high that is in it turn decreases number of certificates. Increasing confidence increases guarantees. Because high confidence makes average trust higher it reduces risk of providing a guarantee and consequently increases number of guarantees provided.

3.

4.

Computer simulations were performed with populations of 15 agents: 3 producers, 3 middlemen, 3 retailers, 6 consumers. Game sessions are performed in continuous real-time and depend only on the performance of the computer. Agents can be involved in only one transaction a time. This organization allows (future) combining of artificial and human agents in one game session. Values of free parameters were selected uniformly from their definition intervals to confirm the models capability to reproduce desired input-output relationships and explore their sensitivities. Figure 5 presents results of experiments performed for two values of confidence: 0.1 and 0.9 across populations with various risk-taking attitude, respectively: no increased-risk-takers (denoted as "neutral" on the X-axis of the charts on figures 3,4,5), 1 out of 3 risk-takers (denoted as "2:1"), 2 out of 3 risk-takers (denoted as "1:2"), and all risk-takers (denoted as "high"). For risk-taking agents weights in (1) were set to: w^ =0.4;w2 =0.4;w3 = 0 . 2 , for agents with neutral risk attitude weights were Wj = 0.2; w^ = 0.4; W3 = 0.4. Percentage of Trace vs Confidence Level

Percentage of Cheat vs Confidance Leve 50% 1 S S 40% -

I 20% I 10%

nHigh

"S 30% -

• High

DLOW

!> 20% -

• Low

1 10%^ (2:1)

0%-

(1:2)

_

Neutral

Risk talking attitude

^

(2:1)

rH 1 • (1:2)

High

Risk talking attitude

Percentage of Certificates vs Confidence Level

Percentage of Guarantees vs Confidence Level

1 "S 30% f 20% |,or. »•

m.

LFn

"1 Neutral

nHigh DLOW

g

1 ,rB

(2:1)

S ) | 30%

sI (1:2)

Risk taking attitude

<5 2 0 %

r^t-n Neutral

(2:1)

(1:2)

Risk taking attitude

Fig. 5. Results of experiments with different levels of confidence

Multi-Agent Model of Trust in a Human Game

101

The difference in output variables with respect to high and low level of confidence is not significant for neutral risk-taking agents. For high-risk-taking agents the amount of traces decreases for highly confident players and increases for lowly confident players. This supports hypothesis #1 only in cases with players possessing high risk-taking value. Number of cheats is high for highly confident players in games with dominating number of risk-neutral players. This partly confirms hypothesis #2. Surprisingly, the results show the opposite for risk-seeking players: some dishonest sellers do not get traced and punished in highly risk-seeking game configurations, so they are encouraged to continue their fraudulent practices. The results confirm the hypotheses about certificates (#3) and guarantees (#4). Increase in number of guarantees for highly confident agents testifies that feedback link through tracing trust. High confidence leads to lower number of traces that means less deceptions are discovered and consequently higher average tracing trust. In all experiments, effects of risk-taking attitude are consistent: high risk-taking leads to more cheating, less certificates and increased willingness to give guarantees and to rely on them. Differences in risk-taking attitude outweigh changes in other parameters. This result corresponds with observations from human games.

6

Conclusion

This paper presents a partially validated model of trust in a trade network environment. At this point in the research project we are not able to validate the model on the agent-level. The model presented approaches the complex dilemma of trust in a trade network environment to the extent that it is able to parallel real human behavior. The cycle (figure 1) has been completed once for the aggregated level only, and for one particular setting of the game. The main contribution is that by defining (simple) models on the individual level we can produce similar outcomes to human sessions on the macro level. Given the number of free variables when using real human participants these aggregated results are most important to compare. Rigorous testing for more settings will lead to a refinemented model that matches aggregated results for all settings. Only then our individual agent model will be looked upon to see if it models individual decisions accurately. Following the falsification theory of Popper [17], even if our approach does not lead to new models of trust, the attempt to test current theories via a model (hypothesis of being similar in behavior to a real human participant) using a welldefined new empirical basis is worthwhile. In the worst case it will lead to reimbursement of existing models. In the best case it could lead to better insights in the use of trust.

102

C M . Jonkeretal.

References 1. 2. 3.

4. 5. 6.

7.

8.

9. 10. 11.

12.

13.

14.

15.

16. 17.

G.J. Hofstede, P.B. Pedersen, and G. Hofstede, Exploring cultures: Exercises, stories and synthetic cultures, Intercultural Press, 2002. T. Camps, P. Diederen, G.J. Hofstede, and B. Vos (eds.), The emerging world of Chains and Networks, Reed Business Information, 2004. S. Meijer and G.J. Hofstede, The Trust and Tracing game. In: Proceedings of 7* Int. workshop on experiential learning. IFIP WG 5.7 SIG conference. May 2003, Aalborg, Denmark. R.D. Duke, and J.L. Geurts, Policy games for strategic management, pathways into the unknown, Dutch University Press, Amsterdam, 2004. D. Crookall, A guide to the literature on simulation / gaming. In: D. Crookall and K Arai, Simulation and gaming across disciplines and cultures, ISAGA, 1997. S. Meijer, and T. Verwaart. Feasibility of Multi-agent Simulation for the Trust-andTracing Game. In: M. Ali and F. Esposito (Eds.), Innovations in Applied Artificial Intelligence, Proceedings of lEA/AIE 2005, LNAI 3533, 2005, pp. 145-154. CM. Jonker, S. Meijer, D. Tykhonov, and T. Verwaart, Modelling and Simulation of Selling and Deceit for the Trust and Tracing Game. In: Proceedings of TRUST 2005: 8th Workshop on Trust in Agent Societies, to appear in LNCS, Springer-Verlag, 2005. CM. Jonker, and J. Treur, 2001, An agent architecture for multi-attribute negotiation. In: B. Nebel (ed.). Proceedings of the 17* Int. Joint Conf. on AI, IJCAI '01, 2001, pp. 1195-1201. T. Grandison, and M. Sloman, A Survey of Trust in Internet Applications, IEEE Communications Surveys, 2000. S. D. Ramchum, D. Hunyh., and N.R. Jennings, Trust in Multi-Agent Systems, Knowledge Engineering Review, 2004. C Castelfranchi, and F. Rino, Social Trust: A Cognitive Approach, In: C Castelfranchi, Y.H. Tan (eds.). Trust and Deception in Virtual Societies, Kluwer Academic Publishers, 2001, pp. 55 - 90. A. Josang, and S. Presti, Analysing the Relationship between Risk and Trust. In: CJensen, S.Poslad,T.Dimitrakos (eds.): Trust Management, Proceedings of iTrust 2004, LNCS 2995, 2004, pp. 135 - 145. CM. Jonker, and J. Treur, Formal analysis of models for the dynamics of trust based on experiences. In: F.J.Garijo, M.Boman (eds.). Proceedings of MAAMAW'99, LNAI 1647, 1999, pp. 221-232. V. Amamor-Boadu, and S.A. Starbird, The value of anonimity in supply chain relationships. In: H.J. Bremmers, S.W.F. Omta, J.H. Trienekens, and E.F.M. Wubben (eds.). Dynamics in Chains and Networks, Wageningen Acedemic Publishers, Holland, 2004 pp. 238 - 244. J.K. Burgoon , G.M. Stoner , J.A. Bonito, and N.E. Dunbar, Trust and Deception in Mediated Communication. In: Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 1, p.44.1, January 06-09, 2003. J.A.M. Vennix, Group Model Building: Facilitating Team Learning Using System Dynamics. Wiley, 1996. Popper, K.R., The logic of scientific discovery. New York, 1986.

A Counterexample for the Bullwhip Effect in a Supply Chain Toshiji Kawagoe^ and Shihomi Wada^ ^ Future University-Hakodate, Department of Complex Systems, 116-2 Kameda-Nakanocho, Hakodate, Hokkaido, 041-8655, Japan. kawagoe@fun. ac . j p ^ Future University-Hakodate, Graduate School Systems Information Science. g31050 [email protected] Summary. In our experiment of a supply chain using Beer Game, to identify the cause of bullwhip effect, the number of firms in a supply chain (two or four firms), and the length of the delay in shipping between firms (one or three weeks) are controlled and compared in the multiagent simulations. We found a counterexample for bullwhip effect such that inventory level of upstream firm was not always larger than that of downstream firm. In addition, contrary to our intuition, such a counterexample was frequently observed under the condition that (1) the number offirmsin a supply chain was many, and that (2) the length of delay was rather longer.

1 Introduction The bullwhip effect is the amplification of the order variability in a supply chain. This phenomenon causes important financial cost due to higher inventory levels. So, reducing the order variability in a supply chain is quite important issue in supply chain management. Lee, Padmandhan, and Whang [3] observe that the bullwhip effect occurs when demand orders variability in the supply chain are amplified as they moved up the supply chain. They point out that there are several causes of the bullwhip effect. • • • •

Demand forecasting updating Order batching Price fluctuation Rationing and shortage gaming

As most of them are related to the issue of information sharing in a supply chain, Moyaux, Chaib-draa, and D'Amours [4, 5] compare three inventory management strategies with different information sharing schemes. They compare three inventory management strategies using the Quebec Wood Supply Game, a variant of Beer Game. Their motivation is to identify which combination of inventory management strategies among firms in a supply chain can be a Nash equilibrium in the sense

104

Toshiji Kawagoe and Shihomi Wada

of Game Theory (Cachon [6]). They find that combinations of more collaborative strategies are Hkely to be Nash equiUbria. One drawback of their study is that, due to the complexity of finding an optimal solution for inventory management strategies, they consider only a small set of tractable strategies in their study. So, the question, which factor is crucial for reducing the bullwhip effect, still remains unsolved. In our experiment using Beer Game, the number offirmsin a supply chain (two or four firms), and the length of the delay in shipping between firms (one or three weeks) are compared in the multiagent simulations. To the best of our knowledge, there is no research of changing such environmental variables to identify the cause of the bullwhip effect. In our result, we found a counterexample for the bullwhip effect such that inventory level of upstream firm was not always larger than that of downstream firm, even in the environment that the number of firms was many and length of delay was longer. This is very similar result that Kawagoe and Wada showed in their experiment with human subjects (Kawagoe and Wada [1]). The organization of the paper is as follows. In the next section, we explain our model of Beer Game and independent variables in our experiment. In section 3, the details of the multiagent simulation are presented and the results are shown. Concluding remarks are given in the final section.

2 Model Beer game models a supply chain with severalfirms(the retailer, the distributor, and the factory). The game proceeds as follows (we present a four-firm three-week-delay case as an example). Step 1. In each week, each firm receives orders from the downstream firm and receives the productsfi*omthe upstream firm which were ordered three weeks ago. (For the retailer, orders from the customer, which follows a certain probability distribution, come every week.) Step 2, Each firm ships the products from his inventory to the downstream firm if he has enough inventory to cover that order. If not, the amount that order minus inventory is recorded in the backorder to be shipped in the next week or later. Step 3. Then eachfirmother than the factory makes an order to the upstream firm to fill his inventory for the next week order and to clear the backorder, if any. Step 4. The factory starts producing new products to fill his inventory for the next week order and to clear the backorder, if any. It takes three weeks for the factory to complete to produce new products. Step 5. This ends a week. Go back to Step 1. Theflowof these steps is shown in Fig. 1.

A Counterexample for the Bullwhip Effect in a Supply Chain

1

F:factory ] ] ]

1

j h p ip

Di-.distributor 1= 1

tl p Ip

c]

1

105

1=

]

1=

]

D2 .distributor lb

] 1=

a

}

ip R:retailer

1

] IZ ] 1=

1

)

Cxustomer

I Products stream czi 1 week shipping delay

| |k Orders stream a 1 week order delay

Fig. 1. Beer Game in (D3, F4) case

3 Experiments In our multiagent simulation, the number of firms in a supply chain (two or four firms), and the length of the delay betweenfirms(one or three weeks), are compared. We also compare the effect of different probability distribution of the customer's orders and inventory strategies of each firm. As for the probability distribution, we adopt four customer's order distributions in Moyaux et al. [5] and add a constant order distribution as a benchmark. The detail of each distribution are shown in Table 1. Note that all of these distributions are not stochastic. To represent the state of eachfirm,the set of parameters are shown in Fig. 2 used in our simulation. In our multiagent simulations, there is no centralized agent or meta agent controlling whole activities in a supply chain, and information of the customer's orders is not shared amongfirms.In this sense, each firm in a supply chain is an agent and decides each inventory level in decentralized way. A simple and deterministic inventory strategy of each firm is as follows (see also Kawagoe and Wada [2]). After knowing orders from the downstream firm and receiving the products from the upstream firm, each firm updates his inventory and ships the products if he has enough inventory. If not, the amount that the order minus inventory is recorded in the backorder to be shipped in later weeks. Then each firm makes an order to fill his inventory to cover the next week order and to clear the backorder, if any. The next week order is predicted by a weighted moving average

106

Toshiji Kawagoe and Shihomi Wada

It = Firm re's inventory in week t Ljf = The amount of demand order which firm x receives from the downstream firm in week t Qt = The amount of demand order which firm x states to the upstream firm in week t

Xt = The amount of products which firm X receives firom the upstream firm in week t St = The amount of products which firm X ships in week t Bt — The amount of backorder which firm X has in week t Tfi%j^x = The amount of order in week (t + 1) predicted by firm x in week t.

Fig. 2. Parameters for representing the state of each firm of past orders. We used both standard and moving average as well as three- and five-week moving average for comparisons. Thus, each firm x updates the state of inventory level and the amount of backorder, and determines the amount of products to be shipped and the amount of demand orders according to the following algorithm Order. We also show in Fig.4 how /f, Bf, Ljf and m^ behave according to that algorithm.

Algorithm Order Input Ux,BfJf,Sf.,ujf.,y^.,Qf ifa;f > / f _ i + X?then set set If It ^<- 00 set Bf ^ 5f_i + cjf - (/f_i 4- Xt) SPf Of ^ 4— J5f Rf + -^ -m ?, , set l?f mf+i se else i f 5 f _ i + c j ? > 7 f _ i + x?then set5f ^ 7 f _ i + x r set 7f ^ 0 set Bf ^ Bf_i + ujf - (7f_i + Xt) set n? ^Bf+m^+i else set5f ^ B f _ i + c j f set7f ^ 7 f _ i + x r - ( B f - i + a ; n set Bf ^ 0 if 7f + m?+i then i7f = 0 else Qf = m?+i - 7f

Our experimental conditions can be written by a five tuples (D, F, 75, L, T). Each term is an independent variable in our experiment, where D = the length of delay, which is either 7^1 = 1 week or 7^3 = 3 weeks, F = the number of firms.

A Counterexample for the Bullwhip Effect in a Supply Chain

107

which is either F 2 = 2 firms or F 4 = 4 firms, IS = inventory strategy, which is either W = the weighted moving average or 5 = the standard moving average, L = the length of weeks moving average, which is either L3 = 3 weeks or L5 = 5 weeks, and T = time series of the customer's order distribution, which is shown in Table 1. Table 1. Time series of customer's order distributions t 12 3 4 5 6 48 49 50 Tl 11 11 11 11 17 17 17 17 17 T2 17 17 17 17 11 11 11 11 11 T5 11 11 11 11 12 13 55 56 57 13 12 11 T6 57 57 57 57 56 55 TC 11 11 11 11 11 11 11 11 11

These independent variables in our experiment are summarized in Fig. 3. Thus, our experiment i s a 2 x 2 x 2 x 2 x 5 design. As in Moyaux et al. [5], Beer Game is repeated 50 weeks in our simulation. D — The length of delay D l A week J93 Three weeks F — The number of firms F2 Two F4 Four

75 = Inventory strategy W Weighted moving average S Standard moving average L = The length of weeks taken for determining moving average L3 Three weeks L5 Five weeks

T = Time series of the customer's order distribution T l Pattern 1 in Moyaux et al.[5] T2 Pattern 2 in Moyaux et al.[5] T5 Pattern 5 in Moyaux et al.[5] T6 Pattern 6 in Moyaux et al.[5] TC Constant (always 11) Fig. 3. Independent variables in our experiment

4 Results Of 80 experimental conditions, there is no significant effect in the factors ST, L, and T. So, from now on, we use ST = W,L = b, and T = TC as a typical case. Our results are summarized in Fig. 5. There are four panes in Fig. 5 and each pane corresponds to each combination of D and F, where If is the firm a;'s inventory at week t.

108

Toshiji Kawagoe and Shihomi Wada

2nd Distributor T

1

r

— ••

r

•

f

1

1

1

1

\ 1

^

1

20

30 week

Factory • "

-

1 '""

—

1

I

1

1

^

1 ^

•

'— A

\

^f

i

11

1

\

\

«

20

30

week

Fig. 4. Time series of inventory, backorder, order, and moving average

1

A Counterexample for the Bullwhip Effect in a Supply Chain JD1,F2

109

D1,F4

e n4 t o r2 y 0

I e n4 t o

r2 y 0

10

20

30

40

50 Week

10

20

30

40

50 Week

Fig. 5. Time series of each firm's inventory level in (W, L5, TC) case Note that only for {D3, FA) case, after t > 32, inventory level for the first distributor, //^S exceeds that of the factory, l[. This confirms qualitatively the existence of a counterexample for bullwhip effect. Of all of our data, similar counterexamples are obtained only when the combination of (i^3, F4) is used. However, this is rather counterintuitive because such counterexamples are obtained even in the environment that the number of firms is many and length of delay is longer. Although there could be many quantitative definition of the bullwhip effect, natural one is that mean inventory level of upstream firm is larger than that of downstream firm. Another one is that variance of inventory level of the upstream firm is larger than that of the downstream firm. To judge our results means quantitatively a counterexample for the bullwhip effect in both senses, descriptive statistics of inventory level of each firm in TC case is shown in Table 2. One can easily see in Table 2 that mean and standard deviation (SD) of inventory level of the first distributor are both larger than that of factory. So, we can say that our results also means a counterexample for the bullwhip effect quantitatively in both senses. To judge whether such a counterexample may disappear in the long run, we also run a simulation with 30,000 weeks. The result of this simulation is shown in Fig. 6. After 5,000th week, inventory level of the first distributor becomes lower than that of the factory, but inventory level of the first distributor exceeds periodically that of the factory after 10,000th week, even the customer's order distribution is constant in this case. So, the counterexample for the bullwhip effect does not disappear even in the long run.

110

Toshiji Kawagoe and Shihomi Wada Table 2. Descriptive statistics of inventory level of each firm TFT

It' li /f Mean 1389.3 8774.9 26082.4 22397.8 SD 1758.1 10545.9 30521.1 25574.6 12 Vledian 12 0 11 0 0 53016 Mode 0 0 0 Min. 0 0 Max. 3781 22118 63990 53016

Sum

69463 438743 1304121 1119889

10000

20000

30000

Week

Fig. 6. Inventory level in the long run in (D3, F4, VT, L5, TC) case

5 Conclusions In our experiment of a supply chain using Beer Game, the number of firms in a supply chain (tv^o or four firms), and the length of the delay between firms (one or three weeks) were compared in the multiagent simulations. In our experiments, we found a counterexample for buUwhip effect such that inventory level of upstream firm was not always larger than that of downstream firm. In addition, contrary to our intuition, such a counterexample was frequently observed under the condition that (1) the number of firms in a supply chain was many (four firms), and that (2) the length of delay was rather longer (three weeks). One may criticize our result because we employed a very simple inventory strategy in our simulation. It is, of course, worthwhile to run another series of experiments by introducing more sophisticated strategies. One may also feel unrealistic that there is no capacity limit for inventory level for each firm in our experiment. Incorporating such restrictions in our environment is also worth considering in the fixture research.

References 1. Kawagoe T, Wada S (2005). "An Experimental Evaluation of the BuUwhip Effect in A Supply Chain". The Third Annual Conference of the European Social Simulation Asso-

A Counterexample for the Bullwhip Effect in a Supply Chain

111

ciation (ESSA) 2005, Accepted. Kawagoe T, Wada S (2005). "How to Define the Bullwhip Effect". North American Association for Computational Social and Organizational Science (NAACSOS) 2005, Accepted. Lee HL, Padmanabhan V, Whang S (1997). "The Bullwhip Effect In Supply Chain," Sloan Management Review, 38, 93-102. Moyaux T, Chaib-draa B, D'Amours S (2004). "Experimental Study of Incentives for Collaboration in the Quebec Wood Supply Game," Management Science, submitted. Moyaux T, Chaib-draa B, D'Amours S (2004). "Multi-Agent Simulation of Collaborative Strategy In A Supply Chain," Autonomous Agents & Multi Agent Systems (AAMAS) 2004. Cachon GP, Netessine S (2003). "Game Theory in Supply Chain Analysis". In D. SimchiLevi, S. D. Wu, and Z.-J. M. Shen, eds. Supply Chain Analysis in the eBusiness Era. Kluwer.

Bottom-Up Approaches

Collective Efficiency in Two-Sided Matching Tomoko Fuku, Akira Namatame^ and Taisei Kaizouji^ ^{g4 3 0 3 6,nama}@nda.ac.jp Department of Computer Science, National Defense Academy, Yokosuka, Kanagawa, 239-8686, Japan •^ k a i z o j i@icu. ac . j p Division of Social Sciences, International Christian University, 3-10-2 Osawa, Mitaka, Tokyo, 181-8585 Japan

Summary. Gale and Shapley originally proposed the two-sided matching algorithm. Deferred Acceptance Algorithm (DA). This is very brilliant method, but it has a demerit which produces. That is if men propose, it produces stable matching which is the best for men and the worst for women, and vise versa. In this paper, we propose a new algorithm with compromise that produce the balanced matching which are almost optimal for both sides. It is an important issue how far agents seek their own interests in a competitive environment. There are overwhelming evidences that support peoples are also motivated by concerns for fairness and reciprocity. We will show that compromise which is individually irrational improves the welfare of the whole groups. The reasonable compromise level is obtained as the function of the size of the group so that the social utility should be maximized. We also obtain large-scale properties of the proposed algorithm.

1

Introduction

Some researchers have started to take a direct role in issues of designing market, e.g. labor market, a venue for bilateral trading require a proper matching. But, in considering the design of markets, is extremely complex. Markets evolve, but they are also designed. The complexity of market design comes from many factors, especially strategic behaviors of participants. A market is two-sided if there are two sets of agents, and if an agent from one side of the market can be matched only with an agent from the other side. One of the main functions of many markets is to match one kind of agent with another: e.g. students and colleges, workers and firms, marriageable men and women. A two-sided matching model is introduced by (Gale and Shapley 1962), and they invented the deferred acceptance algorithm. One of the basic problems in societies is to match one kind of agent with another, e.g. marriageable men and women students and colleges, workers and firms. A two-sided matching model was introduced by Gale and Shapley, and they focused on college admissions and marriage. They proposed that a matching (of students and colleges, or men and women) could be regarded as stable only if it left

116

Tomoko Fuku et al.

no pair of agents on opposite sides of the market who were not matched to each other but would both prefer to be. A natural application of two-sided matching models is to labor markets. (Shapley and Shubik 1972) showed that the properties of stable matching are robust to generalizations of the model which allow both matching and wage determination to be considered together. (Kelso and Crawford 1982) showed how far these results can be generalized when firms, for example, may have complex preferences over the composition of their workforce. Two-sided matching models have proved useful in the empirical study of labor markets, starting with the demonstration in (Roth 1984). Subsequent work has identified natural experiments which show that labor markets organized so as to produce wwstable matching suffer from certain kinds of difficulties which are largely avoided in comparable markets organized to produce stable matching. This work combines the traditions of cooperative and noncooperative game theory, by considering how the strategic environment faced by market participants influences the stability of the resulting market outcome. Much of two-sided matching theory is concerned with determining the conditions under which stable matching exist, and with what algorithms these matching can be achieved. A two-sided matching could be regarded as stable if it left no pair of agents on opposite sides of the market was not matched to each other but would both prefer to be. The relationship between the concept of Pareto optimality and the stability of a matching has been also investigated. Pareto optimality requires that no change exists that betters every individual in the population. The concept of a stable matching is stronger than that of a Pareto optimal matching, in that every stable matching is Pareto optimal, but not every Pareto optimal matching is stable. Pareto optimality requires that no two individuals wish to elope together and would receive the consent of their partners. Stable matching, by contrast, requires that no two individuals wish to elope together, whether or not their partners would consent. For instance let consider two groups of marriageable men and women. A stable matching / / is defined as M-optimal if no male prefers any other stable matching. F-optimality is defined analogously. However, the M-optimal matching is not only the best stable matching for the males, it is also always the worst stable matching for the females. In fact, male and female preferences conflict in this way over any pair of stable matching, not just the M- and F-optimal ones. In this paper, we propose a new algorithm for two-sided matching with some compromise. We discuss the self-interested hypothesis vs. human sociality hypothesis. It is an important issue such as how far agents seek their own interest in a competitive environment? There are overwhelming evidences that support peoples are also motivated by concerns for fairness and reciprocity. We showed that compromise, an individually irrational behavior, improves the welfare of others. We also obtain large-scale properties of some two-sided matching algorithms. We show some compromises of individuals increase global welfare. The optimal compromise level is designed so that the social utility is maximized.

Collective Efficiency in Two-Sided Matching

2

117

A Formulation of Two-Sided Matching Problem

There are two disjoint sets of agents, groups of men = {mj,..., m„}, and women = {f]y">fn}' Associated with each side is the number of positions they have to offer. Agents on each side of the market have (transitive and strict) preferences over agents on the other side, with presented a simple model in which college applicants have ordinal preferences over schools, and colleges have ordinal preferences over applicants. How, given these preferences, could college applicants be matched to schools so as to leave both the students and the colleges as satisfied as possible? The authors derived a clever algorithm (described below) designed to create efficient pairings. Their algorithm matches students and schools in such a way that no student wishes to leave her current school for an alternative institution that would be willing to admit her. Subsequent authors expanded upon Gale and Shapley's work, extending their theoretical framework while applying two-sided matching theory to problems ranging from labor markets to human courtship and marriage. Consider a population, each member of which falls into one of two sets: the set of all males M ={mi, m2, ms,...., m„} and the set of all females F =(/}, f2, fs, •..-,/»} • Let each individual m/ or^ have a list of strict pairing preferences P over the individuals in the other set. For example, a female fj might have preferences P(fJ) ={mi, m4,fj, ma, nij}, meaning that male mi would be her best choice, ra^ would be her second choice, and she would rather remain 'single' (represented by pairing with herself,^) than form a pair with either m2 or ms. A matching is simply a list of all the pairings in the population (where having oneself for a mate means that one remains single). We indicate the mate of an individual X under matching ju by using ju (z)^^^ short. Now we are ready to consider the notion of the stability of a matching (Knuth 1962). An individual is said to block the matching ju if he or she prefers remaining single to taking the mate assigned by //. A pair m and/are said to block the matching ju if they are not matched by ju, but prefer one another to their mates as assigned by matching ju. Put another way, given matching ju, a blocking pair is a pair that would willingly abandon their mates as determined by ju and elope instead with one another. Finally, the matching ju is defined as stable if it is not blocked by any pair of agents.

3

Deferred Acceptance Algorithm and Its Properties

What happens when preferences are not uniform? One of the most remarkable results from two-sided matching theory is that, even under non-uniform preferences, a stable matching (or set of stable matching) exists in every monogamous matching system. To prove this, it is sufficient to describe an algorithm by which a stable matching can be constructed for any such system. We need not suppose that

118

Tomoko Fuku et al.

pairing actually occurs by this algorithm in the system we are considering. Rather, the algorithm simply serves as a tool in the proof that a stable matching exists. Below, we outline the deferred acceptance algorithm (DA).ThQ matching procedure then proceeds repeatedly through the following steps. <Step 1> Each male not currently engaged displays to his favorite female that has not already rejected him. If no acceptable females remain, he remains unmated. <Step 2> Each female who has received one or more courtship displays in this round rejects all but her highest-ranked acceptable male. This may involve rejecting a previously engaged male. <Step n> After a large number of rounds, no new displays will be made. At this point, the algorithm terminates. All females are paired with the male to whom they are currently engaged; individuals not engaged remain unmated. The matching ju generated in this way is easily seen to be stable. No male wishes to leave his mate at ju for a female who prefers him to her mate at ju, because each male reached his current mate by sequentially courting females in order of preference. No female wishes to leave her mate at ju for a male who prefers her to his mate at ju, because she will have already received a courtship display from any male who is not matched to a female that he prefers to her. Reversing the algorithm, so that the females display and the males accept or reject court-ships, will also lead to a stable matching; this matching may be a different one than that found by the male-courtship form of the algorithm. However, the set of individuals remaining unmated is the same in every stable matching of any given monogamous mating system. As above-mentioned. Deferred Acceptance Algorithm is defined (1) Manoptimal stable matching or (2) Woman-optimal stable matching. That is (1) the matching hM produced by the deferred acceptance algorithm with men proposing is the M-optimal stable matching, or, (2) the W-optimal stable matching is the matching h^ produced when the women propose (Gale and Shapley). And it's emerged that the best outcome for one side of the market is the worst for the other, i.e. M-optimal stable matching is the worst for women. And W-optimal stable matching is the worst for men (Knuth 1972). It may be helpfiil to look at this problem with some concrete example. Consider a group of women(Ann, Betty and Carol) and a group of men (Dave, Eddy and Frank). The preferences of those are given in Table 1(3 represents the highest preference and 1 represents the lowest). In this matching system, there are two stable matching. One (call it / / ; ) pairs Betty with Dave, Ann with Eddy, and Carol with Frank. The other, jU2, pairs Betty with Dave, Ann with Frank, and Carol with Eddy. Any other matching will allow at least one blocking individual or pair. In fact, since Betty and Dave are one another's best choices, any matching / / _ which does not pair them together will be blocked by this pair.

Collective Efficiency in Two-Sided Matching

119

Table 1. Preferences relation Dave Eddy Frank Ann Betty Carol

2 3

2

3 3

2

mmm

1 3 2

1 1

1

3

mmm 2

1

Table 2. Preference relation Betty and Dave are removedfromTable. 1 Eddy Frank Ann Carol

2 1

1 2

1 2

2 1

If v^e are interested in yielding all stable matching, we can remove Betty and Dave from the preference list, yielding the reduced preference list given in Table 2. When all preferences are uniform - that is, when all males have the same preferences over females and vice versa - it is easy to see that a unique stable matching exists. To see this for monogamous matching systems, label the members of each sex by the preferences of the other sex (so that the best-ranking male mi is the best choice of the females, m2 is the second choice, etc.). Under this system, the only possible stable matching will be approved.

4

Proposed of New Algorithms

Before we state our algorithms, we define utility measures. In this section, we propose two new algorithms based on the hypothesis bounded rationality. In this paper, we propose a new algorithm for two-sided matching some compromise at individual levels. We also obtain large-scale properties of the proposed algorithm. It is an important issue how far agents seek their own interests in a competitive environment. There are overwhelming evidences that support peoples are also motivated by concerns for fairness and reciprocity. We will show that compromise which is individually irrational improves the welfare of the whole groups. The

120

Tomoko Fuku et al.

reasonable compromise level is obtained as the function of the size of the group so that the social utility should be maximized. There are two disjoint sets of agents, groups of men = {mj,..., rrin}, and women = {fi,...,fn}' Associated with each side is the number of positions they have to offer. Agents on each side of the market have (transitive and strict) preferences over agents on the other side, with presented a simple model in which college applicants have ordinal preferences over schools, and colleges have ordinal preferences over applicants. N : the size of each group T : the preference level of the partner to be matched (/ < T
[/,

=\-TJN

(2) Utility of a pair of agent / and agent j

Pij=U,xUj (3) Social utility

S =Z4

(4.1)

We propose a new algorithm based on concept of compromise of individual. 4.1 Matching with Aggregated Preference : IVIAP When each group is composed of N person, stable matching exists according to N. In addition, the more the scale of the group increases, the more the number of stable matching increases explosively. Because a stable matching exists a lot, according to the match dissatisfaction will be caused in a lot of people. To solve this problem, it thought the only match was produced. It's <Matching with Aggregated Preference : MAP>. The preference order is decided in the entire group. The matching is produced based on it. That is, the negotiation among individuals is not admitted, it is a topdown match method. The sum total of the score of women's preference order for each men mfSMis obtained. To similar, the sum total of the score of men's preference order for each women WjE W is obtained. The pair of order is made from the person with a high total score. 4.2 IVIodified Deferred Algorithm with Compromise : MDA Another algorithm is <Modified Deferred Algorithm with Compromise>. It's natural that the optimal strategy is to behave as if the individuals can profit the maximum. But like prisoner's dilemma game everyone knows, the social best optimum is to act in concert. Similarly, in the marriage problem we wonder if the so-

Collective Efficiency in Two-Sided Matching

121

cial optimal point exists when the individuals compromise acceptable level. It happens that no compromise result in the lowest social utility because of the large difference satisfaction with disappointment, and increasing matching paired slowly. Moreover, it is sure that more compromise result the lowest social utility because many individuals can not be satisfied the outcome. However we can compromise at individual level properly, most of individuals somewhat can increase their satisfaction. Necessarily, we suppose that we can get the most social utility by increasing the pairs with somewhat satisfaction. For the present, it delays the discussion of this algorithm's efficacy till later, we thought about the value of the most optimal compromising level. It follows the simulation result. <Stepl> Each man proposes to his T^ choice. Proposed woman accepts the proposal if it is within the acceptable level, otherwise she rejects it (holding is not permitted). <Step k> Each woman who is not matched increases the acceptable level by one. Any man who was rejected at step k-1 makes a new proposal to his most preferred woman mate who has not yet rejected him. A proposed woman accepts the proposal within the acceptable level, otherwise she rejects it. We lock the value of N(=1000), agents number. We increase the compromise level (x-axis) by 0.1, and plot the calculated value. We can get the parabola like Figure 1. Therefore, maximum point of social utility becomes the most optimum compromise level point. 922 ^ 920

I 918 o

(f)

914 912 20

40

60

80

100

Compromise Level

Fig. 1. The relationship between compromise level and Social Utility (N=1000)

122

Tomoko Fuku et al.

Fig. 2. The relationship between the compromising level and Nj the number of agent of each group The relation between compromise (C) and N is approximation by

C = 1.017 N'-'"'' = a^N

5

{a=\2)

(4.2)

Simulation Results

We generate the preference list of the agents at random. An agent belong to the each group is matched using three algorithm simulation (DA, MAP, MDA). Then, we compare about the distribution of availability about each agents or blocking pair, and the social utility. It follows the simulation result (N = 1,000), that is the distribution of availability.

Fig. 3. The utility distribution of men who propose to women These figures describe the utility distribution of each side. MAP is how to decide of which everyone consents because the preference list of the individual is

Collective Efficiency in Two-Sided Matching

123

consolidated and it decides the matching. However, the person with low effect exists from the person with high effect almost thoroughly, and, as a result, it becomes the one with considerably low effect of the entire society. Therefore, it is understood that a lot of top-down methods of not admitting the individual to negotiate invent a dissatisfied pair. Proposed Agents -DA

30

-MAP

10

MDA

MDA

20

MAP UxJdL^ii-LLk/li-J.ilx-..x^

u^#4i6^Mt I

0.2

0.4 .. ... 0.6 Utility

0.8

Fig. 4. The utility distribution of women who are proposed from men

To be sure, the Figure 3 shows that there are many agents near 1 utility in DA. Opposite, the Figure 4 show that agents utility distributes flatly. Moreover, the yaxis maximum value of the Figure 4 is 40, in contrast Figure 3 is 200. These are why DA is called the M-optimal (Proposing side optimal) algorithm. Then, Figure 4 shows that MDA distributes more flatly than DA. But the utility of both of side are modestly similar. In addition, almost of agents gather around 1. So, they can assume Pair-optimal and high utility. Additionally, the utility of Pair follows Figure 5.

Pairs 160

Pair

Fig. 5. Utility distribution of the Pairs

124

Tomoko Fuku et al.

This figure shows the distribution of utility which pairs have. Almost pairs satisfy the outcome of the matching. Especially, MDA heaps many pairs around utility 1. As pair utility, DA is not as well as proposing utility. That is proposed agents must accept all proposing without reference to the self-preference lists. It is natural that many proposing agents can satisfy the outcome, but proposed agents can not. In other word, DA have many satisfied proposing agents, few proposed agents. Consequently, the utility of the Pair hold lower than MDA. Then, we can get the higher utility of pair with some compromises or borderlines. Social Utility

Fig. 6. Comparison of social utility of three algorithms

We can describe the social utility by liner function. Figure 6 shows social utility in MDA is higher than in DA. Especially, the more the circle scale is bigger. When the circle scale is small, in MDA social utility can be lower, because compromise level is large percentage against N. In small circle scale social utility in DA can be higher than MDA. But we can get high social utility in MDA in large circle.

6

Theoretical Consideration of the Optimal Compromise Level

In section4, we obtained the relationship between the optimal compromise level and the population size by simulation which is obtain in (3.2). In this section, we obtain it theoretical analysis. We define the following terminologies: g(t) : The number of paired couples at the t step f(t) : The number of unmatched agents at the t step C : The compromise level We have the following initial conditions: g(0)=C'/(0)=^-C From the definition, we have the following recursive equations.

g(t) = f(t-\)x{C

+

t)INxf(t-\)IN

(6.1) (6.2)

Collective Efficiency in Two-Sided Matching 125 Fore the number of unmatched pair at the t-th step is obtained as /(O = f{t -1) X (1 - fit -\)x{C + t)IN'

(6.3)



In the case, the contents of preference lists each agents have are at random, paired couple arise at random, too. Each side scale is A^ agents, the compromise level is C, we suppose, g is the number of paired couples on the step,/describes the number of other free agents. <Stepl> Each N agents on one side propose the other side best partner, then, the probability of accepting proposal is CI N. So, on the first step, the number of paired-couple is gio) = NxC/N = C, (6.4) and the number of free agents (who are still single, non-couple) is fio) = N-gio) = N-C. (6.5) <Step t> The agents of f(t - i) propose the /^partners of each preference lists. The probability that the ^* partners of each preference lists is stillfreeis f{t-\)IN, (6.6) and the probability of the accepting proposal is (C-\-t)/N. (6.7) Then, the number of paired couple on the step t is g(t) = f(t-l)*(C + t)/N*f(t-l)/N\ (6.8) On this step, the number of free agents is

m = fit-Y,-g(t).

(6.9)

That is f(t) = fit-l)*(l-f(t-l)*(C + t)/N' (6.10) If we solve the recursive equation (6.3), we have C = l3^[N. Therefore we could get the nice approximation in (4.2).

7

(6.11)

Conclusion

Deferred Acceptance has universally used since it suggested for 40 years. To be sure, this is very brilliant method for two sided matching, but it has a demerit which is a M-optimal algorithm. This research pursued that we produced the moderate equality for each side. We suggested two algorithms. We use agent-based simulation, we inspect each algorithm's quality and social utility. We could show pair-optimal matching by using MDA. I mean, to be sure compromise behavior is an individually irrational, but is not in macro world. Compromise (Patient) behavior link collaboration, and eventuate high efficiency. The main reason is sure that

126

Tomoko Fuku et al.

much compromise profit better partner not the best partner. For many agents, the best partner exists only one, but better partner exist more. Many agents can moderately satisfy, therefore, social utility is getting better globally. In addition, the balance of proposing side and proposed side can keep well. It is sure that proposing side utility gets down totally, but commensurately proposed side utility is getting better. Globally, we can get higher social utility outcome. It may natural that compromise or borderline profit social utility globally, nevertheless DA is the best individually. But we will not insist the new algorithms performance. We suppose, when we design the rule or institution in many of two sided matching problems like to students and colleges, workers and firms, marriageable men and women, compromise which is irrational behavior can bring higher social payoff for many people and the rule itself. In conclusion, self-interested hypothesis vs human sociality hypothesis folows. How far agents seek their own interest in a competitive environment? There are overwhelming evidences that support people are also motivated by concerns for fairness and reciprocity. For this mentioned above question, we showed that compromise, an individually irrational behavior, improves the welfare of others.

References 1.

Crawford, V.P. and Knoer, E.M. (1981). Job matching with heterogeneous firms and workers. Econometrica, 49: 437-450. 2. Cunningham, E.J.A. and Birkhead, T.R. (1988). Sex roles and sexual selection. Anim. Behav., 56: 1311-1321. 3. Gale, D. and Shapley, L.S. (1962). College admissions and the stabihty of marriage. Am. Math.Monthly, 69: 9-15. 4. Gerd Gigerenzer, and Peter M. Todd. (1999). SIMPLE HEURISTICS THAT MAKE US SMART. 5. Knuth, D.E. (1976). Stable Marriage and its Relation to other Combinatorical Problems. CRM Proceedings and Lecture Notes Vol. 10, English language edition. Providence, RI: American Mathematical Society. Originally published in French under the title Mariages stables et leurs relations avec d'autres problems combinatoires. 6. Marie-jose Omero, Michael Dzierzawa, Matteo Marsili, Yi-Cheng Zhang (2004). Scaling Behavior in the Stable Marriage Problem, cond-mat/9708181 7. Michael Dzierzawa, Marie-jose Omero, (2000). Statistics of stable marriage, cond-mat /9708181 8. Mongell, S. and Roth, A.E. (1991). Sorority rush as a two-sided matching mechanism. Am. Econ. Rev.,81: 441-464. 9. Roth, A.E. and Sotomayor, (1990). Two-sided Matching: A Study in Game-theoretic Modeling and Analysis. Econometric Society Monographs No. 18. New York: Cambridge University Press. 10. Shapley, L.S. and Shubik, M. (1972). The assignment game I: The core. Int. J. Game Theory, 1:111-130.

Complex Dynamics, Financial Fragility and Stylized Facts Domenico Delli Gatti,^ Edoardo Gaffeo,^ Mauro Gallegati,^ Gianfranco Giulioni,^ Alan Kirman,^ Antonio Palestrini,® Alberto Russo ^* ^ ITEMQ, Catholic University of Milan, Italy, and SIEC, Universita Politecnica delle Marche, Ancona, Italy; DE and CEEL, University of Trento, Italy, and SIEC, Universita Politecnica delle Marche, Ancona, Italy; ^ DEA and SIEC, Universita Politecnica delle Marche, Ancona, Italy; GREQAM, EHESS, Universite d'Aix-Marseille III, France; ^ DSGSS, Universita di Teramo, Italy, and SIEC, Universita Politecnica delle Marche, Ancona, Italy

1

Introduction

In this paper we develop an agent based model of heterogeneous financially fragile agents that is able to replicate a large number of stylized facts, from industrial dynamics to financial facts, from business cycle to scaling behavior. Mainstream macroeconomics reduces the analysis of the aggregate to that of a single Representative Agent (RA) and is unable, by construction, to explain nonnormal distributions, scaling behavior or the occurrence of large aggregate fluctuations as a consequence of small idios5nicratic shocks. The ubiquitous RA, however, is at odds with the empirical evidence (Stoker 1993)^ is a major problem in the foundation of general equilibrium theory (Kirman 1992)^ and is not coherent

Corresponding author. E-mail: [email protected] A modeling strategy based on the representative agent is not able, by construction, to reproduce the persistent heterogeneity of economic agents, captured by the skewed distribution of several industrial variables, such as firms' size, growth rates etc. Stoker (1993) reviews the empirical literature at disaggregated level, which shows that heterogeneity matters since there are systematic individual differences in economic behavior. Moreover, as Axtell (1999: 41) claims: "given the power law character of actual firms' size distribution, it would seem that equilibrium theories of the firm [...] will never be able to grasp this essential empirical regularity." According to Hildenbrand and Kirman (1988, p. 239): "... There are no assumptions on [...] isolated individuals, which will give us the properties of aggregate behavior, which we need to obtain uniqueness and stability. Thus we are reduced to making assumptions at the aggregate level, which cannot be justified, by the usual individuaUstic assumptions. This problem is usually avoided in the macroeconomic literature by assuming that the economy behaves like an individual. Such an assumption cannot be justified in the context of the standard economic model and the way to solve the problem may involve re-

128

Domenico Delli Gatti et al.

with many econometric investigations and tools (Fomi and Lippi 1997)1 All in all, we may say that macroeconomics (and macroeconometrics) still lacks sound microfoundations, i.e. the attempt of reducing the analysis of the aggregate to the analysis of its constitutive elements has failed. There is no simple, direct, correspondence between individual and aggregate regularities. It may be that, in some cases, aggregate choices correspond to those that could be generated by an individual. However, even in such exceptional cases, the individual in question cannot be thought of as maximizing anything meaningful from the point of view of society's welfare. Indeed our approach is exactly the opposite from the representative individual approach. Instead of trying to impose restrictions on aggregate behavior, by using e.g. the first order conditions obtained from the maximization program of the representative individual, the claim is that the structure of aggregate behavior {macro) actually emerges from the interaction between the agents {micro). In other words, statistical regularities emerge as a self-organized process at the aggregate level: complex patterns of interacting individual behavior may generate certain regularity at the aggregate level. The idea of representing a society by one exemplar denies the fact that the organizational features of the economy play a crucial role in explaining what happens at the aggregate level. In this paper we propose a HIA (Heterogeneous Interacting Agent) approach, which methodologically differs from both Ke3niesian macroeconomics and classical microeconomics. We adopt a bottom-up methodology: from very simple behavioral rules at the individual level we obtain statistical regularities as emergent properties of the model (section 2), which feed back on the individual behavior. In section 3, we show that this approach fits very well a large number of stylized facts. Section 4 concludes.

2

The Model

The model consists of two markets: goods and credit. In each period, there is a "large", time-dependent, number of firms Nt producing a homogenous good. For the sake of simplicity, we assume that firms have no access to the equity market (equity rationing), so that their only sources of finance are internally generated thinking the very basis on which this model is founded." This long quotation summarizes the conclusion drawn by Arrow (1951), Sonnenschein (1972), and Mantel (1976) on the lack of theoretical foundations of the proposition according to which the properties of an aggregate function reflect those of the individual components. ^ If agents are heterogeneous, some standard procedures (e.g. cointegration. Grangercausality, impulse-response functions of structural VARs) loose their significance. Moreover, neglecting heterogeneity in aggregate equations generates spurious evidence of dynamic structure. The difficulty of testing aggregate models based on the RA hypothesis, i.e. to impose aggregate regularity at the individual level, has been long pointed out by Lewbel (1989) and Kirman (1992) with no impact on the mainstream (a notable exceptions is Carroll, 2001).

Complex Dynamics, Financial Fragility and Stylized Facts

129

funds and bank debt. The time / balance-sheet identity states that Ku =^ Au-\- Lu, where Lu are bank loans, while the law of motion of net worth is Au = Au.j + %, that is net worth in previous period plus (minus) profits (losses). Firms are characterized by different financial positions. For the generic firm i, the latter is summarized by the equity ratio, an, that is the ratio of its equity base or net worth to its capital stock, au = At/Ku. Firms use capital as the only input in a linear production technology, Yu = (pKu. Capital productivity ( ^ is constant and uniform across firms, and the capital stock never depreciates. The demand faced by each firm is stochastic: at any time t, individual selling prices are random outcomes around the average market price P/, that is Pit = UitPt, where the random variable u is uniformly distributed with mean 1 and finite variance."^ In this setting, the problem of the firm is the maximization of expected profits net of expected bankruptcy costs: r „ = ( ^ - g r , X v -^(?r,Kl

-A,_,K,)

(1)

where grKu are total variable costs, r being the real interest rate and g > 1 a constant capturing retooling and adjustment costs to be sustained each time period the production process starts; finally, c > 0 parameterizes bankruptcy costs.^ The optimal capital stock, obtained from (1), turns out to be a (non-linear) decreasing function of the interest rate and an increasing (linear) function of the net worth: Kf, = {'^-gru)/ic
(2)

As we said, firms can only raise external funds on the credit market. For the sake of simplicity, we assume that firms interact with an aggregate banking sector, modeled as the reduced form from the working of an oligopolistic banking industry. Because investment is financed by means of retained earnings and by new debt, the demand for credit expressed by the /* firm is equal to the stock of capital not covered by internal resources (equity base): Li=i<^-

gr-u )l{c
(3)

In order to determine the supply of credit and its allocation rule, we assume that there is a risk coefficient u - i.e. a threshold ratio of the banking sector's equity {E) to the total amount of credit extended - that the bank tries to target, either because of a strategy of risk management or as a consequence of prudential regula^ Since firms sell their output at an uncertain price which is only determined after investment and production have taken place, leveraged firms may go bankrupt. Bankruptcy of the /-th firm occurs if its net worth at time / becomes negative, an event which can be easily shown to occur if the individual price falls below a critical threshold. Bankrupt firms exit the market. The probability of bankruptcy, as one might expect, turns out to be an increasing function of the interest rate and the capital stock, and a decreasing function of the equity base inherited from the past. 5 Which are assumed to obey the quadratic form C^ = cYJf .

130

Domenico Delli Gatti et al.

tion on the part of monetary authorities. Therefore the aggregate supply of credit turns out to be approximately a multiple (l/u) of the banking sector's equity:

i;=(l/vK.,

(4)

Credit is allocated to each firm on the basis of the mortgage it offers, which is proportional to its size, and to the amount of cash available to serve debt. 4

= AL? (/:,_, /K,_^)+ (1 - A)^ (^,.1 /A,_i)

(5)

where Ku.] is the firm's capital stock (at the previous period), K^.j is the aggregate capital stock, A^.j is the firm's equity base, Af.j is the aggregate equity base, andO 1 to a probability which depends negatively on the average lending interest rate, l/(l+exp(//(r/ry"))), where ;; and ju are constants.. As the average interest rate increases, expected financial commitments become higher and, ceteris paribus, expected profits are lower, the consequence being that the number of entries shrinks.^

3

Stylized Facts

Agent-based simulations show that the HIAs framework described above is capable to reproduce a remarkable high number of empirically observed regularities, spanning from microeconomics to industrial dynamics to macroeconomics"^. In ^ Entrants' size in terms of their capital stock is drawn from a uniform distribution centered around the mode of the size distribution of incumbent firms: each new firm is endowed with an equity ratio equal to the mode of the equity base distribution of incumbents. "^ It is worthwhile to notice that there are no other models in the economic literature that can account for a so large Hst of empirical findings. According to Stiglitz (1992, p. 51), "the validity of a theory rests on the conformity not of one or two of its predictions to reality, but on all of its predictions". In particular, we obtain a "unified" explanation of stylized facts just replicating them by means of simulations of a HIAs model (that is, without making an assumption for each facts to explain). In other words, starting from simple behavioural rules the interaction of heterogeneous agents reproduces "simultaneously" a series of meso- and macro-regularities (as statistical properties of a self-organizing process)

Complex Dynamics, Financial Fragility and Stylized Facts

131

this section we proceed by listing a series of stylized facts regarding either microeconomic and macroeconomic dynamics, to subsequently check how successful our HIAs model is in replicating these empirical findings. For expositional convenience, we classify our stylized facts into three groups: (a) industrial dynamics, (b) financial facts and (c) business cycle. {di) Industrial dynamics facts (Sutton 1997; Caves 1998; Geroski, 1995): 1. Firms' size distribution is right skew and it is described by a power law (Axtell, 2001; Ramsden and Kiss-Haypal 2000); 2. Firms' growth rates are Laplace distributed (Stanley et al 1996; Bottazzi and Secchi 2003); 3. The probability of surviving is positively correlated with firm's size and firm's age; 4. Probabilities of entry and exit decrease with firms' size; 5. Number of firms firstly increases to a maximum and then tends to remain stable at a lower level; 6. There is a high correlation between entry and exit rates; 7. The variance of the aggregate is lower than that of the individual agents (Lee et al 1998); 8. There exists a Taylor rule (Taylor 1961) according to which the mean and the variance of certain of the characteristics of the firms (capital, employment, added value) evolve linearly through time (Gaffeo et al. 2005); (b) Financial stylized facts: 1. The rate of interest is a- or moderately /7ro-cyclical (Gallegati and Gallegati, 1996); 2. The distribution of the amount of loans is power law (Fujiwara 2003); 3. The distribufion of bad debts is Weibull distributed (Belli Gatti et al 2003a); 4. The distribution of firms' profit is power law (Fujiwara 2003); 5. The distribution of the rate of profit with respect to the equity ratio is asymmetric (Belli Gatti et al 2004); 6. The distribution of the equity ratio is not independent from firms' size (Belli Gatfi et al 2004); 7. A higher equity ratio is associated with lower volatility of profits (Belli Gatti et al. 2004); 8. The rate of return of the capital and the equity ratio are positively correlated (Belli Gatfi et al. 2004); 9. Financial ratios are good predictors of firms failure, i.e., the equity ratio deteriorates almost monotonically as the date of bankruptcy approaches (Beaver, 1966);

that we find in the empirical literature on industrial dynamics, financial and business cycle facts.

132

Domenico Delli Gatti et al.

(c) Business cycle facts: 1. The distribution of exits by age is exponential (Steindl 1965; Fujiwara 2003); 2. The rates of variations of cumulative rates are WeibuU distributed (Di Guilmi et al 2004a); 3. The duration of recession phases is power law, while expansions duration is exponential (Ausloos et al. 2004); 4. Firm size distribution shifts over the business cycle (Delli Gatti et al 2004); 5. The GDP autocorrelation of the simulated output (0.84) is quite close to the actual one (0.93) (Gallegati and Stanca 1999); 6. The GDP standard deviation of the simulated output (2.9) is quite close to the actual one (2.8) (Gallegati and Stanca, 1999); 7. Bankruptcies are power law distributed (Di Guilmi et al 2004b). In general, our model shows that the financial fi-agility of firms (and, consequently, of an economic system characterized by firm-bank relationships) is a very important factor in order to explain a lot of stylized facts^. The empirical evidence and our simulations show that there is a positive relationship between firms' size and financial soundness, here proxied by the equity ratio (stylized fact b6). Consequently, larger firms with high equity ratios are more likely to survive because of a lower probability of bankruptcy (stylized fact a3). In addition, financially sound firms grow faster than firms with low equity ratios. The information gathered from the distribution of the equity ratio over firms' size is decisive to the study of industrial dynamics and its interactions with business cycle, as we find that financial ratios are consistently a good predictor of firms breakdown, and therefore of exits due to bankruptcy. Both the model simulation and the empirical evidence show that the equity ratio deteriorates almost monotonically as the date of bankruptcy approaches (stylized fact b9). The central role of the equity ratio in explaining the interplay between industrial dynamics and business cycle fluctuations is further supported by its relationship with the profit rate (stylized fact b5). All in all, in order to better understand firm dynamics, there are reasons to go more deeply into the interplay between industrial and financial facts, as we suggest with our approach.

4

Conclusions

Our model aims at popularize the HIAs approach to economic modeling: starting fi*om very simple individual rule, agents interaction shape macro-relations as statistical regularities generated by a self-organized process. According to this view, the macroeconomy is a complex system composed of a large number of heterogeneous interacting agents. We presented a simple agent-based model of the levered For a more general discussion about stylized facts replicated by agent-based simulations (not limited to financial facts) see (Delli Gatti et al 2004).

Complex Dynamics, Financial Fragility and Stylized Facts

133

aggregate supply class developed by Greenwald and Stiglitz (1990), whose core is the interaction of heterogeneous financially fragile firms and a banking sector. Simulations of the model replicate surprisingly well an impressive set of stylized facts (from industrial dynamics to financial facts, from business cycle to scaling).

Acknowledgements We thank Corrado Di Guilmi and Edmondo Di Giuseppe for excellent research assistance. Comments from participants to seminars at Santa Fe Institute, the Bank of Italy, Unicredit Bank, Catholic University of Milan, Milano Bicocca, Roma "La Sapienza", Rimini, Trento, Pisa and Fribourg are gratefully acknowledged.

References Amaral L.A.N., Buldyrev, S.V., Havlin, S., Leschhom, H., Maass, P., Salinger, M.A., Stanley, H.E., Scaling Behavior in Economics: I. Empirical Results for Company Growth, Journal de Physique I France, 7, 621-633 (1997). Arrow, K., Social Choice and Individual Values, Wiley & Sons, New York (1951). Audretsch, D.B., Innovation and Industry Evolution. MIT Press, Cambridge, MA (1995). Ausloos, M., Miskiewicz, J., Sanglier, M., The Durations of Recession and Prosperity: Does Their Distribution Follow a Power or an Exponential Law? mimeo (2004). Axtell, R., The Emergence of Firms in a Population of Agents: Local Increasing Returns, Unstable Nash Equilibria, and Power Law Size Distributions, CSED Working Paper No.3 (1999). Axtell, R., Zipf Distribution of U.S. Firm Sizes, Science, 293, 1818-1820 (2001). Beaver, W.H., Financial Ratios as Predictors of Failure, in Empirical Research in Accounting: Selected Studies, supplement to Journal of Accounting Research, HAW {1966). Bottazzi, G., and Secchi, A., Why Are Distributions of Firms Growth Rates Tent-Shaped?, Economic Letters, 80, 415-420 (2003). Brock, W., Scaling in Economics: A Reader's Guide, Industrial and Corporate Change, 8,409-446 (1999). Carroll, C , Requiem for the Representative Consumer? Aggregate Implications of Microeconomic Consumption Behavior, American Economic Review, 90, 110115(2001). Caves, R.E., Industrial Organization and New Findings on the Turnover and Mobility of Firms, Journal of Economic Literature, 27, 1947-1983 (1998).

134

Domenico Delli Gatti et al.

Cooley, T.F, Quadrini V., Financial Markets and Firm Dynamics, The American Economic Review, 91, 1286-1310 (2001). Delli Gatti, D., Di Guilmi, C , Gaffeo, E., Giulioni, G., Gallegati, M., Palestrini, A., Business Cycle Fluctuations and Firms' Size Distribution Dynamics, forthcoming in Advances in Complex Systems (2004). Delli Gatti, D., Di Guilmi, C , Gaffeo, E., Giulioni, G., Gallegati, M., Palestrini, A., A New Approach to Business Fluctuations: Heterogeneous Interacting Agents, Scaling Laws and Financial Fragility, forthcoming in Journal of Economic Behaviour and Organization (2003 a). Di Guilmi, C , Gaffeo, E., Gallegati, M., Empirical Results on the Size Distribution of Business Cycle Phases, PhysicaA, 333, 325-334 (2004a). Di Guilmi, C , Gallegati, M., Ormerod, P., Scaling Invariant Distributions of Firms' Exit in OECD Countries, Physica A, 334, 267-273 (2004b). Dunne, T., Roberts, M.J., Samuelson, L., The Growth and Failure of US Manufacturing Plants, Quarterly Journal of Economics, 104, 671-698 (1989). Evans, D.S., Tests of Alternative Theories of Firm Growth, Journal of Political Economy, 95, 657-674 (1987). Fomi, M., Lippi, M., Aggregation and the Microfoundations of Dynamic Macroeconomics, Oxford University Press, Oxford (1997). Fujiwara Y., Zipf Law in Firms Bankruptcy, mimeo (2003). Gaffeo, E.„ Di Guilmi, C , Gallegati, M., Russo, A., On the Mean/Variance Relationship of the Firm Size Distribution: Evidence and Theory, mimeo (2005). Gallegati, M., Gallegati, M., Volatility and Persistence of Fluctuations: Individual Production Series in Italy, 1890-1985, Applied Economics, 27, 677-688 (1996). Gallegati, M., Stanca, L., The Dynamic Relation between Financial Conditions and Investment: Evidence from a Panel Data, Industrial and Corporate Change, 8, 551-572(1999). Geroski, P.A., What Do We Know About Entry?, International Journal of Industrial Organization, 13, 421-440 (1995). Gibrat, R., Les inegalites economiques; applications: aux inegalites des richesses, a la concentration des entreprises, aux populations des villes, aux statistiques des families, etc., d'une hi nouvelle, la hi de I'effet proportionnel, Librairie du Recueil Sirey, Paris (1931). Greenwald, B., Stiglitz, J., Financial Markets Imperfections and Business Cycles, Quarterly Journal of Economics, 108, 77-113 (1993). Greenwald, B., Stiglitz, J., Macroeconomic Models with Equity and Credit Rationing, in Information, Capital Markets and Investment, edited by Hubbard, R., Chicago University Press, Chicago (1980). Grossman, S., Stiglitz, J., On the Impossibility of Informationally Efficient Markets, American Economic Review, 70, 393-408 (1980). Hildebrand, W., Kirman, A., Equilibrium Analysis, North-Holland, Amsterdam (1988). Ijiri, Y, and Simon, H.A., Business Firm Growth and Size, American Economic Review, 54, 77-89 (1964).

Complex Dynamics, Financial Fragility and Stylized Facts

135

Kirman, A., Whom or What Does the Representative Individual Represent?, Journal of Economic Perspectives, 6, 117-36 (1992). Lee, Y., Amaral L.A.N., Canning D., Meyer M., Stanley, H.E., Universal Features in the Growth Dynamics of Complex Organizations. Physical Review Letrer^, 81, 3275-3278 (1998). Lewbel, A., Exact Aggregation and the Representative Consumer, Quarterly Journal of Economics, 104, 621-633 (1989). Mantel, R., nomothetic Preferences and Community Excess Demand Y\xx\.ziiorvs. Journal of Economic Theory, 12, 197-201 (1976). Ramsden, J., Kiss-Haypal G., Company Size Distribution in Different Countries, PhysicaA, 277, 220-227 (2000). Russo, A., Delli Gatti, D., Gallegati, M., Tecnological Innovation, Financial Fragility and Complex Dynamics, in Salvatori, N., Economic Growth and Distribution: On the Nature and Causes of the Wealth of Nations, Edward Elgax, forthcoming. Sonnenschein, H., Market Excess Demand Functions, Econometrica, 40, 549563 (1972). Stanley, M.H.R., Amaral L.A.N., Buldyrev, S.V., Havlin, S., Leschhom, H., Maass, P., Salinger, M.A., Stanley, H.E., Scaling Behaviour in the Growth of Companies, Nature, 379, 804-806 (1996). Steindl, J., Random Processes and the Growth of Firms, Hafner, New York (1965). Stiglitz, J.E., Methodological Issues and the New Keynesian Economics, in Macroeconomy: A Survey of Research Strategies, edited by Vercelli, A., Dimitri, M., Oxford University Press, Oxford (1992). Stoker, T., Empirical Approaches to the Problem of Aggregation Over Individuals, Journal of Economic Literature, 21, 1827-1874 (1993). Sutton, J., Gibrat's Legacy, Journal of Economic Literature, 35, 40-59 (1997). Taylor, L.R., Aggregation, Variance and the Mean, Nature, 189, 732-735 (1961).

Noisy Trading in the Large IVIarl^et Limit Mikhail Anufriev^ and Giulio Bottazzi^ ^ S.Anna School for Advanced Studies, Pisa manouf r i e v @ s s s u p . i t ^ S.Anna School for Advanced Studies, Pisa b o t t a z z i @ s s s u p . i t

Summary. This paper analyzes to what extent and how the trading activity of a group of heterogeneous agents can be described, in the aggregate, as the result of the investment decision of a single "representative" agent. We consider a two-asset pure exchange economy populated by CRRA traders whose individual demands are functions of the past market history. If individual choices are expressed as noisy versions of a common behavior, and the number of agents is large, one can consider the Large Market Limit of the economy and reduce the model to a low-dimensional stochastic system. We investigate the goodness of this approximation under different market conditions and different agents ecologies. The results of the analysis can be used in the study of the general case with an arbitrary number of heterogeneous agents.

1 Introduction This paper is based on the pure exchange, two-assets economy presented and discussed at length in [1]. It shares some features with early contributions [3,9] and constitutes an analytically tractable model of artificial market. Following a long tradition in the agent-based literature^ we populate this economy with heterogeneous traders. The individual demands for the risky asset are expressed as fractions of their wealth. This implies, as a direct economic consequence, that price and agents' wealths are determined at the same time and that agents with different wealths have different impact on the price formation. One can refer to this particular choice for the description of agents behavior as CRRA (Constant Relative Risk Aversion) framework. We confine the present analysis to the case of "adaptive" agents who, in a spirit analogous to the "technical trading" observed in the real markets, formulate their investment choices as functions of the past aggregate market performances. Inside this general framework, we analyze the conditions under which the aggregate market dynamics resulting from the interactions of many different traders may or may not be described as if only one agent, whose behavior can be characterized as average, operates in the market. Thus, the specific goal of this paper lies in an ^ For the goals and "representative" examples of this stream of research see [8]. In particular, models employing analytical methods were recently reviewed in [6].

138

Mikhail Anufriev and Giulio Bottazzi

identification of the precise "boundaries" outside which the representative agent approach fails. This question is important in the light of the extensive critics to the use of representative agent in the economic models, see e.g. [7] or [5]. Relatedly, our analysis deals with the important issue of the impact of noise in individual micro-behavior on the aggregate macro-dynamics. The question, whether the micro noise is important for the macro level or, instead, it is washed out as in a sort of "central limit theorem", always attracted the highest attention in economics, both from the theoretical and empirical point of views (see e.g. [4]).

2 Model Structure We consider N agents trading in discrete time in a two-asset economy with a riskless asset giving a constant interest rate r/ > 0 and a constant supply (normalized to 1) of risky asset paying a random dividend Dt. The price of the riskless asset isfixedto 1 and at each time step a price Pt of the risky asset isfixedthrough market clearing. Let Wt^n stand for the wealth of agent n at time t and xt^n for its share invested in the risky asset. With after-trade payment of both dividends and riskless interest, the individual wealth evolves accordingly to Wt+i,n

= (1 - Xt^n) m , n (1 + Tf) +

^'"

' ' " (Pt+1 + A + l ) ,

(1)

while the market clearing condition reads N

Y,Xt^nWt,n=Pt

.

(2)

n=l

We assume that investment share xt,n does not depend on the contemporaneous price and wealth which is consistent with the demand derived under constant relative risk aversion (CRRA) type of behavior. Therefore, price and wealth are simultaneously determined, as it also happens in the real markets. Equations (1) and (2) give the evolution of the state variables Wt^n and Pt over time implicitly, provided that the set of investment shares {xt^n} is specified. Concerning the latter we further assume that for each agent n there exists a deterministic smooth investmentfunction fn such that Xt,n = fn{1t-l)

,

(3)

whereIt-i = {Pt-i, Pt-2,... } is the information set commonly available to agents at time t. Agents' investment decisions evolve following the individual prescriptions and taking into consideration past market performance. This behavior is similar to the "technical trading" attitude observed in real markets. The knowledge about the ftindamental dividend process, being complete and time invariant, is not explicitly inserted in the information set, rather is considered embedded in thefimctionalform Of/n.

Noisy Trading in the Large Market Limit

139

With some algebra, and under suitable conditions discussed in [1], the implicit dynamics described in (1) and (2) can be made explicit. In terms of rescaled variables m,n = Wt,n/(1 + rf)\ pt = Pt/{1 + TfY and et = Dt/{Pt-i (1 + r / ) ) the price growth rate r^+i = pt+i/pt — 1 is defined as n+i =

{xt^.i-xt)^-\-et+i{xtXt^i)^ 7—j: ^^;

,

(4)

{xt{l-xt+i))^ where (a)^ denotes the wealth weighted average at time t of some agent specific variable an (^a)^ = —r^=k—V_blL ^^^j^^

^^^^ ^

^^ere

wt = ^

Wt,n

n=l

represents the total wealth in the economy and (pt,n = '^t,n/'^t wealth share.

is the n-th agent's

3 Large Market Limit Having obtained the explicit dynamics for the evolution of price and wealth returns, one would be interested in asymptotic behavior of the system. Return evolution in (4) depends, however, on the evolution of the distribution of weights (^t,n- This is why the problem of determination and stability analysis of steady-states of the return is not trivial. Analysis in [2] represents one possible way to deal with such problem. The return dynamics is supplemented by an explicit evolution of the weights and corresponding multi-dimensional system (whose dimension grows linearly with N) is analyzed. Since investment choices of the agents depend on the return history, in equilibrium these choices are fixed. For correct definition of equilibrium also the distribution of weights has to be invariant under time shift operator. It, essentially, explains why in all generic equilibria identified in [2] only one of the agents has positive wealth share (/?* = !, and why in all other equilibria, with two or more agents having positive wealth shares, all such "survivors" have the same wealth growth rates. It becomes then interesting to investigate if some approximated description of the economy can be obtained using only an "aggregate" account of agent's decision. Indeed, in general terms, for the invariance of the return provided by (4), we need a fiilfiUment of two requirements. First requirement is an invariance (over time) of the investment shares averaged on the distribution of weights {xt^i)t

= {xt)t .

(5)

Since the distribution is also evolving, as a second requirement we need an invariance of this distribution ( x m ) ^ = (^t+i)^+i • (6) The following statement allows to see the implications of this latter requirement about the distribution invariance.

140

Mikhail Anufriev and Giulio Bottazzi

Lemma 1. The wealth-weighted average of market investment choices at time t-\-l, (^xt^i) ^ can be computed using present wealth shares (pt^n according to (^t+i>^_,_i = {^t-hi)t + (^t+i + et+i) ((^t+i ^t)t - (^t+i)i {xt)^)

.

(7)

Now it is clear that if equalities (5) and (6) are satisfied for all t, then return dynamics (4) can be simplified to rt+i = et+il-{xt+i)^ which is, indeed, invariant if (5) and (6) hold. Conditions (5) and (6) are sufficient for an appropriate definition of the equilibrium of the return dynamics. In particular, they both are satisfied in those equilibria which are found and analyzed in [2]. Since they are expressed through some aggregate quantities, it becomes interesting to investigate if some approximated description of the economy can be obtained using only an "aggregate" account of agent's decisions. Consider the following Assumption 1 There exist a function F such that {xt). = F ( r t _ i , r t _ 2 , . . . )• Moreover, the investment choices at successive time steps satisfy {xtxt+i)^

= {xt)^{xt+i)^.

(8)

Notice that the meaning of this Assumption is twofold: it requires that the average investment choice is described by some time-invariant fimction of the past information and, at the same time, that the sample average of the product of the investment shares can be replaced by the product of their sample averages. From Lemma 1 is follows that the last requirement is equivalent to condition (6). When Assumption 1 holds, the dynamics of the economy can be described in terms of the sole aggregate variable {xt)^. It reads (^t+i)^^.i

=F(n,rt_i,...)

{^W)M

- {^t)t + et+i (xt), {xw)t^i {xt)^-{xt)^{xt+i)^_

(9)

/t+i

This system coincides with the dynamics generated by a single trader, and function F describes the "behavior" of this notional "representative" agent. Properties of this system for different specifications of function F are analyzed in [1] and [2]. There are obvious cases when Assumption 1 is satisfied. It is so, e.g., in the case of homogeneous behaviors, when all agents possess the same beliefs and preferences, so that at each time step xt^n = {xt)^i Vn. A more interesting example is constituted by the case of "purely noisy" agents. Suppose that at each time step the investment shares of the N agents are randomly and independently drawn from a common distribution with average value x. In this

Noisy Trading in the Large Market Limit

141

case the information set is irrelevant and in the A^ ^ oo limit, if the share of wealth of each agent goes to zero, one has {xt)^ = x. In this case, (8) is clearly fulfilled, so that conditions of Assumption 1 are replicated, and the dynamics of returns reduces to rt+i = et+i

l-x The price returnfluctuatesaround some average value which depends positively both on the dividend yield and on the average investment share x. We can generalize the "pure noise" model and assume that the investment decision of each agent is a "noisy" version of a basic common choice; formally Xt,n = F{Xt-l) + et,n.

(10)

where the e's are independent (across time and across different agents) random variables with zero mean. When N ^ oothQ sample of independent random variables becomes large, the sample average converges to the average of the variable distribution, so that one can expect that {xt)^ -^ F{Xt-i). Moreover, since shocks are independent across time, one can also expect the fulfillment of (8) for the corresponding limits. Since any "theoretical" relation for the averages of random variables is violated onfinitesamples, we consider this behavior as a sort of "limiting" situation and call it a Large Market Limit (LML). Return dynamics in the LML is approximated by the dynamics in the case of single agent present in the market. This is the reason why the existence of the LML can be linked with the representative agent approach. It is important to mention that inside setting outlined in Section 2 there are many situations when the agent possessing some average behavior does not represent the long-run economic outcome. For instance, in the market with completely different investment functions fn, economy asymptotically follows the behavior of only one agent, however he is not a "representative" agent in the sense of standard economic theory. If, instead, the agents behaviors are described by (10), then the notional agent possessing investment function F is, indeed, "representative".

4 Scope of the LML Application The natural question concerns the scope of application of the Large Market Limit. One would like to know under which conditions on function F and on idiosyncratic noise components et,n in (10) Assumption 1 is satisfied over time. Note that all averages which we consider are weighted with respect to individual wealth shares (/?t,n» which are changing over time. Consequently we cannot straightforwardly rely on any limit theorem and, plausibly, the identification of the appropriate conditions under which Assumption 1 can be applied entails the analysis of the individual wealth dynamics. Additional complication comes from the fact that general setting of the model in the LML is characterized by two different sources of randomness:first,the idios}^!cratic noise components et,n appearing in the definition (10) of the agent investment

Mikhail Anufriev and Giulio Bottazzi

142

N=5, 0=0.001 -—>^— N=50, 0=0.001 - -B-

0.014

0 012

^

+ t

f, t, *

001

. ^ . *-

1°

+ • a^

°'"

-

^ ^fl

""a

" •

° Q

„ *

^OE

S'^ +

^'

0 008

-

0.006

30

35

40

0.03 J

45

50

N=5, 0=0.001 N=50, 0=0.001

0.025 h]

30

--- _

35

40

45

50

55

N= 5, 0=0.01 N=5 0=0.001

0.08

60

::::: _

0.06

0.02 I

0.04 0.015 I

g

^

-1 f \ l

,1

0 -0.02

"\

A J

j \^

K /\ "

0.02

- Pf

T'^ *'T/

' \ I

\t

Kl

' ^"Sj

\l \

-0.04

40

45

50

30

35

40

45

50

55

60

Fig. 1. Convergence to the deterministic LML with increase of the number of agents (Left Panels) and with decrease of the variance of the agent-specific noise (Right Panels). Simulations are for pure noise model (Upper Panels) and for model with linear increasing strategy (Lower Panels).

choice and, second, the exogenous stochastic dividend process {et}. Each of them may affect the dynamics. With the help of computer simulations we try to analyze the impact of both these noise sources here. We do it in tw^o steps. In the first step we consider constant yield scenario and compare the dynamics generated by a finite population of noisy agents with the deterministic dynamics of the "representative" agent in the LML. In the second step we will do the same comparison in the stochastic environment, i.e. when dividend yield is random. As we illustrate below, it turns out that even if differences between the limiting deterministic behavior and the actual stochastic implementation of the model do exist, they become smaller when the size of the population of agents increases, when the variance of the agent-specific noise Ct^n decreases, and when the variance of the dividend process et decreases. 4.1 LML in the Deterministic Skeleton Let us start with the case when dividend yield is constant and focus on the effect of the noisy components in agents investment function. We model noise et,n in (10) as normally distributed independent random variable with zero average and standard deviation cTg. We confine the analysis on the case of linear fimctions. Notice that the

Noisy Trading in the Large Market Limit

143

-0.01 h -0.02 [

30

35

40

50

55

60

0.03 f

N= 5, 0=0.01 N=5 0=0.001

0.08

0.025

0.06 0.04

—t-

'I •

+

0.02

+ +

^/^fTg-w^f-s^fS^^ 0 • ^T^^ °

+

+a

""*^^^ ^^ ""l

-0.02 -0.04

ii

Fig. 2. Convergence to the stochastic LML with increase of the number of agents (Left Panels) and with decrease of the variance of the agent-specific noise (Right Panels). Simulations are for pure noise model (Upper Panels) and for model with linear increasing strategy (Lower Panels).

same analysis, therefore, can be applied to the function of any type in the appropriately small neighborhood of the equilibrium. The results of simulations are presented in Fig. 1, where the solid thick line represents the dynamics in the LML and dotted lines show the return trajectories in the market with many noisy agents. Since the dividend yield is constant, the LML gives a deterministic system in this case. We consider two examples, the pure noise model, which corresponds to (10) with constant function F (the two upper panels), and model with increasing linear function F (the two lower panels). For definiteness, in the first case we fix F = 0.2, while in the second case F{r) = 0.199 + 0.1 r. In both cases the LML deterministic system possesses unique stable equilibrium with r* = 0.01. When function F is constant, convergence to this equilibrium happens almost immediately, while if this function is increasing, there is some transitional period during 40 time steps or so. Fig. 1 shows that if noise is added to the individual demand functions, the observed returns fluctuate. However, these fluctuations are around the corresponding LML-trajectory and, in particular, around equilibrium return after transition period. In the left panels we illustrate the role of the population size, so the standard deviation of the noise terms ae is fixed. We observe that the variance of the fluctuations decreases with the increase of the population size N. In the right panels we, instead.

144

Mikhail Anufriev and Giulio Bottazzi 0.1 0.01 0.001 le-04 I le-05 [ le-06 [ le-07

•••0-.

le-08 le-09 10

100 Population Size

Fig. 3. Average over 100 simulations of the sample mean (boxes) and standard deviation (circles) of the deviation of returns from the LML system as function of the number of agents. Different lines correspond to different values of the variance a^.

fix the number of agents and consider two cases with different standard deviation ae. If this deviation decreases, the variance offluctuationsdecreases as well. Summing it up, in the case of constant dividend yield the dynamics of return generated by the multi-dimensional system of similar (within to the noise) agents converges to the steady-state of the deterministic system under the LML. Since conditions for stability of the latter system are analyzed elsewhere (see [1, 2]), corresponding results can be directly applied to the stability analysis of the many agent case. 4.2 LML for Stochastic Dividend

If the dividend yield is random variable, the dynamics in the LML becomes stochastic. In Fig. 2 we address the question about validity of the LML approximation for this case. As before two upper panels show the simulations for the system with F = 0.2, whereas two lower panels correspond to the case F{r) = 0.199 + 0.1 r. In all simulations the yield of dividend is i.i.d. variable, so that the dynamics in the LML, represented by the thick solid line, is not constant even after transition period. According to visual impression, both when the number of agents becomes large (the left panels) and when the variance of the agent-specific noise decreases (the right panels), the trajectories of the LML become closer to the trajectory of the multi-agent system. In order to obtain some quantitative measure of agreement between the LML and the actual dynamics simulated with both agent-specific and dividend noises, we

Noisy Trading in the Large Market Limit

145

computed the first two moments of the deviation between these two trajectories. In Fig. 3 we report the averages of these moments over 100 independent simulations. It is clear that both an increase in the number of agents and a decrease in the variance of et,n lead to the convergence of the many-agent dynamics towards the LML.

5 Conclusion We introduced in this paper the Large Market Limit which allows to simplify the analysis of the agent-based model built inside the CRRA framework. The LML is a low-dimensional system whose equilibrium and stability analysis can be performed. Through the numerical simulation we showed that the LML provides, at least locally, a reasonable approximation of the original model also for a moderately small (N ~ 50) population of agents. If the LML is, indeed, a good approximation for the dynamics with noisy agents, then the analysis performed in [2] can be extended. There we gave a complete characterization of the dynamics when arbitrary number of heterogeneous CRRA agents operate in the market. The investment functions of them were "frozen", however, so that the imitation of behavior was forbidden. If such imitation took place, the market would end up in the situation like considered in the present paper. This fact, and also the general interest to the question of the limits of the "representative" agent metaphor, are the main driving forces of this paper.

References 1. Anufriev M, Bottazzi G, Pancotto F (2004) Price and wealth asymptotic dynamics with CRRA technical trading strategies. LEM Working Paper 2004/23, Scuola Superiore Sant Anna, Pisa 2. Anufriev M, Bottazzi G (2005) Price and wealth dynamics in a speculative market with an arbitrary number of generic technical traders. LEM Working Paper 2005/06, Scuola Superiore Sant'Anna, Pisa 3. Chiarella C, He X (2001) Asset price and wealth dynamics under heterogeneous expectations. Quantitative Finance 1:509-526 4. Fomi M, Lippi M (1998) Aggregation and the microfoundations of dynamic macroeconomics. Oxford University Press, Oxford 5. Gallegati M, Kirman A (1999) Beyond the Representative Agent. Edward Elgar, Oxford 6. Hommes CH (2005) Heterogeneous agents models in economics and finance. In: Judd K, Tesfatsion L (eds) Handbook of Computational Economics II: Agent-Based Computational Economics. Elsevier, North-Holland (forthcoming) 7. Kirman A (1992) Whom or what does the representative agent represent? Journal of Economic Perspectives 6:117-136 8. LeBaron B (2000) Agent-based computational finance: suggested readings and early research. Journal of Economic Dynamics and Control 24:679-702 9. Levy M, Levy H and Solomon S (2000) Microscopic simulation of financial markets. Academic Press, London.

Emergence in Multi-Agent Systems: Cognitive Hierarchy, Detection, and Complexity Reduction part I: Methodological Issues Jean-Louis Dessalles^ and Denis Phan^ ^ ENST, Paris, France j ld@enst. f r ^ CREM, University of Rennes I, France denis . phan@univ-rennesl. f r

1 Introduction In a pioneering book on "artificial society" and multi-agent simulations in social sciences, (Gilbert and Conte 1995) put the emphasis on "emergence" as a key concept of such approach: "Emergence is one of the most interesting issues to have been addressed by computer scientists over the past few years and has also been a matter of concern in a number of other disciplines, from biology to political science" (op.cit. p.8). More recently. Agent based Computational Economics (ACE) put the emphasis on the question of emergence, following for instance (Tesfatsion 2002a) or (Axtell and Epstein and Young 2001) The present paper provides a formal definition of emergence, operative in multi-agent framework designed by Agent Oriented Programming, and which makes sense from both a cognitive and an economics point of view. Starting with a discussion of the polysemous concept of emergence, the first part of this paper is dedicated to clarifying the question by focussing on the problem of modelling cognitive agents in artificial societies. The key questions are introduced by way of a paradigmatic example. The second part of this paper is dedicated to introducing and discussing operative definitions and related implications. In order to illustrate our formal definition of emergence, a companion paper (Phan and Galam and Dessalles 2005) discusses the ACE population game model of (Axtell and Epstein and Young 2001) and builds a multi-level-model based on the formal framework introduced in this paper.

2 From Emergentism to Emergent Behaviour in ACE IVIodel In this section, we first discuss different definitions of emergence, and the related background. In order to focus on the problem of modelling cognitive agents in artificial society, we next considering a paradigmatic example, and briefly discuss Schelling's model of spatial segregation (Schelling 1969, 1971, 1978), which is a pioneering study of an emerging social phenomenon in social science.

148

Jean-Louis Dessalles and Denis Phan

2.1 Emergence: One Word, Several Meanings The notion of emergence has several meanings. In the vernacular language, emergence denotes both a gradual beginning or coming forth, or a sudden uprising or appearance; to emerge also means to become visible, in example, emergence denotes the act of rising out of afluid.This latter sense is close to its Latin roots, where emergere is the opposite of mergere: to be submerged. In the following, we relate the "act of rising out" to the arising of some phenomenon in a process, and note the fact that to become visible presupposes some observer. In other words, the common sense of emergence is linked to the meaning of a process that produces some phenomenon that might be detected by an observer. In the field of science, emergence has been used by Newton in optics. By the 19th century the word "emergent" is introduced into the fields of biology and philosophy. In the latter, Emergentism has a long history, from Mill's chapter: "Of the Composition of Causes" in System ofLogic (1843) to the contemporary debates about the philosophy of mind, known as "the mind body problem". For a synthesis, see among others: (McLaughlin 1992,1997; Van de Vijver 1997; Emmeche and Koppe and Stjemfelt 1997). Philosophical emergentism deals with questions of both reductionism and holism. (Lewes 1874) for instance places emergence at the interface between levels of organisation. For descriptive emergentism, the properties of the "whole" cannot be defined by the properties of the parts, and results in part from some irreducible macro causal power. In this debate around the definition of emergence, some authors have proposed to distinguish between different kinds of emergence, as for example "nominal", "weak" and "strong" emergence for (Bedau 1997, 2002), or "weak" "ontological", and "strong" emergence for (Gillet 2002a-b). Both authors refer to debates about reductionism as well as about the so-called mind-body problem, discussing in particular the notion of Supervenience, introduced by (Davidson 1970,1980) and discussed by (Kim 1992, 1993, 1995, 1999) from the point of view of emergence. As "weak" emergence deals with upward causation and reductionism, Gillet and Bedeau relate "strong emergence" to the question of "downward causation" (Kim 1992; Bedau 2002) or "macro-determinism", widely advocated by (Sperry 1969, 1986, 1991) to deal with the mind-brain interactions, and by (Campbell 1974) to deal with hierarchically organized biological systems. According to strict downward causation, the behaviour of the parts (down) is determined by the behaviour of the whole (up). For instance, parts of the system may be restrained by some act in conformity with rules given at the system level. Causation would come "downward" in conformity with a holist principle rather than upward, according to a reductionist principle. In this paper, we do not address theses questions directly, as we limit ourselves to discussing social behaviours in artificial societies; but the opposition between downward versus upward causation proves to be a central one in the field of social sciences. According to (Granovetter 1985), the sociologist's approach would be "over socialized" (downward) while the economist's approach would be "under socialized" (upward/methodological individualism). Currently, both approaches have been sophisticated and are often mixed. The present paper is an attempt to integrate them in one single framework, in which the 'whole' is a collective of agents (up-

Emergence in Multi-Agent Systems

149

ward causation / methodological individualism), but the agents are in are to some extent constrained by the whole (downward causation), by way of the ''social dimension'' of their belief as well as their perception of social phenomena. For the purpose of this paper, we rely on the distinction, proposed by (Muller 2002) in the field of multi-agent systems, between "weak" and "strong" emergence. The latter refers to a situation in which agents are able to witness the collective emergent phenomena in which they are involved, which opens the road for both upward and downward causation. In Agent based Computational Economics, "emergence" is strongly related to the Santa Fe approach to complexity (SFAC). In accordance with descriptive emergentism, SFAC calls "emergence" the arising at the macro level of some patterns, structures and properties of a complex adaptive system that are not contained in the property of its parts. But conversely, emergence can often be explained by upward mechanisms. Interactions between parts of a dynamic system are the source of both complex dynamics and emergence. An interesting part of the emergence process concerns the forming of some collective "order" (coherent structures or patterns at the macro level) as a result of agents' interactions within the system's dynamics, in the presence of a specific attractor. For the observer, this collective order makes sense by itself and opens up a radically new global interpretation, because it does not initially make sense as an attribute of the basic entities. Formally, in multi-agent systems, emergence is a central property of dynamic systems based upon interacting autonomous entities (the agents). The knowledge of entities' attributes and rules is not sufficient to predict the behaviour of the whole system. Such a phenomenon results from the confrontation of the entities within a specific structure of interaction. That is, better knowledge of the generic properties of the interaction structures would make it easier to have better knowledge of the emergence process (ie. morphogenetic dynamics). From this point of view, to denote a phenomenon as "emergent" does not mean that it is impossible to explain or to model the related phenomenon. For this reason (Epstein 1999) uses the word "generative" instead of "emergent" in order to avoid a philosophical debate about emergence. Various attempts have been made to define emergence in an "objective" way. Some definitions refer to self-organisation (Varela and Thompson and Rosch 1991), to entropy changes (Kauffman 1990), to non-linearity (Langton 1990), to deviations from predicted behaviour (Rosen 1985; Cariani 1991) or from symmetry (Palmer 1989). Other definitions are closely related to the concept of complexity (Bonabeau et al. 1995a, 1995b; Cariani 1991; Kampis 1991). In statistical physics (Galam 2004), as well as for models in economics or social sciences explicitely based upon theses models (see for instance (Durlauf 1997, 2001) and the pioneering work of (Galam and Gefen andShapir 1982), emergence may be related with an order parameter which discriminates between at least two phases, each one with a different symmetry associated respectively to a zero and non-zero value of the order parameter. Each problem has its specific order parameter. For instance in the Ising model, where individual spins can takes the value {—1, -f 1}, the order parameter is the magnetization M, given by the sum of all the spin values divided by their total number. When

150

Jean-Louis Dessalles and Denis Phan

M = 0, the state is paramagnetic, i.e. disordered in the spin orientations, while long range order appears as soon as M 7^ 0. A majority of spins are then oriented along either —1 or + 1 , and an order is likely to emerge. Two ordered phases are thus possible in principle, but only one is effectively achieved. The order parameter provides a "signature" for the emergent phenomenon. Although these definitions make use of concepts borrowed from physics and information science, they all involve inherently contingent aspects, as the presence of an external observer seems unavoidable. Even a change in entropy supposes that an observer be able to assess the probability of various states. The unavoidable presence of an observer does not preclude, however, the possibility of extending the definition of emergence to include non-human observers or observers that are involved in the emerging phenomenon. In our quest for "strong emergence", we wish to assign the role of the observer to elements of the system itself, as when individuals become aware of phenomena affecting the whole society. This kind of self-observation is only possible because what is observed is a simplified state of the system. Emergence deals precisely with simplification.

2.2 What Does Emerge in Schelling's Model of Spatial Segregation? Schelling's model of spatial segregation (Schelling 1969,1971,1978) is a pioneering example of an emerging phenomenon resulting from social interaction. Schelling's aim was to explain how segregationist residential structures could spontaneously occur, even when people are not so very segregationist themselves. The absence of a global notion of segregationist structures (like the notion of ghettos) in the agent's attributes (preferences) is a crucial feature of this model. Agents do not choose between living or not living in a segregationist structure, but have only local preferences concerning their preferred neighbourhood. Moreover, people have only weak segregationist behaviour, but the play of interactions generates global segregation. In Schelling's original model, agents were placed on a 8-by-8 chessboard as shown in Figure 1 (Java applet).

(l-a) fully integrated population equilibrium (1-b) discontented agents are crossed (1-c) convergence after 4 iterations Source: http://perso.univ-rennesl.fr/denis.phan/complexe/schelling.html and (Phan 2004

Fig. 1. Original (checkerboard) Schelling Model

Emergence in Multi-Agent Systems

151

Taking the "colour" of agents as criterion for discrimination, agents choose a location where to live, depending on their individual tolerance threshold of different colours in their neighbourhood. Agents interact only locally with their 8 direct neighbours (within a so-called "Moore" Neighbourhood). No global representation about the residential structure is available to them. Though agents may be weakly segregationist (each agent would stay in a neighbourhood with up to 62.5% of people with another colour), segregation occurs. Schelling used the following rule: an agent with one or two neighbours will try to move unless one of the two neighbours has the same colour as its own (which means a local tolerance of 50% ); an agent with three to five neighbours requires at least two agents of same colour to stay (that is 33%, 50% and 60% local tolerance), and one with six to eight neighbours will stay if at least three of them are of the same colour (50%, 57,1%), 62,5%) local tolerance). Under Schelling's behavioural assumption, a "fully integrated structure" (Figure 1-a) is an equilibrium (an order) because no agent wishes to move. A "fully integrated structure" is a structural pattern in which agents' colours alternate in all directions. Because of border effects, no agent is located in the comers. The "fully integrated structure" is an unstable equilibrium. A slight perturbation is sufficient to induce a chain reaction and the emergence of local segregationist patterns. In his example, Schelling extracted twenty agents at random, and added five at random in the free spaces. The discontented agents (crossed in Figure 1-b) move at random towards a new location in agreement with their preferences. These moves generate new discontented agents by a chain reaction until a new equilibrium is reached. In such equilibrium, local segregationist patterns appear, like in Figure 1-c. Local interactions are sufficient for spatial homogeneous patterns to occur; spatial segregation is an emerging property of the system's dynamics, while not being an attribute of the individual agents. Sometimes, local integrated (non-homogeneous) patterns may survive in some niches. But such integrated structures are easily perturbed by random changes, while homogeneous structures are more stable (frozen zones). Complementary theoretical developments on Schelling's model of segregation can be found in the growing literature on this subject, for instance among economists like (Zhang 2004a-b), (Panes and Vriend 2003, 2004), or sociologists like (Broch and Mare 2004). Examples of advances in empirical investigations can be found in (Clark 1991), (Sethi and Somanathan 2001), (Koeler and Svoretz 2002), and experimentations in Ruoff, (Schneider 2004). Our aim in this paper is to address emergent phenomena, as instantiated in Schelling's model, in a new way. Emergence is currently debated for its cognitive and sociological aspects, from ontological and epistemic perspectives, in relation with the modem philosophy of mind (for a selection of papers, see for instance (Intellectica 1997), and (Gilbert 1995), for links with sociology. There is also a debate within the artificial intelligence, artificial life and artificial society fields see (Gilbert and Conte 1995) in this later field. Entering or even summarizing those debates would fall outside the scope of the present paper. But some fundamental questions are worth asking about knowing in what way emergence occurs in Shelling's model. Who is the observer? What does the higher level of organisation consist in? For whom does this level make sense? (Figure 2)

152

Jean-Louis Dessalles and Denis Phan Who is observing? Where is this level of oraanisation? ^-

For whom does this

In various definitions of emergence, the presence of an external observer seems unavoidable. Levels of organisation depend on an observer being able to discern subparts in the system and appropriate relations between them. There is no consistent way to say that some new phenomenon occurs at a higher level, be it some new form of operational closure or any form of deviation from expected behaviour, unless there was some pre-existing way to observe that higher level.

3 Emergence in ACE : from Case Studies to a Formal Definition The first and second subsections provide two definitions coherent both with the design of multi-agent systems used in Agent Based Computational economics (Tesfatsion 2002; Phan 2004) and with important related features, like cognitive hierarchy, detection, and complexity. Thefirstone from (Bonabeau and Dessalles 1997) defines the emergence as an unexpected complexity drop in the description of the system by a certain type of observer. The second one from (MuUer 2002) defines emergence as a phenomenon observed at the interface of description levels. The latter definition introduces a useful distinction between "weak" and "strong" emergence. 3.1 Emergence as a Complexity Drop In (Bonabeau and Dessalles 1997), emergence is defined as an unexpected complexity drop in the description of the system by a certain type of observer. Such a definition is claimed to subsume previous definitions of emergence, both structural (dealing with levels of organisation) and epistemological (dealing with deviation fi*om some model's predictions). In each case, the observer is able to detect a structure, such as the presence of relations holding between parts of the system, or some form of behaviour like a characteristic trajectory. Structural emergence occurs whenever the system turns out to be more structured than anticipated. This augmentation of structure can be characterised by a decrease of complexity.

Here, E stands for the amplitude of the emergence, Cexp is the expected structural complexity and Cobs the structural complexity actually observed. Structural

Emergence in Multi-Agent Systems

153

complexity is defined as the algorithmic complexity relative to a given set of structural descriptors. Algorithmic complexity, as defined by Kolmogorov, Chaitin and Solomonov, is defined by the shortest description that can be given of the system using a Turing Machine (Li and Vitanyi 1993). This definition is sometimes considered of little use for finite systems, as the set of all systems of same size can be ordered; since each of these system can be characterised by its rank, nothing prevents the actual system to appear as the simplest one if it happens to be number one. In order to use algorithmic complexity to describe finite system, we abandon the generality of Turing machines, considering that the description tools are imposed by the observer. We define the relative algorithmic complexity (RAC) of a system as the complexity of the shortest description that a given observer can give of the system, relative to the description tools available to that observer. Emergence occurs when RAC abruptly drops down by a significant amount. For our purpose here, we must restrict the definition. We consider a specific class of observers, in order to get closer to what human observers would consider as emergence. Following (Leyton 2001), we impose the observer's description tools to be structured as mathematical groups. In other words, any level of organisation that can be observed has operational closure and is structured as a group, and the only structures that can be observed are the invariant of a group of operations. Moreover, the observer is supposed to have hierarchical detection capabilities. This means that all elements of the system that the observer can consider have themselves a group structure. The observer may be considered as being a 'Leyton machine', for which any structure is obtained through a group-transfer of other structures (Leyton 2001). Let us illustrate how emergence results from a complexity drop in Schelling's model. In a first stage, the external observer reconstructs the system by transferring (in the Leyton sense) one abstract inhabitant to form the entire population. The transfer group, in this case, is the group of 2-D translations. The operation is costly in terms of complexity, as each individual translation has to be instantiated. Then each abstract inhabitant is assigned a colour. This latter operation can be achieved through a transfer by the binary group Z/2Z. In a second stage, the external observer is now able to detect homogeneous clusters. She reconstructs the system in a different way. One first abstract cluster is obtained by translating one abstract inhabitant, as previously. Then this first cluster is itself translated to give the whole set of clusters. Finally, clusters are assigned colours through the binary group. Emergence, in this example, comes from the fact that the second construct is significantly simpler than the first one. The reason is that there are less colour assignments: only one per cluster instead as one per inhabitant. A crucial requirement for the emergence to be noticeable is that the shape of clusters be simple. For the system to be fiilly instantiated, the second construct must reshape the limits of each cluster through various groups of geometrical transformations. If there were no colours, or if the clusters had random shapes, there would be no gain in complexity. Conversely, emergence would be maximum in the extreme case in which all clusters had identical shapes, e.g. if they were square blocks.

154

Jean-Louis Dessalles and Denis Phan

Each transfer group can be seen as an organisation level. In Schelling's model, there could be more levels of organisation, for instance if clusters were arranged in a chessboard-like pattern. The Leytonian construct would be different and less complex in this case: the first cluster would be assigned a colour, then it would be duplicated through a binary symmetry group operating in colour space, then the couple would be transferred through the group of integer translations of the plan. For structural emergence to occur, it is important that there be an unexpected complexity decrease. This may happen either because the higher structure detection was delayed, as when you take time to recognise a Dalmatian dog in a pattern of black and white spots. It may also happen when adding a new observable, instead of increasing the overall complexity of the system for the observer, paradoxically deceases it (Bonabeau and Dessalles 1997). This latter case is well illustrated by our extension of Axtell et al.'s experiment (see Phan and Galam and Dessalles 2005). 3.2 Emergence Occurring in a System with Several Levels Following Forrest's definition of emergent calculation (Forrest 1990), (Miiller 2002) defines emergence in SMA as occurring between two organisation levels, distinguishing the process and the observation of that process. The process concerns the evolution of a system formed by entities in interactions. These interactions may generate observable epiphenomena. At the observation level, epiphenomena are interpreted as emerging through specific calculation (i.e. like order parameter). For Miiller, ''weak emergence'' arises when the observer is external to the system, while "strong emergence" arises when the agents involved in the emerging phenomenon are able to perceive it. In this later configuration the identification of epiphenomena by the agents in interaction in the system will involve di feedbackfi-om the observation to the process. There is a coupling between the process level and the observation level by the way of the agents. Emergence is thus immanent in such a system. More specifically, for Miiller, a phenomenon is emergent if: • • •

(A) There is a system composed of agents in interaction with each other and with their environment. The description of this system as a process is formalized in a language D (B) The dynamics of this system produces a structural phenomenon observable in the "traces of execution" (C) The global phenomenon is observed by an external observer (weak emergence) or by the agents themselves (strong emergence) and is described in a language distinct from D.

When compared with Forrest's definition (Forrest 1990), Miiller's definition presupposes the existence of two languages of description, which are distinct according to the level considered. This distinction only materializes the presence of levels already hypothesised by Forrest. On the other hand, it is interesting to note that Miiller distinguishes the system formed by the interacting agents from the process that governs their behaviour. This enables him to choose the position of the level of observation with respect to the agents. Muller's contribution lies then mainly in the distinc-

Emergence in Multi-Agent Systems

155

tion between two categories of emergence according to the position of the level of observation w.r.t. process. In strong emergence, agents are observers themselves, this de facto entails a feedback loop between the micro (agent based) level of observation and the macro level of the process. In weak emergence, the observer is external with the process and there is no necessarily coupling. Miiller illustrates weak emergence by means of the example of foraging ants which move between their nest and a food source. Each ant deposits on its passage some traces of pheromone which attract the other ants, and create an interaction between them (1). These interactions build a stable and observable phenomenon (2). An external observer may interpret this phenomenon as a "path". Moreover, the accumulation phenomenon based on interaction drives the ant colony to find the shortest path between their nest and a source of food. Emergence is weak because the dynamics depends only on the traces of pheromone (1-2) and not on the qualification of these traces as an "shortest or optimal path", which does not exist in the ants' head. The category of strong emergence is important for to model artificial societies (Gilbert 1995). Indeed, the reflexivity meditated by the agents' "consciousness" appears to be a determinant characteristic that distinguishes systems involving human agents from systems made of non conscious or material entities. In Schelling's model, there would be strong emergence if agents, rather than merely sampling neighbouring densities, were able to perceive forming homogeneous clusters in the town and if their perception could affect their decisions. Strong emergence is particularly important in economic modelling, because the behaviour of agents may be recursively influenced by their perception of emerging properties. Emerging phenomena in a population of agents are expected to be richer and more complex when agents have enough cognitive abilities to perceive the emergent patterns. Such feedback loops between emerging collective patterns and their cognitive components clearly occur among agents in human societies. They may obey laws that are still to be understood. Our aim here is to design a minimal setting in which this kind of strong emergence unambiguously takes place.

4 Conclusion To summarize, if there is strong emergence in the sense of Miiller, the system becomes reflexive, through the mediation of the agents. (A) Agents are equipped with the capacity to observe and to identify an epiphenomenon in the process which represents the evolution of the system in which they interact. This capacity of observation and the field of such observation must then be sufficiently broad to encompass the phenomenon as a global one. (B) The agents can describe this epiphenomenon in a "language" other than that which is used to describe the process (C) The identification of an "emergent" epiphenomenon by the agents involves a change of behaviour, therefore a feedback of the level of observation on the process Emergent phenomena are naturally described in a two-level architecture (Figure 3). In such a framework, objects at the two levels only exist because some observer is able to detect them. The detected object at the upper level is composed by objects of

156

Jean-Louis Dessalles and Denis Phan objects

detectors

concepts

7^^7

^^M /^^=^/z;j7 Fig. 3. Parallelism between hierarchies : description, observations and conceptual level the first level. Correspondingly, the upper level detector is triggered by the activity of lower level detectors. The system's complexity, defined as the minimal description that can be given of its state, drops down by a significant amount when an upper-level detector becomes active, as its activity subsumes the activity of several lower-level detectors. According to this point of view, one can reinterpret the Miiller's definition using a distinction due to (Searle 1995) between entities that are independent of the observer (the process and the phenomena which results from it) and entities that occur within the observer (identification and interpretation of an epiphenomenon). According to this interpretation, emergence becomes a category relative to an observer, and in the case of a human observer (or an agent supposed to be represented an human), a subjective category. Note that Miiller's definitions and the above definition of structural emergence as complexity drop are compatible. Miiller's distinction between two description languages presupposes that the upper language, available to the observer, provides it with a simpler description of the epiphenomena than what was available at the process level. In order to illustrate our formal definition of emergence, a companion paper (Phan and Galam and Dessalles 2005) discusses the ACE model of (Axtell and Epstein and Young 2001) and builds an extension based on the formal fi'amework introduced in this paper. In the basic model, agents tend to correlate their partners' behaviour with fortuitous visible but meaningless characteristics (tags). On some occasions, these fortuitous tags turn out to be reliable indicators of dominant and submissive behaviour in an iterative Nash bargaining tournament. One limit of this model is that dominant and submissive classes remain implicit within the system. Classes only emerge in the eye of external observers (weak emergence). We enhance the model to allow for strong emergence. In a two-level framework, agents get an explicit representation of the dominant class whenever that class emerges, thus implementing strong emergence.

References 1. Axtell R. Epstein J.M., Young H.P. (2001) The Emergence of Classes in a Multi-agent Bargaining Model in Durlauf, Young eds., Social dynamics, The MIT Press, p. 191-212 2. Bedau, M.A. (1997) Weak Emergence, Nous, Vol. 31, Supplement: Philosophical Perspectives, 11, Mind, Causation, and World, p. 375-399 3. Bedau, M. A.( (2002) Downward causation and the autonomy of weak emergence, Principia 6-1 June, special issue on Emergence and Downward Causation, p. 5-50

Emergence in Multi-Agent Systems

157

4. Bonabeau E., Dessalles J-L. (1997) Detection and Emergence, Intellectica, 25:89-94. 5. Bonabeau, E., Dessalles, J.-L., Grumbach, A. (1995a) Characterizing emergent phenomena (1): A critical review. Rev. Int. Syst. 9 p.327-346. 6. Bonabeau, E., Dessalles, J.-L., Grumbach, A. (1995b) Characterizing emergent phenomena (2): a conceptual framework. Rev. Int. Syst. 9, p.347-369. 7. Broch, Mare (2003) Neighborhood Choice and Neighborhood Change California Center for Population Research On-Line Working Paper Series CCPR-003-03, June. 8. Bourgine P., Nadal J.P. eds. (2004) Cognitive Economics, Spinger Verlag 9. Campbell DT (1974), Downward causation, in hierarchically organized biological systems. In: Studies in the Philosophy of Biology: Reduction and Related Problems, Ayala, Dobzhansky, eds. Macmillan, London 10. Cariani, P. (1991) Adaptivity and emergence in organisms and devices, World Futures 31, p. 49-70, Gordon & Breach Science Publishers 11. Clark W.A.V. (1991) Residential Preferences and Neighborhood Racial Segregation: A Test of the Schelling Segregation Model, Demography 28, p. 1-19. 12. Davidson, D. (1970) Mental Events, in Lawrence Foster and J. W. Swanson, eds.. Experience and Theory, Duckworth, London. 13. Davidson D. (1980) Actions and Events, Clarendon, Oxford 14. Durlauf S.N. (1997) Statistical Mechanics Approaches to Socioeconomic Behavior, in Arthur, Durlauf, Lane (eds). The Economy as an Evolving Complex System II, Santa Fe Institute Studies in the Sciences of Complexity, Volume XXVII, Addison-Wesley. 15. Durlauf.S.N. (2001) A Framework for the Study of Individual Behavior and Social Interactions, Sociological Methodology Volume 31 Issue 1, 1 January p. 47-87 16. Emmeche, C, Koppe, S. & Stjemfelt, F. 1997. Explaining emergence-towards an ontology of levels. Journal for General Philosophy of Science, 28: 83-119. 17. Epstein J.M. (1999) Agent-Based Computational Models and Generative Social Science, Complexity, Vol.4-5, p.41-60. 18. Ferber, J., Labbani O., Miiller J.-R, Bourjault. A. (1997) Formalizing emergent collective behaviours: Preliminary report. International Workshop on Decentralized Intelligent and Multi-Agent Systems, St. Petersburg, p. 113-122. 19. Forrest S. (1990) Emergent Computation : Self-organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks, in Emergent computation, MIT Press, Cambridge, M.A., 1990, p. 1-11. 20. Galam S., (2004), Spontaneous Symmetry Breaking, in Bourgine, Nadal. eds. op. cit. p. 157-168. 21. Galam S., Gefen Y. , Shapir Y. (1982) Sociophysics: a new approach of sociological collective behavior. Journal of Mathematical Sociology 9,1 22. Gilbert N. (1995) Emergence in social simulations, in Gilbert, Conte. eds. 114-156. 23. Gilbert N., Conte R. eds. (1995) Artificial Societies : The Computer Simulation of Social Life UCL Press London. 24. Gillett C. (2002a) The Varieties of Emergence: Their Purposes, Obligations and Importance, Grazer Philosophische Studien, vol.65, p.89-115. 25. Gillett C. (2002b) Strong Emergence as a Defense of Non-Reductive Physicalism: A Physicalist Metaphysics for 'Downward' Determination, Principia 6-1 June, special issue on Emergence and Downward Causation,, p.83-114. 26. Granovetter M.S. (1985) Economic Action and Social Structure: The problem of embeddedness, American Journal of Sociology, 91 p. 481-510. 27. Kampis, G. (1991) Self-Modifying Systems in Biology and Cognitive Science Pergamon 28. Kim, J. (1992) Downward causation, in emergentism and nonreductive materialism, in Beckermann, Flohr, Kim, eds.. Emergence or Reduction? Walter de Gruyter.

158

Jean-Louis Dessalles and Denis Phan

29. Kim, J. (1993) Supervenience and Mind, Cambridge: Cambridge University Press. 30. Kim, J. (1995) Emergent properties, in T. Honderich, ed. The Oxford Companion to Philosophy, Oxford University Press p. 224. 31. Kim, J. (1999) Making sense of Emergence, Philosophical Studies, 95 p.3-36. 32. Koehler G. Skvoretz J. (2002) Preferences and Residential Segregation: A Case Study of University Housing WP 33. Langton C. (1989) Computations at the edge of chaos, Physica D, 42 34. Li M. Vitanyi P.M.B. (1993) An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York, Berlin 35. Lewes, G, H. (1874) in Emergence, Dictionnaire de la langue philosophique, Foulquie. 36. Leyton, M. (2001) A generative theory of shape. New York: Springer Verlag 37. McLaughlin B. P. (1992) The rise and fall of British emergentism, in Beckermann, Flohr, Kim, eds.. Emergence or Reduction? Berlin: Walter de Gruyter. 38. McLaughlin B. P. (1997) Emergence and supervenience, Intellectica 25 p.25-43. 39. Mill, J.S. (1843) System of Logic, Longmans, Green, Reader, and Dyer, London. 40. MuUer J.P. (2002) Des systemes autonomes aux systemes multi-agents : interactions, emergence et systemes complexes Rapport de I'HDR, 8 novembre, 58p. 41. Panes, R., Vriend, N.J. (2003) Schelling's Spatial Proximity Model of Segregation Revisited, Dept. of Economics Working Paper No. 487 Queen Mary, University of London 42. Phan D. (2004) From Agent-Based Computational Economics towards Cognitive Economics, in Bourgine, Nadal, eds., op.cit. p. 371-398. 43. Phan D., Galam S., Dessalles J.L. (2005) Emergence in multi-agent systems: cognitive hierarchy, detection, and complexity reduction part II: Axtell, Epstein and Young's model of emergence revisited, CEF 2005,11th International Conference on Computing in Economics and Finance, June 23-25, 2005, Washington, DC 44. Rosen, R. (1978) Fundamentals of measurement and representation of natural systems. North Holland. 45. RuofiF G. Schneider G (2004) Segregation in the Class Room: An Empirical Test of the Schelling Model, meeting of the working group on decision theory of the German Political Science Association, University of Bamberg, July 9-10 46. Schelling T.S. (1969) Models of Segregation, American Economic Review, Papers and Proceedings, 59-2, p.488-493. 47. Schelling T. S.(1971) Djniamic Models of Segregation, Journal of Mathematical Sociology; l,p.l43-186. 48. Schelling T.S. (1978) Micromotives and Macrobehavior WW. Norton and Co, N.Y. 49. Searle J.R. (1995) The Construction of Social Reality, Free Press 50. Sethi R. Somanathan R (2004), Inequahty and Segregation, Journal of PoliticalEconomy p.112-6 51. Sperry, R. W.(1969) A modified concept of consciousness, Psychol. Rev. 76, 532-536 52. Sperry, R. W. (1986) Discussion: Macro- versus micro-determinism (A response to Klee), Philosophy of Science, 53(2), p.265-270. 53. Sperry, R. W. (1991) In defense of mentalism and emergent interaction. Journal of Mind and Behavior 12, p. 221-245 54. Tesfatsion L. (2002) Economic Agents and Markets as Emergent Phenomena, Proceedings of the National Academy of Sciences U.S.A., Vol. 99-S3, p.7191-7192 55. Tesfatsion L. (2002b) Agent-Based Computational Economics: Growing Economies from the Bottom Up, Artificial Life, Volume 8 - 1 , MIT Press, p.55-82. 56. Van de Vijver G. (1997) Emergence et explication, Intellectica 25 p.7-23. 57. Varela F., Thompson E., Rosch E. (1991) The embodied mind, MIT Press.

Emergence in Multi-Agent Systems

159

58. Zhang J. (2004a) Residential Segregation in an AU-Integrationist World, Journal of Economic Behaviour & Organization, 54, p.533,550 59. Zhang J. (2004b) A Dynamic Model of Residential Segregation, Journal of Mathematical Sociology, 28, p.147-170

Methodological Issues

The Implications of Case-Based Reasoning in Strategic Contexts Luis R. Izquierdo and Nicholas M. Gotts The Macaulay Institute, Craigiebuckler, AB15 8QH, Aberdeen, UK

Summary. This paper characterises the transient dynamics and the long-term behaviour of a game theoretical model where players' decisions at any particular time are guided by a single similar situation they experienced in the past - a simple form of case-based reasoning. The transient dynamics of the model are very dependent on the process by which players learn how to play the game in any given situation. The long-run behaviour of the model varies significantly depending on whether players can occasionally explore different actions or not. When the probability of experimentation is small but non-zero, only a subset of the outcomes that are possible in the absence of experimentation persists in the long-run. In this paper we present some features that characterise such a subset of stochastically stable outcomes.

1

Introduction

This paper deals with the formal study of social interactions which can be meaningfully modelled as decision problems of strategy and, as such, using game theory as a framework. Game theory is a branch of mathematics devoted to the formal analysis of decision making in social interactions where the outcome depends on the decisions made by potentially several individuals. It is a useful framework to accurately and formally describe interdependent decision-making processes, and it also provides a collection of solution concepts that narrow the set of expected outcomes in such processes. The most widespread solution concept in game theory is the Nash equilibrium, which is a set of strategies, one for each player, such that no player, knowing the strategy of the other(s), could improve her expected payoff by unilaterally changing her own strategy. Though extremely useful, game theory is at present somewhat limited in the sense that it is dominated by assumptions of full rationality, it generally ignores the dynamics of social processes, and it often requires disturbing and unrealistic hypotheses about players' assumptions on other players' cognitive capabilities and beliefs in order to derive specific predictions. Furthermore it is often the case that even with heroic assumptions about the computational power and beliefs that every player attributes to every other player, game theory cannot reduce the set of

164

Luis R. Izquierdo and Nicholas M. Gotts

expected outcomes significantly (e.g. consider the Folk Theorem in the repeated Prisoner's Dilemma). While acknowledging that the work conducted in game theory up until now has been extremely useful, a growing inter-disciplinary community of scientists is starting to believe that the time has come to develop game theory past the boundaries of full rationality, common-knowledge of rationality^ consistently aligned beliefs (Hargreaves Heap and Varoufakis 1995), static equilibria, and long-term convergence. These concerns have led many scientists to develop models of social interactions within the framework of game theory but (a) assuming players behave in ways that are cognitively more plausible than fully rational behaviour and common knowledge of rationality (e.g. assuming players learn from experience), and (b) paying special attention to the dynamics of such models and not only to their long-term properties. These investigations are being undertaken experimentally and formally (both analytically and using computer simulation), and special emphasis is being paid to the study of backward-looking learning algorithms, which seem to be more plausible than the forward-looking methods of reasoning employed in orthodox game theory. The latter appear to be very demanding for human agents (let alone other animals) and remain undefined in the absence of strong assumptions about other players' behaviour and beliefs. Some of the decision-making algorithms that have attracted the attention of researchers in game theory are: reinforcement learning (with experimental studies conducted by e.g. (Erev et al. 1999), theoretical work done by e.g. (Bendor et al. 2001), and studies of the dynamics carried out by e.g. (Macy and Flache 2002)), belief learning (with theoretical work on fictitious play developed by e.g. (Fudenberg and Levine 1998)), the EWA (Experience Weighted Attraction) model (Camerer 2003), which is a hybrid of the reinforcement and belief learning, and finally, case-based reasoning (Izquierdo et al. 2004). This paper advances the work conducted by (Izquierdo et al. 2004) on the implications of case-based reasoning in strategic contexts. In particular, we study the transient and the long-run dynamics of a game theoretical model where players' decisions at any particular time are guided by a single similar situation they experienced in the past - a simple form of case-based reasoning. It is assumed in this paper that players suffer from trembling hands, i.e. they occasionally explore different actions with small probability. These trembles make the model slightly more realistic and reduce the set of expected outcomes in the long-term. Some outcomes that can be observed infinitely often in the model without trembles are not stable in the model with trembles no matter how unlikely trembles are as long as they are possible. This is so because the properties of the process with trembles when the probability of trembles tends to zero differ from those of the process where the probability of trembles is exactly zero. In this paper we present some features that characterise the set of outcomes which are stable in the presence of small trembles. ^ Common knowledge of rationality means that every player assumes: (a) that all players are instrumentally rational, and (b) that all players are aware of other players' rationalityrelated assumptions (this produces an infinite recursion of shared assumptions).

The Implications of Case-Based Reasoning in Strategic Contexts

2

165

Case-Based Reasoning

Case-Based Reasoning (CBR) arose out of cognitive science research in the late 1970s (Schank and Abelson 1977; Schank 1982); since then, several psychological studies have provided support for its importance as problem-solving process in human reasoning, especially for novel or difficult tasks (see (Ross 1989) for a summary). Case-based reasoning is a form of reasoning by analogy within a particular domain. It consists of "solving a problem by remembering a previous similar situation and by reusing information and knowledge of that situation" (Aamodt and Plaza 1994). Case-based reasoners do not employ abstract rules as the basis to make their decisions, but instead they use similar experiences they have had in the past. Such experiences are stored in the form of cases. A case is "a contextualised piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goals of the reasoner" (Kolodner 1993, p. 13). Thus, when a casebased reasoner has to solve a problem, she is reminded of a similar situation that she encountered in the past, of what she did then, and of the outcome that resulted in the recalled situation. She then uses that 'similar past case' as a basis to solve the problem in the present. Case-based reasoning generally consists of four main tasks (Aamodt and Plaza 1994): 1. Retrieve the most similar case or cases. Generally a case in CBR is rich in information and quite complex. (Aamodt and Plaza 1994) say: "a feature vector holding some values and a corresponding class is not what we would call a typical case description" (because it is too trivial). Thus, performing similarity judgements is an integral part of CBR. 2. Reuse the information and knowledge in the retrieved case to solve the current problem. The retrieved knowledge cannot always be directly applied, so some adaptation is sometimes required, 3. Revise the proposed solution. This involves the evaluation of the proposed solution. 4. Retain the relevant information for the future - i.e. learn. Case-based reasoning is often used as a problem-solving technique in domains where the distinction between success and failure is either fairly trivial or is made externally. However, in decision-making contexts in general, the distinction between what is satisfactory and what is not can be far from trivial, and thus, the question of whether a particular decision used in the past should be repeated, or a new decision should be explored is crucial. This dilemma naturally gives rise to Simon's notions of satisficing, as noted by (Gilboa and Schmeidler 2001). (Gilboa and Schmeidler 2001) have developed Case-Based Decision Theory (CBDT), a formal theory of decision based on past experiences which was initially inspired by case-based reasoning. Having said that, as noted by the authors, CBDT has not much in common with CBR beyond Hume's basic argument that "from causes which appear similar we expect similar effects". The main difference between CBR and CBDT is that while a defining feature of CBR is that "thought and action in a given situation are guided by a single distinctive prior case" (Loui

166

Luis R. Izquierdo and Nicholas M. Gotts

1999), in CBDT decision-makers rank available acts according to the similarityweighted sum of utilities that resulted in all available cases. Anyhow, like (Gilboa and Schmeidler 2001), we do not see reasoning by cases as a substitute for expected utility theory, but as a complement: reasoning by cases seems more plausible than expected utility theory when dealing with novel decision problems, or in situations where probabilities cannot easily be assigned to different states of the world (uncertainty, as opposed to risk), or if such states of the world cannot be easily constructed (ignorance). To our knowledge, the implications of CBR in strategic contexts were explored for the first time by (Izquierdo et al. 2004). (Izquierdo et al. 2004) developed an agent-based model, CASD^, where individuals use a very simple form of CBR. The main focus of the model was to investigate the implications of reasoning by a single distinctive past experience in strategic environments; for that reason, issues like knowledge representation and similarity assessments, which are central in CBR, are largely ignored in CASD. (Izquierdo et al. 2004) investigated the ability of case-based reasoners to cooperate in social dilemmas. Social dilemmas are especially challenging for orthodox game theory because the predictions of the theory in such contexts are often counterintuitive and have been rejected almost invariably by empirical evidence (see, for instance, work reviewed by (Colman 1995) in chapters 7 and 9). They also offer a promising arena to distinguish the differences between reasoning by cases (or outcomes^) and reasoning by rules (or strategies). The following explains why. Although defining rational strategies in interdependent decision-making problems is by no means trivial, it seems sensible to assume that a) rational players choose dominant strategies, and b) rational players do not choose dominated strategies. Similarly, even though defining rational outcomes cannot be done without controversy, it also seems sensible to agree that rational outcomes must be Pareto optimal. Assuming only those necessary conditions for the rationality of strategies and outcomes, we can state that in the one-shot Prisoner's Dilemma (PD) and other social dilemmas, even though there is a clear causal link between strategies and outcomes, rational strategies lead to outcomes which are not rational, whereas rational outcomes are generated by strategies which are not rational. Using their model as a "tool to think with", (Izquierdo et al. 2004) developed the concept of iterative elimination of dominated outcomes, which describes a logical process through which case-based reasoners can arrive at sensible {i.e. Pareto optimal) outcomes in games. Dominated outcomes are outcomes which are not individually rational - i.e. there is at least one player who is obtaining a payoff below her Maximin"^. The idea behind the process of iterative elimination of dominated outcomes is that players cannot rationally accept outcomes where they are not obtaining at least their Maximin (rational players are not exploitable). When ^ CASD is available online under GNU General Public Licence, together with a user guide, at http ://www .macaulay. ac.uk/fearlus/casd/. ^ An outcome is a particular combination of decisions, each of them made by one player. ^ The largest possible payoff a player can guarantee themselves irrespective of the other players' actions.

The Implications of Case-Based Reasoning in Strategic Contexts

167

players who do not accept outcomes where they get a payoff lower than Maximin meet, they might learn by playing the game the fact that their opponent is not exploitable either. If this occurs, it will be mutual belief that dominated outcomes cannot be sustained because at least one of the players will not accept them. That inference (and the consequent disregard of dominated outcomes by every player) can make an outcome which was not previously dominated in effect be dominated. In other words, the concept of dominance can be applied to outcomes iteratively just as it is applied iteratively to strategies. While useful as a "tool to think with", the specific model (Izquierdo et al. 2004) used was unrealistic in the sense that their simulations would necessarily end up with the agents locked in to a persistent cycle. In this paper we advance their work by developing their model further. In particular, in our model, players suffer from trembling hands (Selten 1975) - i.e. they occasionally experiment (or make mistakes) with small probability. This new functionality makes the model more realistic and allows us to make more specific predictions. In particular, we will characterise the set of outcomes where the system spends a significant proportion of time in the long-term when players experiment with very low probability. Such a set of outcomes is a subset of the set of outcomes that can be observed in the model without experimentation. As an example, we will see that in the prisoner's dilemma, mutual cooperation belongs to the latter set but not to the former.

3

The Model

The model we study here is a generalisation of CASD. In our model individuals play repeatedly a game, once per time-step. Every time they play, each player / retains a case (an experience), which comprises: - The time-step t when the case occurred. - The perceived state of the world at the beginning of time-step t, which is determined by all the decisions undertaken by each player in the game (including the case-holder) in the preceding m, (for memory) time-steps. Thus, if there are n players in the game, each of whom can select one among a set of a possible actions, an agent with memory m will be able to identify d"'"" different states of the world. Thus, player Ts perceived state of the world at the beginning of time-step / consists of the m, preceding decisions made by every player. - The decision made by the case-holder in that situation, in time-step t, having observed the state of the world in that same time-step. - The payoff that the case-holder obtained after having decided in time-step t. The number of cases that players can keep in memory is unlimited. At the time of making a decision, players decide what action to select by retrieving the most recent case which occurred in a similar situation for each one of the actions available to them. This set of cases, which is potentially empty, is denoted Q. A case is perceived by the player to have occurred in a similar situation if and only if its state of the world is a perfect match with the current state of the world observed

168

Luis R. Izquierdo and Nicholas M. Gotts

by the case-holder. Thus, the only function of the perceived state of the world is to determine whether two situations look similar to the player or not. In a certain situation {i.e. for a given perceived state of the world) any particular player / will face one of the following three possibilities: - If the set C, is empty, player i selects one action at random. - If at least one action of those available to player / is represented in C, but not all of them are, then player / selects randomly among those actions with the highest payoff obtained in the set C,. If the selected action provided a payoff at least equal to player f s Aspiration Threshold (AT), then the action is considered satisfactory and will be undertaken. Otherwise, one of the actions that are not represented in Cj will be selected at random. - If every action available to player i is represented in Q, then player / selects randomly among those actions with the highest payoff obtained in the set Cj. Thus, players in this model satisfice in the sense that, when making decisions in situations that look familiar to them, they explore new actions if and only if the best decision taken in similar past situations did not meet their aspirations (and there are new actions to try). As mentioned before, we also assume that players suffer from trembling hands: there is some small probability sdj ^ 0 that player i selects her action randomly instead of following the algorithm above. The ratio A/ A, determines player /'s relative tendency to experiment compared with player y's. The factor e is a general measure of the frequency of experimentation in the whole population of players. The event that / experiments is assumed to be independent of the event thaty experiments for every i ^J. Different players may experiment in different ways, but it is assumed that player f s probability of selecting any action a available to her when experimenting (qi(a)) is non-zero, potentially different for different actions, and independent of time for all i; these conditions can be relaxed to some extent. This completes the specifications of the model where players suffer from trembling hands, which will be referred to as the perturbed model. This paper will present some mathematical results valid when the overall probability of experimentation s tends to zero; all such results are independent of 1/ and of the particular way each of the players experiments. When presenting simulation results, it will be assumed that A, = 1 for all z, and that players select one of their actions randomly and without any bias when experimenting.

4

Results and Discussion

In the unperturbed model (s = 0), players lock in to cycles, so they cannot experience all the different situations they would regard as different: they indefinitely go through cycles made up of a (usually small) subset of all the situations they can distinguish (Izquierdo et al. 2004). However, in the perturbed model every situation that players can distinctively recognise can occur with non-zero probability, so eventually every such situation happens, and it happens infinitely often.

The Implications of Case-Based Reasoning in Strategic Contexts

169

More formally, note that both the perturbed and the unperturbed model can be formulated as fmite-state discrete-time Markov chains, but there is a crucial difference between them: the unperturbed model will end up in one of many possible cycles (the period of some of these cycles is potentially equal to one), whereas the perturbed process has one unique limiting distribution. Thus, when players suffer from trembling hands, the indefinite cycles where players were locked in the unperturbed model are broken, and outcomes that occurred infinitely often in the unperturbed process (like mutual cooperation in the prisoner's dilemma (Izquierdo et al. 2004)) turn out not to be robust to small trembles. In the following two sections we study the transient and the long-run behaviour of the perturbed process. 4.1

Transient Dynamics

As one would expect, the short-term dynamics of the perturbed process - i.e. when only a few trembles have taken place - are initially similar to the dynamics of the unperturbed process. How many 'a few trembles' are depends on the players' memory and aspiration thresholds; how quickly those 'few trembles' occur depends on the probability of trembles happening. Fig, 1 shows the proportion of outcomes where both players are cooperating (cooperation rate) in the Prisoner's Dilemma (PD) for different values of both players' memory m and aspiration threshold ^ r , and for different values of the overall probability of trembles e. The cooperation rates shown in Fig. 1 are calculated over time-steps 1001 to 1100. A word of caution about Fig. 1 is that, because it shows the data collected at a predetermined range of time-steps (1001-1100), it represents the short-term behaviour of those series for which 1000 time-steps are not enough to approach their long-term behaviour (e.g. mi= 5) but, on the other hand, it represents the long-run behaviour for some other series (e.g. those series for which 1000 time-steps are enough to reach it, like series with m, = 0, and e i^ 0.001). If enough number of trembles have taken place in every situation distinctively perceived by any player, then the dynamics of the perturbed model will resemble its long-run behaviour, which is independent of the players' memory and of their aspiration thresholds. Aspiration thresholds are irrelevant in the long-term because sooner or later all players will conduct every possible action in every possible situation, so their selection algorithm will not take the value of their aspiration threshold into account. That the actual values m, are also irrelevant in the long-term can be explained as follows: divide the set of possible situations in the game into disjoint classes such that all the situations in a certain class look similar to every agent. The decision processes occurring within each of those classes define a finite-state irreducible aperiodic discrete-time Markov chain. This Markov chain is exactly the same in every class because the decision processes are exactly the same in any one situation. Because eventually every class will be infinitely many times revisited, every class will reach its asymptotic behaviour, which is the same for every class. The values of w, affect the number of classes, but not the decision processes within them so, in the long-run, the values of ^^7, are irrelevant.

170

Luis R. Izquierdo and Nicholas M. Gotts

PD: Average cooperation rate calculated overtime-steps 1001-1100

0.7

Trembles probability € -0 •••• 0.001 \---£t- -0.01 X- 0.1

l\1emory=0 IVIemory=1 Memory=2 Memory=3 l\/lemory=4 Memory=5

Fig. 1. Average proportion of outcomes where both players are cooperating in the Prisoner's Dilemma (PD), calculated over 100 time-steps starting at time-step 1000, and using 500 simulation runs for each data point. The payoffs in the game are represented by its initial letter: S for Suckers, P for Punishment, R for Reward, and T for Temptation Thus, for example, the long-run cooperation rate in the PD (calculated analytically) is 4.98510-^ for e = 0.1, 4.97810-^ for e = 0.01, and 4.998-10-^ for e = 0.001. As we can see in Fig. 1, the series with low memory (w/= 0 or m/= 1) and high probability of trembles (e = 0.1 or s = 0.01) quickly converge to their limiting values; for those parameterisations 1000 time-steps are sufficient to closely approach the long-run behaviour of the process. If we represented the data in Fig. 1 after a sufficiently high number of time-steps, the value of every data point with ei^O would only depend on the probability of trembles s (and on X, and qi(') generally), and it would approach the analytically calculated values presented above (calculated for A, = 1, and ^,() unbiased). Something which is clear in Fig. 1 is that whereas mutual cooperation usually forms part of the cycles in the unperturbed process, it cannot be sustained in the long-term when small trembles occur. Hence the short-term behaviour of the perturbed model is a transition from a distribution similar to that corresponding to the unperturbed model to a very different distribution which is only dependent on the probabilities with which trembles occur. Thus, at the beginning of a simulation, as shown by (Izquierdo et al. 2004) in the unperturbed model, the behaviour of the system does not only depend on what is learnt by each player in any given situation, but also, and very strongly, on how it is learnt, and aspiration thresholds play a major role on that learning process. Players' decisions lead them to situations which require new decisions.

The Implications of Case-Based Reasoning in Strategic Contexts

171

which in turn lead players to new situations. Decisions and situations interweave in complex and highly path-dependent ways that are governed by the process by which players arrive at their final decisions for each situation. As time goes by, more and more trembles occur in each situation distinctively perceived by the players, and consequently the behaviour of the system starts to change. Eventually, players' aspiration thresholds and the values of w, become irrelevant, so the initial mighty impacts of these parameters are lost forever in the mist of the past.

4.2

Long-Run Behaviour

Having seen that the asymptotic behaviour of the model is only dependent on the structure of trembles, a natural question is: What outcomes can be observed with probability bounded away from zero in the long-run as the probability of trembles e tends to zero? Following (Young 1993), such outcomes will be called stochastically stable. It turns out that whether an outcome is stochastically stable or not is independent of A/ and of ^/(O (Young 1993). (Young 1993) provides a general method to identify stochastically stable states in a wide range of models by solving a series of shortest path problems in a graph. In or model there are more states than outcomes, but identifying stochastically stable outcomes when the set of stochastically stable states is known is straightforward. Young's method uncovers an important feature of stochastic stability: stochastic stability selects states which are easiest to flow into from all possible states of the system. This contrasts with most notions of equilibrium based on full rationality. As (Young 1993) notes, risk dominance "selects the equilibrium that is easiest to flow from every other equilibrium considered in isolation". Similarly, Nash stability is determined only by unilateral deviations from the equilibrium. In this section we present some features to identify stochastically stable outcomes when reasoning is based on singletons of distinct prior outcomes. We start with a necessary condition for outcomes to be stochastically stable. Proposition 1. Every stochastically stable outcome is individually rational. Proof. Bearing in mind that players' memory and aspiration thresholds are not relevant in the long-term, let us focus on a perturbed model (e ^ 0, 2, ^ 0, and qi{') > 0) where every agent has memory m, = 0 and any arbitrary aspiration threshold. Since sooner or later all players will conduct every possible action, in studying the long-term behaviour of the system there is no point in considering states of the system where some of the actions have not been selected yet. Therefore let us define a state of the system by the payoff obtained by each player the last time they conducted each of the actions available to them. (This highlights why aspiration thresholds are irrelevant in the long-term). The set of possible states may be smaller than every possible combination of payoffs, since in some games some combinations cannot occur in the course of a simulation. The model thus defined is a finitestate irreducible aperiodic discrete-time Markov chain, which is denoted P^. Let P^ be the Markov process P^ when e = 0 (which is generally reducible). The proof rests on two arguments. The first argument, which is an immediate application of theorem 4 in (Young 1993), is that every stochastically stable state is a recurrent state oiP^. The second argument is that the outcome succeeding any recurrent state of P^ is

172

Luis R. Izquierdo and Nicholas M. Gotts

necessarily individually rational. The following proves an alternative (but equivalent) formulation of the second argument: if an outcome which is not individually rational can succeed state s in P^, then 5 is a transient state of P^. We will prove this second argument by showing that if an outcome which is not individually rational succeeds s, then s will never be revisited. Let / be the state succeeding s when an outcome which is not individually rational has occurred. Let A be one of the players who has received a payoff below her Maximin after s. Let a be the action that A chose, and/?,(y4, a) the payoff she obtained, which is part of state t. Since A selected action a in state s, the payoff/>^(^, a) that A attaches to action a in state s is at least equal to her Maximin. Thus,/>X^, a) > Pt(A, a). SincQ pt(A, a) is below ^'s Maximin, A will never select action a ever again, so the payoff p.(A, a) that A will attach to action a in any subsequent state will remain unmodified. Therefore, state s, in which A attaches PsiA, a) to action a will never be revisited again. This completes the proof of the second argument, and hence the proof of the proposition.

1^ 3 1^ 3

1^ 2 1^

21 1 1

1 1 I4

2

A

2

4|

1 ^ I4

1

2

1 1

1^ 1

^ I4 1 1 13 ^ 1 1 1 3|

2

1 1 1 ^ 2 1 I4

1 1 I2 B

3|

3

C

D

21

4|

1

2

2 E

1

1

3

1 I2

3

1

F

Fig. 2. Stochastically stable outcomes (highlighted in white) in various 2-player 2-strategy games. Payoffs are numeric for the sake of clarity, but only their relative order for each player is relevant Proposition 1 is a useful condition to identify outcomes which cannot be stochastically stable but, except in very simple games (e.g. see Fig. 2A), it is not sufficient to characterise the set of stochastically stable outcomes. To try to identify features that make outcomes stochastically stable we developed a computer program^ that calculates the exact long-run probability that any 2-player game spends in each possible outcome when the probability of trembles tends to zero. Using this program, we came to the following conclusions: - Stochastically stable outcomes are not necessarily Nash equilibria (e.g. see the game of Chicken in Fig. 2B). - In fact, some players in some stochastically stable outcomes may be choosing strictly dominated strategies (e.g. see the game represented in Fig. 2C). - Nash equilibria are not necessarily stochastically stable (e.g. see the game of Stag Hunt in Fig. 2D). - Stochastically stable outcomes can be Pareto dominated by outcomes which are not stochastically stable (e.g. see the Prisoner's Dilemma game in Fig. 2E). However, it can be proved that stochastically stable outcomes cannot be Pareto dominated by outcomes which are one tremble away and which are not stochastically stable. Thus, in the game represented in Fig. 2C, for example, if we knew that outcome (3,3) is stochastically stable, then we could infer that (4,4) would have to be stochastically stable too. ' This computer program is available online at http://www.macaulay.ac.uk/fearlus/casd/.

The Implications of Case-Based Reasoning in Strategic Contexts

173

- Stochastically stable outcomes can Pareto dominate outcomes which are not stochastically stable (e.g. see game represented in Fig. 2A). Intuitively, note that trembles can destabilise outcomes in two different ways: by giving the deviator a higher (or equal) payoff, or by giving any of the nondeviators a lower payoff^. The first possibility is related to the concept of Nash equilibrium, whilst the second is related to the concept of "protection" (Bendor et al. 2001). An outcome is protected if unilateral deviations by any player do not hurt any of the other players. (Bendor et al. 2001) show that under a very wide range of conditions, reinforcement learning converges to individually rational outcomes which are either Pareto optimal or a protected Nash equilibrium. The same is not true for the model we study in this paper (see the game represented in Fig. 2F), but protected strict Nash equilibria are very relevant here too: if there is a protected strict Nash equilibrium in a game, then there is at least one state which is robust to any one single tremble, and the outcome that follows such state in the absence of trembles is the protected strict Nash equilibrium. In fact, it can be shown that the only stochastically stable outcome in any 2-player 2-strategy game with a (necessarily unique) protected strict Nash equilibrium is such equilibrium. The extension of this result to more general games is left for future work.

5

Conclusions

This paper has explored the implications in strategic contexts of reasoning by single and distinctive past experiences as opposed to reasoning by abstract rules (strategies). While the short-term dynamics of models where players base their decisions on past experiences are very dependent on the specifics of such models, a very wide range of models behave similarly in the long-term. In particular, a large collection of models where players experiment from time to time share the same set of stochastically stable outcomes (outcomes that persist in the long-run when trembles are very rare). Stochastically stable outcomes are necessarily individually rational, but a clear relationship between them and Nash equilibria, or Pareto optimality, has not been found. Nash equilibria may, or may not, be stochastically stable, and stochastically stable outcomes may, or may not, be Nash equilibria. The same applies for Pareto optimal outcomes. A concept that is indeed closely related to stochastic stability is the concept of protected strict Nash equilibrium. In particular, in 2-player 2strategy games with a protected strict Nash equilibrium (which is necessarily unique), the only stochastically stable is such an equilibrium. Future work will be devoted to investigate whether this relation holds in more general games.

^ Non-deviators could get a lower payoff after a tremble and still keep choosing the same action if the payoff obtained when the tremble occurs is higher than any of the payoffs that the non-deviator obtained when she last selected each of the other possible actions.

174

Luis R. Izquierdo and Nicholas M. Gotts

Acknowledgements We would like to gratefully acknowledge financial support from the Scottish Executive Environment and Rural Affairs Department. We are also indebted to Segis Izquierdo and Gary Polhill for their valuable comments on this paper.

References Aamodt A, Plaza E (1994) Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Communications lOS Press 7(l):39-59 Bendor J, Mookherjee D, Ray D (2001) Aspiration-based reinforcement learning in repeated interaction games: an overview. International Game Theory Review 3(23):159-174 Camerer C F (2003) Behavioral game theory: experiments in strategic interaction. Princeton University Press, New York Colman A M (1995) Game theory and its applications in the social and biological sciences. 2nd edition. Butterworth-Heinemann, Oxford, UK Erev I, Bereby-Meyer Y, Roth A E (1999) The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models. Journal of Economic Behavior & Organization 39(1):111-128 Fudenberg D, Levine D (1998) The theory of learning in games. MIT Press, Cambridge, MA Gilboa I, Schmeidler D (2001) A Theory of Case-Based Decisions. Cambridge University Press, Cambridge, UK Hargreaves Heap S P, Varoufakis Y (1995) Game theory: a critical introduction. Routledge, London Izquierdo L R, Gotts N M, Polhill J G (2004) Case-based reasoning, social dilemmas, and a new equilibrium concept. Journal of Artificial Societies and Social Simulation 7(3) Kolodner J L (1993) Case-Based Reasoning. Morgan Kaufman Publishers, San Mateo, USA Loui R (1999) Case-Based Reasoning and Analogy. In: Wilson RA, Keil FC (eds) The MIT Encyclopedia of the Cognitive Sciences, The MIT Press, Cambridge, USA, pp 99-101 Macy M W, Flache A (2002) Learning dynamics in social dilemmas. Proceedings of the National Academy of Sciences USA, 99(3):7229-7236 Ross B (1989) Some Psychological Results on Case-based Reasoning. In Hammond K (ed) Proceedings of the DARPA Case-Based Reasoning Workshop, pp 144-147, Morgan Kaufmann Publishers, San Mateo, USA Schank R (1982) Dynamic Memory: a Theory of Reminding and Learning in Computers and People. Cambridge University Press, Cambridge, UK Schank R, Abelson R P (1977) Scripts, Plans, Goals and Understanding. Lawrence Erlbaum Associates, Hillsdale, New Jersey Selten R (1975) Re-examination of the perfectness concept for equilibrium points in extensive games. International Journal of Game Theory, 4:25-55 Young H P (1993) The evolution of conventions. Econometrica, 61(1): 57-84

A Model of Myerson-Nash Equilibria in Networks Paolo Pin^ Dipartimento di Economia - Universita Ca' Foscari di Venezia Cannaregio 873 - 30121 Venezia [email protected]

Summary. The paper studies network formation in undirected graphs. We assume a two-stage game: agents propose connections that if realized have a fixed cost; then, given the obtained graph and its exogenous surplus (the value function), they bargain on the split. We claim that, when the surplusfi^omconnections is super-additive, the bargaining process can be solved with the Myerson Value allocation rule, an adaptation of Shapley's to graphs. This will lead to an (only theoretically, not in computations) easy characterisation of equilibria, refining the notion ofpairwise stability. We then focus our attention on the heuristical analysis of a tractable case. We run simulations, starting from different initial conditions, in order to qualitatively characterize alternative possible equilibria. For part of this last purpose we are using the simulated annealing approach, with theoretical justification for its adoption.

1 Introduction Social and economic situations in which a network-like structure should be taken into account are many and various, from cooperation networks between industries or researchers, to interrelated citations among webpages or publications, to political or trade agreements between different countries. Historically the first area of research to deal with the subject has been sociology, considering real data of relations between restricted groups of people. These analyses revealed the now popular small world phenomenon and the scale-free distribution of links between nodes. Mathematicians and physicians have considered various mechanical models of network formation, trying to fit observed or desired properties. The first such example is given by (Erdos and Remyi 1960), a first attempt with a flavour of rational behaviour is (Albert and Barabasi 1999), introducing preferential attachments (it is more likely for a new-entrant to attach to nodes with many links). Economic literature on the argument was very scarce until ten years ago, being exceptions the two contributions of (Myerson 1977; Myerson and Aumann 1988), proposing the Myerson Value (MV) (an analogous to the Shapley Value) for splitting a surplus specifically dependent on the shape of the network. qSystematic game

176

Paolo Pin

theoretical treatment of network formation starts with (Jackson and Wolinsky 1996), where agents are fully rational and decide strategically how to form their relations. The main point in most of the existing papers about non-cooperative game theory on undirected graphs formation is to determine: (i) a surplus from the realized connections; (ii) a distribution of this payoff, due to the architecture of the given network; (iii) finally a concept of equilibrium and hence stability. Steps (i) and (ii) are usually solved exogenously and, even if theory allows it, endogenous payoff due to bargaining has seldom been analysed because of the computational hardness of the problem. Myerson initial contribution is usually present as an example of allocation rule, but seldom used because it is by definition the result of a cooperative interaction and its calculation is an NP-hard problem. A usefiil result is however given by (Gul 1989) (originally on the Shapley value): under a specific model of bargaining MV can be considered the result of non-cooperative bargaining among the agents. We take then as allocation rule the MV, which can now be seen as endogenous. As final step (since we reason by backward induction) we define a classical strategy of link proposal by the agents, with a fixed cost for link formation. We apply a refining of (Jackson's and Wolinsky's 1996) pairwise stability, which is now, thanks to some nice properties of MV, strictly related to Nash equilibrium. Our equilibrium concept is far from being exactly determined by the initial conditions. However, instead of applying successive refinements with the aim of uniqueness, we consider a simple case of 6 nodes ^ and apply simulations on it. The obtained statistics will help us in evaluating the likelihood of the different possible equilibria, given alternative initial conditions, taken from popular random network formation processes. We adopt stochastic processes of graph mutations through improving paths, as described in (Jackson and Watts 2002). In the final group of simulation we try to include the possible disturbance of errors, having in mind the concept of stochastic stability, given by (Young 1998). For this purpose we use simulated annealing (SA), we are however not applying SA to optimize a global fimction (looking for social optima) but to add noise to the system. Using SA as a source of random noise is admissible because of certain properties it exhibits, which fit well in a boundedly rational version of our fi-amework. Section 2 presents the model and the theoretical results. Sections 3 and 4 illustrate the outputs of the simulations, the latter describes also S A and explains the reasons for which we think it models reasonably a behaviour with increasing rationality (i.e. learning) from the agents. Section 5 concludes.

2 The Model Let's consider a number A/" of agents, with \Af\ = N > 3. A graph (network) G is a set of links between the agents, formally G C J\f x J\f. A link is then a (directed ^ The number of equilibria seems to grow exponentially with N. N = 6 is the minimum case with a wide multiplicity of equilibria, however limited to a treatable number (~ 20).

A Model of Myerson-Nash Equilibria in Networks

177

couple) of elements from A/*: ga,b = (a, b) e 0,3. link may also be indicated by the greek letters rj,(^,6... Q will be the set of all possible G on A/*. We call graph architecture the class of equivalence in Q that can be obtained with permutations of the elements of A/*. Most of the functions and the properties in the paper will be invariant under permutations of the elements, so that we will consider graphs with this class of equivalence in mind. Subgraph of G will be synonym of subset, we will indicate also G\A = {(a, b) : (a, b) e G^a ^ A^b ^ A} when A C J\f, by another abuse of notation G\a = G\{a}. Clearly g\A QG^^AC J\f. A graph is undirected if ga,b ^ Q => 9b,a € G, irreflexive if ga,a ^ ^ V a G A/". From now on we will consider only undirected and irreflexive graphs, calling l{a) the number of links involving a and L{G) the total number of links in G (so that E„6ArK«) = 2 - L ( G ) ) . Every G on J\f defines a topology on it. A path Ha^b ii^ G between a and b is an ordered set of agents (a, a 2 , . . . a^, b)neN such that {ga,a2,9a2,a3, • --gan^bjuen Q G. XciHa.b) = \Ha,b\ — 1 is the length of the path. Ha,a is a cycle (in irreflexive graphs XG{Ha,a) > !)• The distance between a and 6 in G is dcia, b) = min{AG(i^a,6)} if defined, otherwise d(a, 6) = oo if /9 Ha,b- The diameter of a graph is Do = m8ix{dG{a, b) : a, 6 € G}. Ifdcia^ b) < oo (i.e. there is a path between a and 6) we say that a and b are connected (we will write a \X\G b). The definition of cluster is consequential: rcia) = {(a, 6) : a X\G b} C G. Here the cluster is a subgraph, we will extend the meaning of F so as to include also the implied subset of J\f. We will write F \Z G to mean that T is a cluster in G, and hence C C G A/^ if the elements of C are all and only the ones in the couples of F. G is connected if DG < oo ==> V a G A/*, / G (<^) = G (i.e. there is only one cluster). A node c is essential for a and 6 if a M & and, for all paths (a, a 2 , . . . a^, 6), c G { a 2 , . . . fln} (we will write a \x\ b). An undirected, irreflexive, connected graph without cycles is a tree. In a tree there is only one path between any two nodes, so that, if they are not directly linked, every node on the path is essential to them. When a graph is connected the distance makes our topology a metric. Let us leave geometry and enter game theory, adding some costs and profits for the agents. We will consider a fixed exogenous cost /c > 0 for any agent, for any link she has. Considering the global economy, every link will then cost 2 - k to the aggregate N agents. The value of A; may change for the purpose of comparative statics, but never within a single game. Since the presence of links will be the only source of loss for the agents, we will consider gross and net profits, depending on whether the cost of connections is considered or not. A value function is a relation F : ^ -^ E, a global profit is associated to every possible network. If A C A/* we will write VG{A) to indicate V{{{a, b) e A x A : {a^b)eG}). A value function is anonymous if it is invariant under permutations of A/*, it is strictly

178

Paolo Pin

super-additive if V G G ^, V J, K G G, nonempty and disjoint: V{J) + V[K)

<

V{J\JK). Definition 1 The connected couples value function V^^

is:

CCGAT

V^^ counts the number of connected couples in G. Every cluster of AA in G gives a contribution to V^^ of M i l ^ t i l , where \C\ is the cardinality of the cluster. If G is connected V^^{G) = ^ 2~"^ • ^^ i^ straightforward that F ^ ^ is anonymous. To see that it is also strictly super-additive we should consider that there are no subsets of G, except from the empty network, concerning nodes without links (since a graph is the set of its links by definition). We will consider V^^ as the gross value function of our model, being V^^ — 2 • A: • L(') the net one. An efficient network is one maximizing the value function. When k < ^^^ the efficient networks for our net value function are trees, that minimize the number of links (to N — 1) maintaining connectivity; when A; > ^^^^ the empty network is the efficient one. An allocation rule is a function A:GxV^ R ^ , such that Y^iLi A ( G , V) = V{G). The allocation rule is the way by which the profit is divided among the agents, it is therefore usually considered as imposed exogenously. It can however also be seen as an agreement between the agents, a cooperative solution to avoid coalitions' defections, or also, and this will be our case, the expected result of non-cooperative bargaining. Also an allocation rule is defined anonymous if it is invariant under permutations of J\f (a value function can be anonymous but the allocation not). An allocation rule isfair if Aa{G, V)-Aa{G\ga,b. V) = Ab{G, V)-Ab{G\ga,b. V) \/V :g-^R, Geg, a,beJ\f. Fairness means that the addition or the removal of any link have the same effect on its two nodes. This does not mean that they alone get half of the benefit (or damage) from the new network, whose effects could heavily influence the other's payoff Example 1 (Goyal and Vega Redondo 2004) also consider a model in which V^^ is the value function and the cost of forming a link is fixed. For every connected couple (a, 6) they compute which other agents {ci} are essential in their connection (i.e. \/ ce {ci}, al^b). They then split the unit of surplus in equal parts among a, b and all the elements of{ci}. Formally their net allocation rule is (EN stands for essential nodes, Vcc Is now a constant):

A^^(G) = Y^

^

aMG6 2 + |{c:aDS3G6}|

h y^

^

^a^^^2+\{d:b^Gc}\

'

The first summatory is what a gets directly from the connections, the second one is what she gains as a necessary intermediary. A^^ is anonymous but, as will be seen, notfair D

A Model of Myerson-Nash Equilibria in Networks

Definition 2 The Myerson value (MV) allocation rule A^^

Ar{G,V)=

Y:

179

is:

(I^|-I)y-I^l)'ry^(^)_y^(5\{,})] . •

SCAT: aeS

It is clear from the definition that the MV is exactly the Shapley value (allocation) of V on G, this is why the MV depends directly on the structure of the network. Tlieorem 1 (Myerson, 1977) V value function V, 3 one and only one anonymous andfair allocation rule, this is the MV. D From now on we will assume that the allocation rule is the MV, we will simply write A^^{G) for ^ ^ ^ ( G , y ^ ^ ) . The reason for this choice will appear clear from the result of Theorem 2, due to (Gul 1989), and the following corollary, which comes as an immediate inference. Before that a remark: do we have a MV for our net value function V^^ — 2 • /c • !/(•)? The net allocation rule A^^{G^ V^^) — k - l{a) is again anonymous and fair, so that, by theorem 1, it is the MV of the net value fianction. Gul imagines A^ agents, with common discount factor S, and a value function V on whose split is played an iterated (possibly at infinity) non-cooperative bargaining process. The bargaining works as follows: agents a and b are randomly drawn, a proposes to 6 a price to sell her the rights she owns on V, b can accept (and a leaves the market forever) or refuse (and nothing happens while time goes by). a's strategy for this game will be a fiinction a{t, ht-i, b) = (z/, A), meaning that if at time t, given past history / i t - i , a meets 6 (as buyer or seller), a will offer u for 6's rights on V (if buyer) and she will accept any amount greater or equal to A for leaving (if seller). We will call such a game a Gul Bargaining. Theorem 2 (Gul, 1989) In a Gul bargaining game between N agents, on a strictly super-additive V, 3 6 < 1 such that there is a unique stationary subgame perfect Nash equilibrium E. The payoffs ofE converge to the Shapley value as N -^ oo. D Corollary 3 Consider N agents connected by a graph G, where V^^ is the value function; in a Gul bargaining game between the agents, on V^^{G), 3 5 < 1 such that there is a unique stationary subgame perfect Nash equilibrium E. The payoffs ofE converge to A^^ as N —^ oo. D We have now a rule to solve the allocation problem for any static graph G on J\f. Let's suppose a two-stage game where agents form a network, obtain a payoff (given by the value function) and bargain on it. We can use A^^ for the second stage and then reason by backward induction. As set of strategies for the first stage we will adopt the most popular (and probably intuitive) one, following (Jackson and Wolinsky 1996): a's strategy is a vector a G {0,1}^~^, every element of a stands for one of the other agents, a^ = 1 indicates that a is willing to form a link with h. A link pa, 6 is formed only if a?, = ba = 1. We call a a link proposal strategy.

180

Paolo Pin

Even if theory allows it, it is difficult to characterize Nash equilibria under link proposal strategies. This is because there are a lot of possible strategy profiles that give the same network (and hence same payoffs) as output. A trivial example is when all strategies are the null vector, this is always a Nash equilibrium, the resulting network is the empty one. For this reason, in network formation theory, resulting graphs (an equivalence class of strategy profiles) are studied as equilibria instead of single strategy profiles. (Jackson and Wolinsky 1996) dQfinQ pairwise stability: G is pairwise stable to V and A if and only if no couple have an incentive to create or erase the link between them. Definitions Being A^^{G,V^^) - k - l{a) the (net) allocation rule, G is a Myerson-Nash equilibrium (MNE) iff:

yaeM,yrc{rjeG:

aeri}, A^^iG.v^^) > A^^ {G\r,v^^)+h\r\ .u

With V^^ as value function, A^^ as gross allocation rule, and fixed cost k for link formation, a graph is (MNE) if no agent (node) has an incentive to create a new link or sever any of her existing ones. MNE is strict when inequalities are strict. Because of fairness the first condition in definition 3 is equivalent to the first one in the definition of Jackson and Wolinsky, so that under fairness MNE is a refinement of pairwise stability. Let's suppose that an agent knows only her own strategy and the network that comes out, in the sense that for all the non existing links she does not know if one of the two agents wanted the edge to be formed. She will particularly not know if, for any link she refused, the other agent was propense. She knows however, by fairness, that, if a single new link is profitable to her, it will be also for her counterpart. When instead a new link is already added, the MV may change abruptly, so that the order by which connections are formed is decisive and the intentions of other agents are not straightforward any more. Previous limitations do not apply to links removal, that can be executed unilaterally. Example 2 Let us consider all the possible MV distribution, under V^^,for N = A (figure 4, page 182). The only MNE are the empty network III (for k > ^), the complete network IV (for k < ^ ^ , the circle I (for -^ < k < ^) and the star VII (for\
A Model of Myerson-Nash Equilibria in Networks

181

her "'"720^720^ ~ ^ — 2.32 — 5 = —2.68; join her diametrical opposite (the most rewarding one), getting ^ ^ ^ - 15 ~ 12.07 - 15 = - 2 . 9 3 ; or, finally, cut her two links, with a null payoff. Hence, when k = h, the 16-nodes circle is pairwise stable butnotMNE. D

Fig. 1. 3 networks for 16 nodes

Let us end the section with three results^ and two conjectures. Proposition 4 The empty network is MNE for k > \, the complete network is for

k < 7 v ( i ^ ' ^^^ ^^^^ ^^for i
^ .

D

Proposition 5 Every MNE is either the empty network or a connected graph. D Conjecture 6 For every G and k > 0,3 an MNE. D Proposition 7 Under V^^, the two coincide. D

in general, A^^

^ A^^,

however, when G is a tree,

In small world phenomena 6 is di famous number^. Example 1 shows that, V /c, a circle is not MNE for N = 16. It is in fact an equilibrium, for some k, for N < 15 (where the diameter, i.e. the highest distance, is < 6), and is not, for any k, for greater N (where the diameter is > 6). Moreover, a queue (as graph q in figure 4, page 183) is MNE, for some k, only as long as iV < 7. This is again a restriction on the diameter DQ to 6. Conjecture 8 For every MNE G,\/ k,\/ N, DG<6.

U

3 Simulations Uniqueness of MNE is not granted, e.^f. in figure 4, for /c = ^, both networks I and VI can be stable. We do not consider these sunspots of equilibria a drawback of our model, but a richness. Instead of refining MNE with the aim of a deterministic achievement of the only possible equilibrium, we will try to characterize alternative outcomes by their likelihood ^ Proofs can be found in the Appendix of a larger version of this paper at h t t p : / / v e n u s .unive . it/pin/Paolo_PinJyiyerson.pdf. ^ As an example (Watts 1999) cites the anecdote by which every person in the U.S. is 6 handshakes far from the President.

182

Paolo Pin

to happen. For this analysis we run computer simulations, from different starting configurations and with different values ofk, to infer the diffQrQnt probabilities. We need first some theoretical background.

I '^

lc<5/12

MNE if 1/12< k<5/12 '

^

^

k<./12 \

Z

I k>./12

13/12

,3/12


O

IIZ ^^3„2 I

V

IV 3/2

k
19/12

III

II

"^

"^^/n

^

k
4/3

7/6

5/6

^^^^^

.

A ••••* NI

0

o o

o

)

0

MNE if k>l/2

T k>7/6

\

//.

.VII k
o

r

'k<7/6 ^^'^ J\ ,

I ,,3,2

VI 17/12

0^

™

/

MNE if k<1/12 Fig. 2. Myerson allocations and improving paths for AT = 4 (MNE are strict) (Jackson and Watts 2002) consider the set Q of all possible graphs on H and the directed meta-network 0 of all the possible improvements under pairwise stability. They call an improving path (IP) a path on 0 . Since 0 is a directed graph we may find endpoints or closed cycles; endpoints will be pairwise stable equilibria on M. In mathematical terms the system induced by improving paths is a discrete Markov chain while equilibria are absorbing states or, in the language of dynamical systems, basins of attraction. This framework can easily be adapted to our model, it is just a matter of adding some more arrows (directed links) in the meta-network 0 when a single agent can profitably sever more than one link in a non-meta-network G. Example 4 Figure 2 illustrates how the meta-network 0 changes, according to k, when N = 4. Here only strict MNE are considered. The arrow from the star VII to the empty network III, when k > ^, would not be allowed by pairwise stability but is possible here because of the MNE refinement. Graph W, among others, shows that the achievement of a particular equilibrium is a non-deterministic process. If we are in II, and ^ < k < ^, we could indifferently go left and end in the circle I, or go down to VI and then right to the star VII. D We are running our simulations with IP on AT = 6. Even with such a low value a systematic sketch of the complete meta-network (as suggested in figure 2, for N = i) becomes an impossible task. We will consider different starting configurations: the empty network (when A: < | ) , the complete one and some random ones with fixed number of links. To construct the random ones we will use both the model of (Erdos and Remyi 1960) and the preferential attachment model of (Albert and Barabasi

A Model of Myerson-Nash Equilibria in Networks

183

1999). For the latter we take the two cases mo = m = 1 and mo = m = 2, calling II = mo = m. We will check statistically the outcomes of improving paths for three indicative values of A: ( | , I and 1) running 100 simulations for every case^. It is clear that, when the outcome of the random network generator is already a MNE, IP will have no effect on it.

iyi2< k <9/20

1/10< k <13/60

h

i/6< k <7/6

i/5< k <8/lS

i

j

k 9169/60

•"/»2

2/5< k <21/20

5/12< k <3y60

519/12 ^ ^ ^ ^ ^^ ^^^^^

^ O^ 6 ^^^^^ ^ ^^^^^

9*^9/20

^

^^^"^ O „,^^ ^ ^23/15

^23^^^ ^ «97/60 5/6< k <239/180

Fig. 3. Legend of the equilibria obtained in the simulations for AT = 6

)29/20 p41/15 V139/60 A

1/10 < k<3/20

4/15 < k < l / 3

V / ^ y56/15

o

Q . _ X ( 49/20

1/2 < k

8/15 < k < 47/60

b 199/60

21/20 < k < 29/20

Fig. 4. Other possible equilibria for AT = 6

Figure 3 is a legend for the MNE obtained in the simulations. Every network is a strict MNE in the interval of A: written below it. The graphs in figure 3 are however not the exhaustive set of all the MNE for N = 6, \/ k > 0. Other possible MNE are listed in figure 4. It should be noted that the empty network o could be an MNE for A: = | and (strictly) fork = 1, but does however ^100 simulations with 6 nodes and 3 values of k (1800 simulations in all) may seem a poor dataset. We think that our result are nevertheless statistically significant.

184

Paolo Pin

Table 1. Improving paths -finalequilibria out of 100 trials for each k and starting configuration (*) significative, (**) large deviations A- - i Starting from... | 11 '^- 5 Empty (n) 1 r e 4 9 , c 3 7 , alO, no links | b3,dl Erdos c 50*, e 38, a 10, 5 hnks 1 bl,dl Barabasi/i == 1 1e 4 2 , c 3 0 , d l 3 * * , 5 links 1 a 8, b 7 Erdos c 63*, e 22*, a 12, 10 links b 2, d 1 Barabasi)U = 2 e51,c35, all,b3 10 links Complete c72**,el7**, all 15 links

k= l

I

—

j 2 8 , k 2 6 , f 19, e37,f33, g 28, d 2 il5,hll,dl g 5 r , f 2 6 , j26,h25*,dll**, dl4**,e9* il8,kl4*,f6 e47,f28, J29,k29,f25, g 24, d 1 hll,i5,dl e37,f31, J33,k26,hl6, g 30, d 2 f 15,1 8, d 2 e47,f29, k29,j27,f25, g23,dl hl2,i7

never appear in the simulations with IP. This is a first hint on how admissible MNE can be very unlikely absorbing states under improving paths' stochastic processes. Table 1 shows the results: in every cell we have the frequency of every MNE that happened to be an outcome of the 100 simulations. Comparison should be made along columns, where k, net allocation rule, and hence the set of all possible equilibria are fixed. (*) and (**) indicate significative and large deviations. Most considered networks in the literature are the empty one o, the star d and the circle f. What instead comes out from the computer experiments is that they are certainly not the most likely outcomes of IP when TV = 6. Unexpected configurations dominate. We can infer that c is very probable when k = | , it gives equal gross and net payoffs (since every node has 3 edges) to the agents but is surely less efficient than the circle, e is even more surprising since it is a strong basin of attraction for a wide set of k's, including both \ and ^. When A: = 1 we have almost only trees (efficient networks), but the most likely seem to be the most asymmetric j and k. A comment can be made on the first column, about the reverse likelihood of c and e between two sets of initial configurations: when the starting graph is empty or Barabasi-generated, at one side, when it is complete or Erdos-like, at the other. Let us consider that: (i) how IP moves from the empty network is a process similar to preferential attachment; (ii) e has less links and is hence more efficient than c. We can infer that, under preferential attachments conditions, more efficient networks are reached.

A Model of Myerson-Nash Equilibria in Networks

185

4 Simulated Annealing Our purpose in this section is to analyse four MNEs from the previous one: the popular star d and circle f compared to the successful c and e. In particular we will act in the environment of the meta-network 0 , move the system slightly away from an equilibrium and check how often we will get back to the initial MNE we deviated from. In the language of dynamical systems we are roughly measuring the area of stability of the equilibria^. The shift from MNE is done adding a disturbance with simulated annealing. (Kirkpatrick et al. 1983) proposed simulated annealing (SA) as a heuristic optimization algorithm. SA works very well on discrete systems where a value function V has to be maximized and its basic rules are simple. Let us suppose that we are in a configuration AQ and a random single step in the discrete system is proposed, so to reach configuration Ai. If the shift improves V the move is accepted and we will restart from ^ i . If it does not, and the loss is VQ — VI, we accept the move only with probability e~^'^^^~^^\ ^ is a parameter that in the original physical set-up is the inverse of temperature. During the process jS is increased (the cooling), it is clear that at the limit /^ —> oo only good moves will be accepted. Moreover the authors proved that if the cooling process (5 ^ oois slow enough (usually it is linear), global optima are reached. We are adapting SA to 0 . Let us fix a starting graph as GQ. TWO random nodes a, b e N will be selected, they may be linked or not. If they have benefit in the gross allocation by changing the status of their link (sever it if present, form it if absent) they will do so, moving to graph d . If not they will change only with probability exp(-/? • \A^^{Go) - A^^{Gi)\), wich is equal by fairness to exp(-/3.|Xr^(Go)-^r^(Gi)|). (3 can be considered the homogeneous rationality of the agents in J\f. It can be argued that /^'s increase imitates a learning process. At the limit /? —> oo the process is analogous to IP under pairwise stability. The simulated process is as if agents proceed with bounded rationality, considering the effects of bargaining. If they are satisfied they confirm their actions, otherwise they may either come back or experiment what will happen. As the learning process goes on, experimentation becomes less and less likely. As premitted we are not using SA to find equilibria, but to add a small disturbance to the system, attempting only few steps of it^. It should be noted that, since starting configurations Go are MNE, the first deviation is always accepted with probability exp(-/?o • \A^^{Go) - A^^{Gi)\). After that noise is added, with the short SA process, we apply again IP ( i e . go on with (5 = oo and the additional refinements of MNE), to reach a new (possibly the same) MNE. We compare four MNE for TV = 6, the ones labelled c, d, e and f in figure 3 (page 183). In every simulation we take as Go the original MNE; we add a small decreasing ^ In discrete systems there is only one definition of stability: always strong and never asymptotic. For our purpose it would coincide with (Young 1998) stochastic stability. We are making 3 loops for each (3 G {1, 2,3,4}.

186

Paolo Pin

noise with SA (as if we began with irrational agents that, however, quickly learn) and reach a (possibly) new graph GSA, not far from GQ\ we run IP from GSA and see how often we reach GQ again. We are not checking for stability, as in dynamical systems, or for stochastic stability, as in (Young 1998), since they are both definitions and admit only positive or negative answer. We suppose that our MNE are stable and we want to measure how strong (probable) absorbing states they are in their neighborhood of similar ones. This, both geometrical and probabilistic, measure is what in physics is commonly called stability. We hope not to create ambiguity by calling it m-stability. Table 2. Improving paths after simulated annealing -finalequilibria out of 100 trials for each A; and starting MNE (**) large deviations

(c)

11

A- -

'^ ~

-i^

240

'^ ~

120

'^ — 16

MNE for

c 66, a 20, a62**,cl7, c 7 5 , a 2 1 , m 2, b 2 bl7,m3 10 ^ '^ -^ 60 1 b 8 , 1 8 , u 1." "17 \ h — Jk= l Star (d) ^ - 12 g > /C > -g- d79, h 18, ±99, d98, 12, p i hi o2 (e)

M ^ — fin

t>k>^A ^

AC ^

1 ml2 \ h -

Circle (f) 5

e88,

20

^

f58,p29,

1 ill,h 1

A - - 11 ^ — 30

e77, n23

'^ -

40

'^ — 60

e 7 5 , f 19, g6 '^ -

80

£65, p 24, f 4 8 , j 3 8 , 17, h 4 k22,13

Table 2 shows the results. We have tried three different /c's for every starting MNE, these are the three quartiles of the intervals under which the MNE are equilibria. Since the /c's vary from row to row, elements in the same columns should not be compared as in previous section. Figures 3 and 4 serve again as legend for the results. Because of the arbitrary choice of the ^s, the numbers are valid only for relative comparison, nevertheless the underlined ones can easily be accepted as a rough measures of m-stability, in the empirical sense we defined above. The simulations concerning MNE c shows,firstof all, that m-stability varies widely with k and is not a concavefimctionon the domain of existence (the k interval), as could be expected. There is no linear relation between the likelihoods in table 1 and the m-stability, there is instead between the m-stability and the division given by the allocation rule, c and f arefixllysymmetric and divide equally the value fiinction among the 6 agents; e allows for some differences (2 nodes get almost 3, the other 4 a bit more than 2); d is almost a monopoly, the central agent gets almost 6 while the other 5 share the rest. It can be argued, by the elements we have from the simulations, that m-stability is

A Model of Myerson-Nash Equilibria in Networks

187

strong in the networks where the allocation rule does not split equally the resources. From the data we have, we can say that m-stability of an MNE G is correlated with the (properly defined) distance between A^^{G, y ( ^ ^ ) ) and a power law distribution of the value fiinction.

5 Conclusion Our proposal is to model, with empirically acceptable rules, a game of network formation with a variety of equilibria. We try to characterize the equilibria analytically, as far as possible, and with some simulations, in order to identify typical properties of real networks such as the small world phenomenon and the scale free distribution of links. Our value fiinction and strategies are the ones of the connection model of (Jackson and Wolinsky 1996) with no decay, the same value fiinction has been recently used by (Goyal and Vega Redondo 2004). Our allocation rule is the (Myerson 1977) Value, that has been proved to be the exact result of bargaining in networks under very specific conditions, and we think is anyhow a good approximation of any kind of non-cooperative anonymous bargaining. This allocation rule exhibits a property (fairness) that allow us to apply a refinement of pairwise stability, again from (Jackson and Wolinsky 1996), that intuitively relaxes the necessity of coordination in 2-agents deviations. Only a few thousands of simulations could be run, because of the NP-hardness of the problem, but the obtained results are however significant for what concerns the few conclusions we get. For different values of the fixed cost k of forming a link, some unexpected equilibria are much more likely to appear than others. It can be argued,fi*omthe data, that preferential attachment induces a reduction in the total number of links and hence moves toward efficiency (i.e. maximization of the total payoff). We have tested then for stability in the dynamical sense of the term, finding that more equi-distributed equilibria are much less stable than those showing scale-free distribution of links and payoffs.

References 1. Albert R, Barabasi AL (1999) Emerging of Scaling in Random Networks. Science 286: 509-512. 2. Aumann RJ, Myerson RB (1988) Endogenus formation of links between players and coalitions: an application to the Shapley Value. In: Roth A (ed.) The Shapley Value: 175191, Cambridge University Press. 3. Dutta B, Jackson MO (2003), On the Formation of Networks and Groups. In Networks and Groups, eds. Dutta B, Jackson MO, 1-16, Springer. 4. Dutta B, Mutuswami S (1997) Stable Networks. Journal of Economic Theory 76: 322344.

188

Paolo Pin

5. Erdos P, Remyi A (1960), On the Evolution of Random Graphs. Publication of the Mathematical Institute of the Hungarian Academy of Sciences 5: 17-61. 6. Goyal S, Vega Redondo F (2004) Structural holes in social networks. December 2004 version, http://privatewww.essex.ac.uk/~sgoyal/structuralholes-dec21 .pdf. 7. Gul F (1989) Bargaining foundations of Shapley value. Econometrica 57: 81-95. 8. Kirkpatrick S, Gelatt CD, Vecchi M (1983), Optimization by Simulated Annealing, ^c/e«ce 220: 671-680. 9. Jackson MO (2005), Allocation rules for network games. Games and Economic Behavior 51: 128-154. 10. Jackson MO, Rogers BW (2005), Search in the Formation of Large Networks: How Random are Socially Generated Networks?. January 2005 version, http://www.hss.caltech.edu/ jacksonm/netpower.pdf. 11. Jackson MO, Watts A (2002) The Evolution of Social and Economic Networks. Journal of Economic Theory 106: 265-295. 12. Jackson MO, Wolinski (1996) A Strategic Model of Social and Economic Networks. Journal of Economic Theory 71: 44-14. 13. Myerson RB (1977) Graphs and cooperation in games. Mathematics of Operations Research 2: 225-229. 14. Mutuswami S, Winter E (2002) Subscription Mechanism for Network Formation. Journal of Economic Theory 106: 242-264. 15. von den Nouweland A, Slikker M (2000), Network formation models with costs for establishing links. Review of Economic Studies 5: 333-362. 16. Watts A (2001), A Dynamic Model of Network Formation. Games and Economic Behavior 34:331-341. 17. Watts DJ (1999), Small worlds: the dynamics of networks between order and randomness, Princeton University Press. 18. Young HP (1998), Individual Strategy and Social Structure, Princeton University Press.

Market Dynamics

stock Price Dynamics in Artificial Multi-Agent Stock Markets A.O.I. Hoffmann, S.A. Delre, J.H. von Eije and W. Jager Faculty of Management and Organization University of Groningen Landleven 5 9700 AV Groningen The Netherlands

[email protected]*

1

Introduction

Worldwide financial markets have hit the news numerous times in the recent past due to the striking behavior of both (individual) investors active on these markets as well as the aberrant market developments that resulted from this aggregate investor behavior. In the last two decades, the two most obvious examples are the stock market crash of 1987 and the Internet bubble, and currently investors in Nano-technology may follow a hype (Surowiecki 2004; The Economist 2005). Program trading, overvaluation and illiquidity could together explain part of the 1987 crash. The literature also suggests explanations for the Internet bubble, like the role of the media (Bhattacharya et al. 2004), processes of groupthink (Valliere and Peterson 2004), irrational exuberance (Shiller 2005), and overly optimistic expectations of the business prospects for companies active in this sector (Schleifer 2000). Also computer-simulated markets with individual adaptive agents may be used in trying to explain observed market phenomena. A summary can be found in e.g., (LeBaron 2000). These models range from relatively simple and straightforward (Day and Huang 1990; Lettau 1997), to more complex and revolutionary models like the Santa Fe Artificial Stock Market of (Arthur et al. 1997). The authors of this paper hope to contribute to the explanations of the observed financial market phenomena by incorporating social needs, social interactions and social networks of investors as introduced in Hoffmann and Jager (in press). The objective is to identify critical micro-level factors that drive investors' behavior and to explain

The authors gratefully acknowledge T.L.J. Broekhuizen, J.J. Hotho and B. Lijnema for their useful comments on earlier drafts of this paper. The usual disclaimer appHes.

192

A.O.I Hoffmann et al.

macro-level phenomena that may be the result of aggregate micro-level investor behavior. This paper is organized as follows. The second section describes the background of the study. The third section introduces the simulation model. The fourth and fifth sections are devoted to a number of simulation experiments. The sixth section discusses the results until now and indicates the limitations of the study as well as areas for future research.

2

Background

In addition to more financially oriented needs for e.g., monetary profits, investors may also strive to satisfy other, more social needs by the act of investing. For example, an environmentalist may satisfy three different needs by purchasing green stocks. First, he^ may obtain financial gains by doing so and in this way satisfy the need for subsistence (Max-Neef 1992). Second, he may satisfy a need to identify himself with green companies. Third, by investing in this company he may indirectly fulfill his need to participate with other environmentalists. When investors try to satisfy needs that are more socially oriented, their decision-making behavior may also become more socially oriented and social interaction is expected to increase. Moreover, investors may then also be inclined to use the investing behavior of other investors as an input for their own decision-making or they may even simply copy that behavior (Deutsch and Gerard 1955; Hirschleifer 1993; Suls et al. 2002; Mangleburg et al. 2004). This does not imply that socially oriented decision-making only originates from social needs. Uncertainty, for example, may also be a driver of socially oriented behavior (Festinger 1954; Jones and Gerard 1967; DiMaggio and Louch 1998; Cialdini and Goldstein 2004). Socially oriented decision-making may, moreover, occur in social networks that connect friends with friends of friends and so on (Watts and Strogatz 1998; Watts 2001; Janssen and Jager 2003). A practical example is expected to enhance the understanding of the way the above propositions may work out in investment practice. On Monday morning, investor Alpha comes to work and receives some interesting information on a publicly traded company. During a lunch meeting on the same day, he shares this information with acquaintances and colleagues. This people then might start to trade on this information. A friend of these friends, investor Beta, desires to belong to this group of investors and also purchases the stock. After a short period of time, investors trade either because of the company related information they received (social informative influence) or because they want to belong to the group of investing friends (social normative influence). At the end of the week, a formerly tranquil stock may be making headlines due to an unexpected increase in volatility. ^ Whenever the authors for reasons of simplicity and consistency use male forms like 'he' or 'his', the reader can obviously substitute them for female forms like 'she' or 'her'.

Stock Price Dynamics in Artificial Multi-Agent Stock Markets

193

We hypothesize that beta investors, that is investors who exhibit socially oriented decision-making, may ignite such aberrations. Recent research on (social) networks has demonstrated that many large networks display a scale-free powerlaw distribution for node connectivity (Barabasi and Albert 1999). In terms of market dynamics, this may imply that a small proportion of investors having many contacts, so-called 'hubs', may have an exceptional influence on the investing behavior of others. An approach that wants to explain market volatility over time might do wise to incorporate the concept of beta investors as well as the concept of networks.

3

The Simulation Model

As a first attempt to implement our framework, we start with the relatively simple model of investor behavior of (Day and Huang 1990). In the model, investors can follow either a more fundamentally based "rational" strategy (called the astrategy) or a more socially based strategy (called the p-strategy). The a-strategy is based on a comparison between the current price p and a given long-run investment value u. Whenever the market price is below the long-run investment value, the a-investor buys. Whenever the market price is above the long-run investment value, the a-investor sells. When the market price equals the long-run investment value, the a-investor holds. This behavior is limited by a topping price M and a bottoming price m. The p-strategy suggests more socially oriented behavior. Pinvestors buy when they expect an upward price trend (whenever the current price p is above a given current fundamental value v that initially equals u) and sell when they expect a downward price trend (whenever p is below v). This kind of behavior is akin to herding behavior (see e.g., Orlean 1989; Scharfstein and Stein 1990; De Bondt and Forbes 1999; Bikhchandani and Sharma 2000; Cont and Bouchaud 2000; Hirschleifer and Teoh 2003; Banerjee and Fudenberg 2004; Lux 2004). The last participant on the market is the market maker. The market maker sets the market price according to the combined total excess demand or supply of the a- and P-investors. The extent to which investors follow an a-strategy or a p-strategy is weighted by the parameter S that represents the social susceptibility of an investor. Stock markets and stocks alike may differ to the extent that investors focus more on fundamental characteristics of a share like price/earnings ratios and beta's, or focus more on social aspects of a share like information about which shares friends, colleagues or prominent finance experts buy. Different investors have different values of S. That is, there is heterogeneity in the investors with regard to their strategies. Moreover, investors may change their S given the circumstances, which leads to dynamism in the strategies they use. We can formalize the above in the following simple formula for total excess demand that is based on (Day and Huang 1990): E = (l-S)-(u-p)-f(p) + S - ( p - v )

(3.1)

194

A.O.I Hoffmann et al.

Here f (p) is a weighting function that represents the chance of lost opportunity caused by either failing to buy when market prices are low or failing to sell when market prices are high.

4

Simulation Experiments with Simultaneous Updating

In the first series of simulation experiments, we tested for different values of S. When S is very low or close to zero, indicating that fundamental traders dominate the market, we expect the price to stabilize and to remain at a constant level after a number of time steps. This equilibrium price, however, is higher than the starting price and the long-run investment value. So, the small proportion of P-investors did not succeed in pushing the market out of equilibrium into chaotic price behavior, but did induce an upward shock. In figure 4.1, we present an example of the price dynamics for this first series of experiments (with p = 0.501; u = 0.500; S = 0.03 and 10 agents).

0.6 0.58 0.56 0.54 0.52 0.5 0.48 0.46 0.44 00 CNI

CO

CD

LO LO

CD

CO

CNI

cx)

o o

Time Fig. 4.1. Price series from experiment 2 In the second experiment we increased the number of agents, which resulted in more chaotic price dynamics as can be observed in real stock markets. However, increasing the value of S (i.e. the weighting of the P-strategy) has a similar effect. In figure 4.2, we present an example of the price dynamics for this second series of experiments (with p = 0.501; u = 0.500; S = 0.02 and 100 agents). DurbinWatson tests and GARCH analysis indicated both autocorrelation and heteroscedasticity to be present in the price series from this series of experiments. Both effects can also be observed in real-life stock market returns. However, the distribution of returns was far from (log) normal, which is in contrast to the distribution of returns that are observed in the real stock markets.

Stock Price Dynamics in Artificial Multi-Agent Stock Markets

195

0.8 0.6

•^

0.4

r'^XATMYWlJ^^

"^0.2

o4 cj)

r^ '^

Lo CO CNI CO

' ^ o r ^ L o c O ' ^ o r ^ ' ^ ' ^ L O c D r ^ o o o o c j )

Time Fig. 4.2. Price series from experiment 2 In a further experiment, we increased the weighting of the p-strategy over certain boundaries, which leads to stock price dynamics that very quickly get out of bounds. In these situations, the price dynamics become so excessive, that we instructed the computer to stop the simulation. This is visible in figure 4.3 (where we used p = 0.501; u = 0.500; S = 0.100 and 100 agents) by the absence of price developments after a short period of very volatile stock price dynamics.

1 0.8 0.6 0.4 0.2 0 O)

00 CNI

r^

CO

CD

^

LO LO

^ CD

CO

CNI

r^

cx)

• ^ —

CJ)

o o

Time Fig. 4.3. Price series from experiment 3 In the first simulation experiments, the investors were homogeneous, i.e. S was set at a certain value and this value was the same for all investors. We hypothesized that as heterogeneity would be introduced in investors' strategies, market volatility would reduce. We expected that with heterogeneous agents, a higher average level of S would be needed to cause the same market volatility as found with homogenous agents. In further experiments, which will not be discussed here into detail.

196

A.O.I Hoffmann et al.

we introduced heterogeneity by not using one fixed average value of S, but using a range of randomly generated values of S to lie within 0 and a set value. This situation would represent the actual market situation in a more realistic way, as it is a well-known fact that not all investors follow exactly the same strategy. Although this setting eventually resulted in the same average S for the experiment as when directly setting S at a certain average value, we expected a more balanced overall market behaviour, as different strategies may cancel each other out. However, simulation experiments that were performed with this adjusted setting demonstrated this effect to be only minor. This might have been caused by the fact that we did not introduce the social network yet and therefore there was no direct social interaction among investors.

5

Simulation Experiments with Sequential Updating

A persistent problem in the previous simulation experiments was that the parameter space for which useful price series, like those of figure 4.2, were obtained was very small. This problem was caused by the fact, that in the simulation model that we adapted from (Day and Huang 1990), all investors made their decisions to buy, sell or hold at exactly the same moment in time. This is not realistic, as in real stock markets prices may be changing in response to the trades of individual market participants. In order to mitigate this problem, we subsequently decided to use sequential updating of the market. Now, a random process is responsible for the order in which the investors are selected to make their decision. After each individual trade, the market is cleared and a new market price is calculated. Then, another investor trades, reacting partly to the trade of the other investor, as the price has now probably changed. In the following, we will report on a number of experiments that were performed after this change in the market updating mechanism. In the fourth experiment, investors are still homogeneous, that is, they all have the same value of S. In this case, the price always reaches equilibrium. No matter how strong the P-strategy is, the a-strategy is always able to bring the market to equilibrium, though at a higher level. However, the price level at which an equilibrium is reached depends on the value of S. Higher values of S lead to equilibria at higher levels. In figure 5.1, the price developments for different levels of S are presented (with p = 0.501; u = 0.500 and 100 agents).

Stock Price Dynamics in Artificial Multi-Agent Stock Markets

197

O O O O r ^ C D L O ^ C O C M ' ^ O

Time Fig. 5.1. Price series from experiment 4 In the fifth experiment, the investors are heterogeneous, with every investor having a different value for S. This setting leads to a change in the price dynamics. Now, the price developments only become chaotic when the P-strategy is strong enough, that is, at higher levels of S. At relatively low levels of S, the price still reaches equilibrium. In figure 5.2, we report on the price developments that result from three different settings of S; randomly assigned between respectively 0 and 1, 0.4 and 1 and 0.5 and 1. In the latter two cases, the graphs of the returns had distributions with a single top in the middle, though only the simulation where S was allowed to vary between 0.5 and 1 gave returns for which normality could not be refuted according to the Bera-Jarque test.

-S=[0.5, 1.0] S=[0.0, 1.0] -S=[0.4, 1.0]

o

Time Fig. 5.2. Price series from experiment 5 An ultimate requirement of stock market research is that one tries to explain outcomes in terms of real stock price data. Because traders that follow market developments (beta investors) may generate returns and risk fluctuations over time, we estimated the Garch (1,1) equation (Brooks 2002) for the Down Jones index over

198

A.O.I Hoffmann et al.

the period of September 7th until November 24th 2004 (57 daily observations). This represented a recent period without special holidays. We used dummies for days of the week (Monday till Thursday). The resulting constant term for each equation represented the expected returns on Friday. Table 5.1 indicates the estimated coefficients for the Dow Jones index, the concomitant variance equation as well as the results from experiment 5 with S=[0.5,1.0] and S=[0.4,1.0] respectively. Table 5.1. Arch and Garch estimates Dow Jones percentage returns CoefProb. ficient

c MONDAY TUESDAY WEDNESDAY THURSDAY

0.269 -0.190 -0.118 -0.091 -0.381

0.283 0.642 0.732 0.756 0.190

S=[0.5,1.0]

S=[0.4,1.0]

CoefJ. . ™ . ^ Prob. ficient

CoefJ. . ™ . ^ Prob. ficient

0.065

0.007

0.000

0.845

0.508 0.059 0.000

0.000 0.503 0.610

0.358 0.000 0.000

Variance Equation RESID(-l)/^2 GARCH(-l)

c

0.057 -0.150 1.025

0.200 0.066 0.000

0.001 0.306 0.803

Adj. R-squared DW statistic

-0.151 1.716

EquationI Statistics -0.034 1.923

-0.034 1.601

Then we compared these real life results with the outcomes of our simulations for at random allocated beta investors that gave price fluctuations over time (see figure 5.2 and table 5.1). Because our simulation results are rather arbitrarily chosen, we do not compare the estimated coefficients, but we like to refer to the similarity of the significances found for the Dow Jones index as well as our estimates based on randomly allocated beta investors (especially S=[0.5-1.0]) indicated with italics. The similarity of these findings suggest that Arch and Garch effects of real life stock markets might be attributed to beta investors that randomly enter the market. A next step in the research is therefore the remodeling of market participants' behavior in such a way that the behavioral outcomes mimic stock markets characteristics like mean return, risk, and the development of risk over time.

Stock Price Dynamics in Artificial Multi-Agent Stock Markets

6

199

Discussion

In the experiments discussed in this paper, p-investors derived social information directly from the market. In subsequent simulation experiments, investors may also obtain (social) information from their social networks. They may observe the behavior of other investors in their social network and use this behavior as a benchmark for their own behavior. In this paper, we did not include the arrival of new information on stocks in the market. However, we made some preliminary simulations by randomly changing the current fundamental value v of individual investors. This news creates turbulence in the market when investors are homogeneous and increases the turbulence when investors are heterogeneous. In future papers we will report on the results of including the investor's social network in our multi-agent simulation model. Moreover, we will experiment with different network structures and different ways of introducing information on the long-run investment value u and the current fundamental value v in the network. As different network types have different information diffusion characteristics (Cowan and Jonard, 2003) we expect to reveal how different network structures affect stock market dynamics in different ways. Furthermore, varying the number of hubs and their importance as well as network sizes is expected to contribute to a better understanding of the role of social networks on investor behavior and stock market dynamics. One limitation may remain in the model, even after incorporating sequential updating and news. This is that the market dynamics are generated by the actions of the investors, but the cognition of the investors is never affected by the evolution of the market. In order to make the model more interesting, we need to include a feedback mechanism that influences the decision making of the investors (Arthur 1994, 1995). An example of such a feedback mechanism might be that investors change their strategies according to the returns they get.

References Arthur W B (1994) Bounded Rationality and Inductive Behavior (the El Farol Problem). American Economic Review 84: 406-411 Arthur W B (1995) Complexity in Economic and Financial Markets. Complexity 1: 20-25 Arthur W B, Holland J, LeBaron B, Palmer R T P (1997) Asset pricing under endogenous expectations in an artificial stock market. In: Arthur W B, Durlauf S, Lane D (eds)The economy as an evolving complex system 11. Addison-Wesley, Reading MA, pp 15-44 Banerjee A, Fudenberg D (2004) Word-of-mouth learning. Games and Economic Behavior 46: 1-22 Barabasi A-L Albert R (1999) Emergence of scaling in random networks. Science 286: 509-512 Bhattacharya U, Galpin N, Ray R, Yu X (2004) The role of the media in the Internet IPO bubble.

200

A.O.I Hoffmann et al.

Bikhchandani S, Sharma S (2000) Herd behavior in financial markets: a review. IMF Staff Papers 47: 279-310 Brooks, C (2002) Introductory Econometrics for Finance. Cambridge University Press, Cambridge, U.K. Cialdini, R B, Goldstein, N J (2004) Social Influence: Compliance and Conformity. Annual Review of Psychology 55: 591-621 Cont R, Bouchaud J (2000) Herd behavior and aggregate fluctuations in financial markets. Macroeconomic dynamics 4: 170-196 Cowan R, Jonard N (2003) Network structure and the diffusion of knowledge. Journal of Economic Dynamics and Control Day R H, Huang W (1990) Bulls, bears and market sheep. Journal of Economic Behavior and Organization 14: 299-329 De Bondt W F M, Forbes W P (1999) Herding in analyst earning forecasts: evidence from the United Kingdom. European Financial Management 5: 143-163 Deutsch M H, Gerard B (1955) A study of normative and informative social influences upon individual judgment. Journal of Abnormal and Social Psychology 51: 629-636 DiMaggio P, Louch H (1998) Socially embedded consumer transactions: for what kind of purchases do people most often use networks? American Sociological Review 63: 619637 Festinger L (1954) A theory of social comparison processes. Human Relations 7: 117-140 Hirschleifer D (1993) The blind leading the blind: social influence, fads and informational cascades. Finance 24 Hirschleifer D, Teoh S H (2003) Herd behaviour and cascading in capital markets: a review and synthesis. European Financial Management 9: 25-66 Hoffmann A O I, Jager W (in press) The effect of different needs, decision-making processes and network structures on investor behavior and stock market dynamics: a simulation approach. ICFAI Journal of Behavioral Finance Janssen M A, Jager W (2003) Self-organisation of market dynamics: consumer psychology and social networks. Artificial Life 9 Jones E E, Gerard H B (1967) Foundations of social psychology. John Wiley and Sons, New York LeBaron B (2000) Agent-based computational finance: suggested readings and early research. Journal of Economic Dynamics and Control 24: 679-702 Lettau M (1997) Explaining the facts with adaptive agents: the case of mutual fund flows. Journal of Economic Dynamics and Control 21: 1117-1148 Lux T (2004) Herd behaviour, bubbles and crashes. The Economic Journal 105: 881-896 Mangleburg T F, Doney P M, Bristol T (2004) Shopping with friends and teens' susceptibility to peer influence. Journal of Retailing 80: 101-116 Max-Neef M (1992) Development and Human Needs, in P Ekins and M Max-Neef (eds), Real-life economics: understanding wealth creation. Routledge, London/New York Orlean A (1989) Mimetic contagion and speculative bubbles. Theory and Decision 27: 6392 Scharfstein D S, Stein J C (1990) Herd behavior and investment. American Economic Review 80: 465-479 Schiller R J (2005) Irrational Exuberance. 2nd edn. Princeton University Press, New Jersey Shleifer A (2000) Inefficient Markets: An Introduction To Behavioral Finance. Oxford University Press, New York

Stock Price Dynamics in Artificial Multi-Agent Stock Markets

201

Suls J, Martin R, Wheeler L (2002) Social comparison: why, with whom, and with what effect? Current Directions in Psychological Science 11: 159-163 Surowiecki J (2004) The Financial Page: Bring on the nanobubble. The New Yorker 68 The Economist (2005) Beyond the nanohype. 366: 23-24. Valliere D, Peterson R (2004) Inflating the bubble: examining dot-com investor behaviour. Venture Capital 6: 1-22 Watts A (2001) A dynamic model of network formation. Games and Economic Behavior 34:331-341 Watts A, Strogatz S H (1998) Collective dynamics of'small-world' networks. Nature 393: 440-442

Market Failure Caused by Quality Uncertainty Segismundo S. Izquierdo\ Luis R. Izquierdo^, Jose M. Galan^, Cesareo Hernandez^ ^University of Valladolid (Spain) ^The Macaulay Institute (Aberdeen, UK) ^University of Burgos (Spain)

Summary. The classical argument used to explain why markets can fail when there is product quality variability (e.g. the used car market) relies heavily on the presence of asymmetric information -i.e. there must exist some reliable quality indicators that can be observed by sellers, but not by buyers. Using computer simulation, this paper illustrates how such market failures can occur even in the absence of asymmetric information. The mere assumption that buyers estimate the quality of the product they buy using their past experience in previous purchases is enough to observe prices drop, market efficiency losses, and systematic underestimation of actual product quality. This alternative explanation is shown to be valid for a very wide range of learning rules and in various market contexts.

1

Introduction

In this paper we investigate the impact of product quality variability in markets. The discussion will be based on several computer agent-based market simulations. The main features of the artificial market we are considering are: (a) There is only one type of product, whose quality follows a specific, predetermined, probability distribution, (b) buyers in the market estimate the quality of the product following simple rules based on their own experience, and (c) a buyer's product valuation (or reservation price) is proportional to the quality she expects to get. With these conditions, it is shown in this paper that, when {symmetric) quality variability is introduced or increased, market prices fall down and the average quality of the product expected by buyers drops below the real average quality of the product in the market. Consequently, market efficiency* is also reduced. These results emerge from the market interactions among individuals trying to learn the average quality of the product they are repeatedly buying, and are robust * Market efficiency is a measure of social welfare. It is the sum of sellers' surplus and buyers' surplus. When a transaction between a buyer and a seller is made, the seller's surplus is the difference between the price of the item (income) and the item's marginal cost; the buyer's surplus is the difference between the maximum price that she would have paid for the item (reservation price, or marginal value) and the price actually paid (cost).

204

Segismundo S. Izquierdo et al.

to changes in the learning rule and market design. No risk-aversion rule is implemented. Although both the real quality distribution and the buyers' learning rules are unbiased, when several agents get together and interact through a market, an asymmetric-bending effect on market demand emerges, which causes prices to fall down and reduces the efficiency of the market. Our results suggest an alternative theory to explain the classical and welldocumented problem of market failure caused by asymmetric information (Akerlof 1970). The classical explanation assumes a pressing supply of low-quality items that lowers the average quality of the product in the market ("adverse selection"). However, in our model, even letting the average product quality remain constant, a market failure, caused by the combination of quality variability plus individual learning, emerges. As to the practical implications, our theory could explain the success of warranties and quality variability reduction policies in markets in which asymmetric information does not seem to be a clear issue. However, as is often the case, there can be alternative theories that could also explain the same aggregate effects, and this theory is still to be tested empirically.

2

Perfect Competition and Quality Uncertainty

The classical model of perfect competition has proven to be a useful framework for a large number of real markets. The model establishes that, under certain hypotheses, free trade will produce an equilibrium market price and a traded volume at the crossing point of supply and demand. From an efficiency point of view, this is also the optimal production level and production distribution that a theoretical central planner with all the information should choose. The hypotheses of perfect competition are quite restrictive: product homogeneity, multiple buyers and sellers, perfect information and profit-maximizing agents. However, for many common market institutions, the model has proven to be robust to deviations in several of these hypotheses. For instance, up to certain limits, the efficiency of double auctions has proven to be robust when the hypotheses of perfect information (Smith 1962), large number of players, and players' cognitive capabilities (Code and Sunder 1993; Bergstrom 2003; Duffy 2005) are relaxed (the last two references also discuss some other market institutions). The hypothesis of product homogeneity is strongly related to the problem of quality distribution. Note that, in general, the actual quality of an item is a random variable, and it can only be observed when the product is used or consumed (think, for instance, of any product with a variable life service, like a light bulb). However, it is often the case that a probability distribution for the quality of the product may be known or estimated a priori. We will assume that a product is homogeneous if any two items of that product have the same quality probability distribution. Breaches in the hypothesis of product homogeneity combined with asymmetric information have been the subject of intense economic research: "The Market for Lemons: Quality Uncertainty and the Market Mechanism" (Akerlof 1970), is

Market Failure Caused by Quality Uncertainty

205

thought to be (by the 2001 Nobel prize commission) the single most important study in the literature on economics of information (The Royal Swedish Academy of Sciences 2001). It provides a fruitful framework for the analysis of many real markets, like those of insurance policies and used cars. However, besides product heterogeneity, this framework relays heavily on the existence of some reliable quality indicators that can be observed by sellers, but not by buyers -i.e. asymmetric information. With less restrictive (more general) conditions, our study provides a complementary explanation for market failure when quality uncertainty is present. The classical explanation of market failure caused by asymmetric information rests on the phenomenon of "adverse selection", which can be explained along these lines: there are high-quality and low-quality items, but buyers can not distinguish quality when purchasing, so all items are sold at the same price; for a seller, a low quality-item is more profitable than a high-quality one, so the market is flooded with low-quality items; the reduction of the average quality lowers quality expectations, demand and prices, making high-quality items even unprofitable; this 'diminishing quality D diminishing price' vicious cycle can go on to the extent of destroying the market. In contrast to this explanation, which requires (and mixes) the effects of asymmetric information and product heterogeneity, we do not impose any sort of asymmetry or a breach in the hypothesis of product homogeneity (though we do assume that there is a quality probability distribution). In this setup, our paper discusses the effects of quality variability on the market, assuming that buyers follow simple learning rules to estimate the quality of a product.

3

Design of the Experiments

As a guiding line for our discussion we will present the results of several computer simulated markets with individuals who form expectations on the quality of the product using their own past experience. The aggregate effect can be more easily understood using a simplified model in which a market institution produces a price and a traded quantity at the crossing point of supply and demand (like a Walrasian tatonnement), though our results are also shown to be robust to other market mechanisms. The main features of our simplified model are: • Buyers and sellers trade in sessions. In each session, each buyer can buy at most one product ('single-item demand'). . • The quality q of every produced product follows a symmetric distribution centred on 1. • Supply is a linear function and does not vary from one session to the next. • Demand is formed by summing up the individual reservation prices of buyers. Initial reservation prices are such that the initial demand is linear. Buyers' reservation price then varies according to their quality expectations. The reserva-

206

Segismundo S. Izquierdo et al.

tion price of buyer / in session n is equal to her initial reservation price multiplied by her current estimated quality (^,. „) for the product. • In each session, the market is centrally cleared at the crossing point of supply and demand, and all the buyers who have bought a product update their quality expectations according to their experience with the product just bought. In particular, buyers (indexed in i) use the following updating rule: ^M.i =(l-^)-^„„+^-? (3-1) with an initial estimate q. ^ = E(^) = 1. Note that X (learning rate) measures the responsiveness of buyers' quality estimates to new data. Note also that this is an individual learning rule (Vriend 2000), as each buyer's quality estimates are based only upon her own past experience. This simple model is enough to illustrate a market failure caused by quality uncertainty. For now, we will not discuss whether the agents' learning rules in our model are realistic or not, as our results will be shown later to be valid for much wider class of learning rules. As a matter of fact, following the Keep-It-Simple principle (Axelrod 1997), we restrict the use of our model to its capacity to illustrate and give insight into the global implications of individual decision rules, and we believe that this is performed best by using simple, tractable and robust models.

4

Results and Discussion

Using the simplified model described on the previous section, consider a market with an initial situation (/ = 0) like the one shown on Figure 1, which corresponds to the following parameterisation: there are 200 buyers, and buyer / (i = 1, 2, 3, ..., 200) has initial reservation price equal to /; thus the initial demand is such that at price/? ip < 200), the number of products demanded is the integer part of (201 p). In each session, the number of items offered at price p (supply function) is the integer part of p. The market price is taken to be the average ask-bid price for the last traded unit (crossing point of supply and demand). Therefore reference conditions (i.e. no quality variability) are: price = 100.5, traded volume = 100. These conditions would be indefinitely maintained if there was no product variability, or if the learning rate X was equal to zero. We now introduce quality variability and individual quality learning. Surprisingly, in our model with symmetric quality variability, inefficient market dynamics emerge, prices drop below reference conditions, and buyers systematically underestimate the actual quality of the product. We investigated the robustness of our results using different quality symmetric distributions (uniform, triangular, trimmed normal), obtaining the same patterns for all of them. Robustness to the learning rule will be discussed later.

Market Failure Caused by Quality Uncertainty

207

Figure 1 shows some results corresponding to a uniform quality distribution q U[0, 2] and a learning rate >- = 0.3, with every buyer's initial quality estimate q. ^ equal to 1. The degeneration of the demand function can be clearly seen in the first periods. After a certain number of periods the demand function seems rather stable and the results of consecutive trading sessions look very similar. However, as we will show later, with these conditions and given enough time, no trading would eventually take place.

350 r

Price

300 250

Supply ^ ^

Demand

150

"x\.^^^^^t=o

100

M,

^ . ^

tVio^^x^

[\^J^000 " t = 4000

0

20

40

60

—

80

100

120

140

160

180

200

Quantity

Fig. 1. Effects of quality learning on demand. The quality distribution function for this graph is a uniform distribution U[0, 2]. The initial linear demand (/ = 0) is represented, together with the demand functions after 10, 2000, and 4000 trading periods, when changes occur very slowly

208

Segismundo S. Izquierdo et al.

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500 SeSSion

Trading

Fig. 2. Effects of quality variability on price level (top), traded volume (middle) and mean expected quality (bottom). The reference situation (w^ithout quality variability) is a price level of 100.5, a traded volume of 100 and an expected quality of 1 x10 Total surplus

(j/^pli^^^^'il^^^^^^ 50

100

150

200

250

300

350

400

450

500

10000 Duyers surplus 5000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

Trading 500^ session

6000

Fig. 3. Effects of quality variability on total surplus (top), buyers surplus (middle) and sellers surplus (bottom). The reference situation (w^ithout quality variability) is a total surplus of 10,000: 5,000 for buyers and 5,000 for sellers

Market Failure Caused by Quality Uncertainty

209

Figure 2 shows the corresponding evolution of prices, traded volumes and average expected qualities. All these variables drop below reference conditions. In Figure 3 we can see the corresponding evolution of surpluses, and the loss of efficiency can be appreciated. Because of the drop in prices, the greatest loss of surplus (benefits) is for sellers. The general pattern (decreasing prices, decreasing expected quality, monotonously decreasing number of traded units, and loss of efficiency) is consistent throughout simulations, and also for different values of X and for the different quality distributions considered. A surprising result of our model is that the average quality of the product in the market is constant, but the average perceived quality is lower than the real one; in fact, most buyers perceive a lower quality than the real one, even though their learning rule is not biased. The reason for this apparent paradox is that those buyers who at some period(s) get bad products and reduce their quality expectations may stop buying the product, either temporarily (if prices go down enough) or even permanently, remaining forever with low expectations. These low expectations will never increase again if these individuals cannot buy the product once more: their confidence in the product would be undermined forever. The phenomenon is more clearly seen if we assume that supply is horizontal at a given price level, let us say 50 € (products are sold at 50 €, but not below). If by purchasing a series of "bad" products a buyer's reservation price can drop below 50 €, she will stop buying the product. In fact, any learning rule that allows quality expectations to occasionally drop below that threshold will have the same effect, finally leading to a market collapse. More generally, and for any learning rule, if supply is constant and those buyers who do not purchase the product do not change their reservation prices, then the number of traded units must always be monotonous decreasing (this proposition can be proved by noting that, in these conditions, the extramarginal part of the demand, i.e., that part to the right of the crossing point with supply, can never ascend). Furthermore, in these conditions, let MaCt be the highest marginal cost of all the units traded at session t, i.e., the marginal cost of the last unit traded. If the combination of learning rule and quality distribution is such that, on any session, there is a positive probability of (after a number of sessions) some reservation price(s) dropping below MaCt, then the market will eventually collapse. Note that this is the case for our simplified model, because every reserve price may fall below the minimum marginal cost.

5

Robustness to the Market Mechanism

In this section we check the robustness of our results to changes in the market mechanism by implementing a continuous double auction, as used in stock markets and many other economic institutions (Duffy 2005).

210

Segismundo S. Izquierdo et al.

Following (Gode and Sunder 1993), we selected zero-intelligence-constrained (ZI-C) agents: buyers who place random bids between 0 and their reservation prices, and sellers who place random asks between their (marginal) costs and a maximum limit (see the model details below). The convergence properties of double auctions with ZI-C agents have been analysed by (Cliff and Bruten 1997). They also propose some modifications of the ZI-C agents, like the ZIP agents, that could be argued to be better representatives of human behaviour (Das et al. 2001), but ZI-C agents provide a good starting point to test the effects of a given market measure on the evolution of a double auction, and simple modifications of these agents have been applied to understand phenomena like asset price bubbles and crashes as observed in laboratory market experiments (Duffy and Unver 2006). The main features of this continuous double auction model are: • There are 5 buyers and 5 sellers, sorted out in a queue alternating buyer with seller. They are sequentially prompted to post an offer (ask or bid). Starting with the first agent (buyer or seller), a round is completed when the last (10*) agent is prompted to post an offer. • Marginal costs for seller y are MaCj = 1 + 5 «,, where rij is the number of units sold by sellery during an auction. • The quality q of every produced product follows a symmetric distribution centred on 1. • Reserve prices for buyer / are Rt = q. (200 - 5 «,), where nt is the number of units purchased by buyer / during an auction and q. is her current estimated quality (with q. = 1 at the beginning of an auction). After every purchase, the quality estimate is updated using Eq. (3.1). • When prompted for an offer, sellery posts a random ask in the interval [MaCj, 200], and buyer / posts a random bid in the interval [0, Rf]. The best (highest) bid and the best (lowest) ask are centrally kept, and whenever there is a match (best bid higher than or equal to best ask), a transaction is made between the corresponding buyer and seller. The price of the item is the best bid or the best ask, whichever was first posted. • After a transaction is made, past offers are cleared and the auction goes on for the next item, continuing the round where it was stopped. An auction ends when there are 100 rounds on end without transactions (when, after a round with transactions, every agent is prompted again for 100 times and no new transaction is made). A surprising result of double auctions with ZI-C agents (who just make random offers in a profitable range) is the high economic efficiency that can be obtained, very close to the maximum achievable and also very close to the efficiency obtained in experiments with human subjects (Gode and Sunder 1993 ). Besides, under certain conditions (Cliff and Bruten 1997), the price level tends, as the auction evolves, to the level predicted by perfect competition (crossing point of supply and demand).

Market Failure Caused by Quality Uncertainty

211

In our double auction model, the reference conditions (crossing point of supply and demand with k =0) are 100 units traded and a price close to 100. Table 1 shows the final price (price of the last unit traded), number of units traded, and efficiency for several auctions before introducing quality variability. Table 2 shows the equivalent results when quality variability {q ~ U[0,2]) and learning (A = 0.5) were introduced. Table 1. Results of several auctions, without quality variability Auction number 1 Final price Units Efficiency

4

102.8

99.3

102.1

100

98

98

100%

99.8%

99.8%

5

5-105

105-305

Average

Average

104.1

96.8

100.7

100.4

101

97

99.6

99.7

100%

99.6%

99.8%

99.8%

Table 2. Results of several auctions, with quality variability q ~ U[0,2] and learning Q. = 0.5) Auction number 3

1 Final price Units Efficiency

5

73.9

87.7

88.7

83.6

78

82

88

88

77.5%

87.2%

99.4%

92.5%

5-105 Av-

105-305

erage

Average

68.6

83.3

84.2

72

79.8

80.8

78.3%

90.5%

91.4%

Table 3. Estimated quality after closing. Results of several auctions with quality variability q ~ U[0,2] and learning (X = 0.5) Auction number Buyer number

1

2

3

4

5

1-100 Average

1

0.41

0.82

0.30

0.95

0.46

0.59

2

0.78

0.63

0.59

0.33

0.87

0.58

3

0.71

0.40

0.29

0.60

0.37

0.53

4

1.50

0.75

0.55

0.35

0.52

0.62

5

0.80

1.06

0.43

0.55

0.35

0.58

Average

0.84

0.73

0.43

0.56

0.51

0.58

212

Segismundo S. Izquierdo et al.

In short, when quality variability and individual learning were introduced in this double auction environment with ZI-C agents, the general patterns were lower prices, lower perceived quality, less trading, and loss of efficiency. Besides, at the end of most auctions, every buyer underestimates the real quality of the traded product (Table 3). These results show that the market failure we are describing also happens in a continuous double auction environment: it is robust to changes in the market mechanism. Intuitively, in this double auction model most buyers finally underestimate the real quality because low expectations have an "attracting" power: the lower the quality expectation, the lower the probability of purchasing a new product and changing the quality expectation. When expectations get low, buying and, consequently, learning, are reduced or even stopped. In fact, it can be graphically checked that, in this model, the demand function tends to get depressed as an auction evolves.

5

Conclusions

We have studied the effect of quality uncertainty in markets using a simple agentbased computer model to gain insight into aggregate market behaviour. We have shown the emergence of a market failure that is not due to buyers' risk aversion, but to some buyers occasionally forming long-lasting low quality expectations, and even leaving the market. The results have proven to be robust to different market institutions, quality distributions and individual learning rules. Our findings offer a new viewpoint and a new explanation for some wellknown aggregate market effects associated to quality variability, as well as to the success of some managerial policies, like the use of warranties or the "zero defects" policy. These effects have traditionally been explained assuming large differences in average quality (product heterogeneity) and asymmetric information, leading to adverse selection. In contrast, our model only assumes learning in an environment of quality uncertainty, which would lead to the same aggregate effects. In order to select between these competing theories, some of the consequences of this model, like the expected difference between average perceived quality and average real quality, are falsifiable and can be tested empirically.

References Akerlof G A (1970) The Market for Lemons: Quality Uncertainty and the Market Mechanism. Quarterly Journal of Economics 84: 488-500 Axelrod R (1997) The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton: Princeton University Press. Bergstrom T (2003) Vernon Smith's Insomnia and the Dawn of Economics as Experimental Science. Scandinavian Journal of Economics 105: 181-205

Market Failure Caused by Quality Uncertainty

213

Cliff D, Bruten J (1997) Minimal-intelligence agents for bargaining behaviours in market environments. Hewlett-Packard Laboratories Technical Report HPL-97-91 Das R, Hanson JE, Kephart JO, Tesauro G (2001) Agent-Human Interactions in the Continuous Double Auction. Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI-01) Duffy J (2005) Agent-Based Models and Human Subject Experiments. To appear in: Handbook of Computational Economics, vol 2. Elsevier, Amsterdam Duffy J, Unver MU (2006, forthcoming) Asset Price Bubbles and Crashes With Near ZeroIntelligence Traders. Economic Theory 27: 537-563 Gode D K, Sunder S (1993) Allocative Efficiency of Markets with Zero-Intelligence Traders: Markets as a Partial Substitute for Individual Rationality. Journal of Political Economy 101: 119-137 The Royal Swedish Academy of Sciences (2001) Advanced information on the 2001 Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel. Available at http://nobelprize.org/economics/laureates/2001/ecoadv.pdfon May 20* 2005. Smith V L (1962) An Experimental Study of Competitive Market behavior. Journal of Political Economy 70: 111-137 Vriend N (2000) An Illustration of the Essential Difference Between Individual and Social Learning, and its Consequence for Computational Analyses. Journal of Economic Dynamics and Control 24: 1-19

Learning and the Price Dynamics of a Double-Auction Financial Market with Portfolio Traders Andrea Consiglio^, Valerio Lacagnina^, and Annalisa Russino^ ^ Dip. di Scienze Statistiche e Matematiche, Viale delle Scienze, Palermo, Italy. [email protected] ^ Dip. di Scienze Statistiche e Matematiche, Viale delle Scienze, Palermo, Italy. [email protected] ^ Dip. di Scienze Statistiche e Matematiche, Viale delle Scienze, Palermo, Italy. [email protected]

1 Introduction We study the dynamics of price adjustments in a market where portfolio traders with bounded rationality and limited resources interact through a continuous, electronic open book. The present work extends the model developed in [7] introducing endogenous target individual portfolio holdings. In [7] we analyzed the impact on price changes of the trading mechanism modelling an economy populated by agents homogeneous in terms of their trading strategy. Each agent traded to reach an exogenously assigned target portfolio. We showed that the institutional setting of a double-auction market is sufficient to generate nonnormal univariate marginal distributions of assets' returns and temporal patterns resembling those observed in real markets (such as serial dependence in volatility and in trading volume). In this paper we introduce a scenario optimization model to determine target portfolio allocations. Each agent, given his view about the joint distribution of returns, his initial level of wealth, and a target growth rate to reach within his investment horizon, must choose a portfolio that allows him to reach the desired growth rate. Using a set of scenarios extracted from the joint distribution of returns, the optimization model maximizes the upside movements of the portfolio penalizing the shortfalls below the target. The market trading activity depends on the heterogeneity of agents' beliefs. We allow agents to hold arbitrary priors about the univariate marginal distribution of returns, and we make agents update those distribution using past realized market prices. We concentrate our attention on analyzing the impact of the agents learning process about the marginal returns distributions assuming that agents have a constant common view of the assets' association structure. They correctly apply a

216

Andrea Consiglio et al.

copula function to generate the joint distribution of returns to be used to determine the optimal portfolio allocations. Recently, a growing literature has investigated the consequences of learning about the returns process"^. That is, there has been an increasing interest in analyzing what are the implications of relaxing the assumption that agents hold correct expectations. In particular, it has been posed the fundamental question of understanding if t3^ical asset-pricing anomalies (like returns predictability, and excess volatility) can be generated by a learning process about the underlying economy. The artificial market model that we develop in this paper represents a controlled setting that allows to study in details the interactions between the dynamics of the learning process and the evolution of asset prices. We do not assume a specific asset pricing model. Agents, given their view about the joint distribution of asset returns, choose the portfolio allocation strategy that maximize their objective function. Traders interact directly through the order book. Prices, at each instant in time, are not market clearing prices, but they change continuously as the result of the sequential arrival of investors. We focus on the price discovery process, that is we study the process with which the market adjusts over time to converge towards a long run equilibrium. Our aim is to highlight the channels through which the learning process and the market microstructure affects the dynamics of asset prices. In this first draft, we define the structure of the model and we report some preliminary results showing how the price dynamics is affected by the evolution of the optimal asset allocations. The paper is structured as follows. Section 2 describes the market structure. Section 3 introduces the assumptions we make in terms of agents' behavior. Section 4 presents the calibration used for our simulations and discusses the preliminary results obtained.

2 The Market Setting We consider an economy with M agents and N risky assets. The market works as a double-auction automated system. Agents, trading to reach their own target portfolio, enter the market sequentially. At each time step k within a trading day t, we randomly extract one agent to enter the market^. The selected agent will enter the market, and he will post his orders, if Pi (E) is greater than a random number drawn from a uniform distribution over the [0,1] interval. The probability Pi{E) is an increasing function of the total imbalance between the target and the current portfolio.

PiiE) = f{Ai), and, N

' [4], [1], [5], [13], [10] ^ Agents are sampled with replacement.

(1)

Learning and the Price Dynamics

217

where h*j{t,t + r ) is the agent's optimal target allocation for asset j , x\j{k) represents the agent's current holding in asset j , Pj{k) is the current price for asset j , and W/(/c) is the agent's total wealth given current prices and agent's holdings. The activation function Pi (E) reflects the urgency of trading for the candidate agent. Agents are more impatient to trade, the more distant is their current wealth allocation from their target portfolio^. Correspondingly, the filtering device Pi{E) makes the effective probability of entering the market dependent on portfolio's imbalance. When a trader enters the market he faces an exchange book with orders to buy and to sell. Agents can trade immediately at the current quotes, placing market orders, or they can submit limit orders that are stored in the exchange book and will be executed if matching orders arrive before the end of the trading day. Limit orders will be executed using first price priority and then time precedence. At each moment in time during the day, the exchange book, divided in a buy side and a sell side, shows all the orders that have been issued up to that time and that have not found a matching order. For each order, the order size, the limit price, and the posting time are reported. The limit price is the maximum price that a buyer is willing to pay to purchase the listed quantity for a buy order, and the minimum price that a seller is willing to accept to sell the signed quantity for a sell order. Limit order prices are ordered from highest to the lowest on the buy side, and from the lowest to the highest on the sell side. Prices move in discrete steps, and, during each trading day, the minimum tick size depends on the daily opening price. At the end of the trading day all orders are cancelled. The spot price at each time step k is either the last transaction price or the last midquote, if a change in the quotes occurred.

3 The Agents' Behavior Agents' behavior is specified in terms of order flow strategy (number of units to buy or to sell) and order submission strategy (type of order to submit, market or limit order). Agents trade to rebalance their portfolio. That is, at each moment in time they trade to adjust their portfolio according to their optimal target allocations. At time step k during trading day t, the number of units of the j - t h asset that the i-th agent is willing to trade is given by.

htj{t,t + QU^)

T)Wl{k)-xlj{k)PJ{k)

(2)

where [-J denotes the integer part. If qj^ (k) > 0, the trader issues a buy order; if qlj{k) < 0, the trader issues a sell order. The target allocations h*j{t,t + r ) . * The relation between the urgency to trade and the size of portfolio's imbalance can be explained as the result of the existence of fixed direct or indirect trading costs (such as a fixed fee that must be paid to access the market or the costs associated with the effort (and time) needed to monitor and to adjusts portfolio's deviations.

218

Andrea Consiglio et al.

where r represents the length of the investment horizon, are the optimal solutions of the agent's portfolio choice problem. Agents are cash constrained. In particular, borrowing and short-selling are not allowed and the agent's orders can be submitted only if the money needed (MN) is not greater than the money available (MA): N

N

MNi{k) = ^li^sy{k)q^ik)Aj{k)

+J2l{LB}ik) [Qij{k)-q'M

j=i

PjAk)

j=i

(3) N

MMk) = Ci{k)^J2l{^,y{k)q'j{k)Bj{k),'^

(4)

where / | ^ | (k) is an indicator variable denoting for each risky asset j if the agent wants to issue an order; qj (k) is the minimum between the quantity that the agents wants to trade at current prices, Qij (k), and the quantity available at the current quote, Qj{k); and Ci{k) is the cash available to the i-th agent. The event A can be a market order to buy (MB), a limit order to buy (LB), or a market order to sell (MS). If MNi{k) < MAi{k), then all the orders that the agent wants to issue will be submitted. If MNi{k) > MAi{k), then for each asset j the number of units to trade is scaled down until MNi{k) = MAi{k). The quantity adjustment keeps constant, with respect to the total, the percentage of money to allocate in each asset^. The adjustments imposed by the budget constraint are performed giving priority to the submission of market orders to buy. Only if some money remains available after all market orders to buy have been processed, the procedure to check for the availability of money for submitting the desired limit orders to buy starts. Otherwise, the limit orders to buy are all cancelled. The order submission criterion is exogenously specified. That is, we assume that traders want to satisfy their trading needs as soon as possible, and thus they will submit a market order at the current quote for the quantity they need to trade. Limit orders are used only if for some j the corresponding qjj (k) is greater than the quantity available at the current quote. In this case the agent places a market order for the quantity available, and for the residual quantity, given by qjj (k) — Q*- (/c), he will submit a limit order. The associated limit price will be such that the order will be first on the appropriate side of the book, so we have that, Pl,{k) =Bj(fc) + €*

Pl{k)=A]ik)-eK where Bj{k) and A*j{k) are respectively the best bid and the best ask in the order book, and e* is the minimum tick size for trading day t. When there are no orders on ^ To simplify the notation we drop the superscript indicating the trading day ^ For each asset, the number of shares to purchase is computed as the integer part of the product of the original number of shares and the ratio between the money available and the money needed. The residual generated by the rounding procedure is automatically converted in cash.

Learning and the Price Dynamics

219

the relevant side of the book to match with, the agent will place directly a limit order for the whole quantity needed, qjj (k), at a price that will make him first on the book.

3.1 The Learning Process We assume that investors have imperfect information about the joint distribution of returns, and that they must learn about the unknown returns generating process using the available information. In particular, we allow agents to hold arbitrary marginal prior densities for the assets returns. We use a multinomial model to represent agents' views about the univariate distributions of returns. For each asset, we divide the support of the returns distribution in C classes, representing the number of possible events at each trial. At each instant in time, one of the C possible events will occur. Assuming that the probability of observing a return falling in one of the C classes, Oc, remains constant from time to time and that the Oc's are independent (that is, assuming that the process is stationary and independent), the prior marginal returns distribution of each asset can be modelled as a Dirichlet with parameters ( a i , . . . , a c ) . Thus, we assign to the agents populating our economy arbitrary prior densities given by:

^''^^^~ naa)...na,c)^'

•"^^

^^^

where Oi,...,Oc > 0; Ylc=i ^c = 1, ^ = 1 , . . . , M , and j = 1 , . . . , AT. Agents will use the history of observed market returns to update their beliefs in a bayesian fashion. Letting Vc be the number of returns observed, for the j-th asset, in class c during the time period between two successive updating days, the posterior distribution of the i-th agent for the returns of asset j will be Dirichlet with parameters {{an + fji)) • • • 5 {<^ic + ^ic))- To determine the optimal portfolio composition, agents should know the joint probability distribution of assets returns. We assume that agents share a common constant view of the securities association structure, and that they correctly use a copula function^ to generate the AT-variate returns distributionfi*omtheir arbitrary set of N univariate distributions. The main advantage of using copulas to model a joint distribution is that the univariate marginals and the association structure can be modelled separately and then combined to get the joint distribution. In our context, introducing a copula function allows us to concentrate our attention on modelling agents' heterogeneity in terms of arbitrary prior univariate marginal distributions of asset returns. We use The Gaussian copula^^ to model

^ See [12]. ^° Recently [11], using a sample of 22 stocks with high market capitalization traded on the NYSE, have tested the hypothesis that the association between stocks can be modelled by the Gaussian copula. They found that the "Gaussian copula provides the most parsimonious description of the dependence between stock returns, apart from crisis periods". The Gaussian copula underestimates the tail dependence, but, as stressed by the authors, the estimates of tail dependence obtained under alternative assumptions, like the Student copula, are not substantially improved.

220

Andrea Consiglio et al.

the dependence structure between the risky asset. Given their own marginal univariate returns distributions and the assigned copula, agents extract from the multivariate distribution of returns a number S of scenarios. Each scenario specifies a return for each of the N risky assets for all the time periods in the investment horizon of the agent. Every scenario represents a possible future realization of returns, for the A^ assets, given the agent's joint probability distribution. Agents use the S extracted scenarios to determine the optimal composition of their portfolio. 3.2 The Portfolio Model

We use a prospect-type utility function [9] to model the portfoUo choice problem. Each investor has an initial level of wealth and a target growth rate to reach within his investment horizon. The investor must determine an asset allocation strategy so that the portfolio growth rate will be sufficient to reach the target. We model the utility function in terms of deviations, measured at regular intervals, from a specified target, and we assume that investors are more sensitive to downside movements. Our approach is inspired to the descriptive models about investors' choices followed by [3, 2]. The target portfolio holdings are determined using the scenario optimization model developed in [6]. Let ut and dt be two random variables which define the upsides and downsides in each time period t = 1,2,..., T. Given a capitalization factor Mt^\ the random variable DT which accounts for the final deficit is given by T

The upsides collected during the planning horizon are added up to define the random variable of the final surplus T

UT = ^MtUt.

(7)

t=i

The investor will determine his portfolio by solving the following multi-objective programming model Maximize

E[UT] - A E[DT]

(8)

hen,

(9)

h

s.t. where A > 0 is the risk aversion parameter. By representing the probability distributions of the asset returns through a set of scenarios, the reward and the risk function formulated here turn out to be linear^^. ^^ In general, Mt is defined as e-^* ^(*)'^*, where r{s) is the stochastic process of the spot interest rate. In our artificial market model the spot interest rate is assumed constant over time and equal to zero. ^^ See [6] for a more general derivation of the model, and [8] for useful insights about bilinear utility functions for portfolio management.

Learning and the Price Dynamics

221

4 Preliminary Results Our first objective is to understand how the price dynamics is affected by the agents' learning process. We concentrate our attention on the asset-pricing implications of introducing heterogeneity in agents' prior beliefs. Thus, we initially define a setting where agents are equal in terms of endowments, trading strategies, investment horizon, risk profile, institutional constraints, and type of information used to update prior beliefs. The only source of heterogeneity is given by the different set of prior univariate marginal distributions of returns that we assign to each group of agents. We run our simulations with a population of M = 6000 potentially active traders, T = 2400 trading days, and N = 3 risky assets. Each trading day is divided in K = 360 time steps corresponding to a trading day of six hours, assuming a time step k equal to one minute. Every agent gets an initial endowment in each of the N stocks of our economy of 50 shares, and a cash endowment of Ci = € 1000. Initial prices are set equal to =€100. Agents are divided in G = 6 equally sized groups. All the agents in a group share the same view about the joint distribution of returns. In particular, we create agents' heterogeneity assigning to each group of agents a different set of prior univariate marginal distributions of returns. We divide the population in pessimists and optimists. That is, we assume that the univariate marginal priors reflect agents expectations about market performance. A group of pessimist agents will have a set of prior distributions, one for each of the N risky assets, with a mode close to zero, while a group of optimists will have priors with modes shifted towards the upper end of the returns interval. All agents have the same type of objective function, the same risk measure A = 5, and the same investment horizon of H = 120 days, corresponding to six months assuming a trading week of five days. The target portfolio return, gi, is different between pessimists and optimists agents: gi = 1.5% for pessimists and gi = 3% for agents with optimistic views. To maintain an active market over time, we simulate a setting where agents have not entered the market at the same time. That is, we assume that every / = 20 days, one group reaches the end of his investment horizon, and all the agents in the group update their view about the joint distribution of returns using the history of observed returns. The posterior distribution of returns is then used to determine new optimal target allocations hlj{t,t + r ) . To create a history of prices that agents can use to update their priors, for the first 120 days, we run our simulations, using randomly assigned target allocation vectors. That is, during the initial period each group of agents gets target allocation vectors sampled from a Dirichlet(l,... 1;1). To maintain, over the entire length of the simulations, a market structure homogeneous in terms of trading activity, during the initial period, we randomly extract one group of agents every / = 20 days, and we assign to all the agents in the selected group new random target allocations. The minimum tick size is 1% of the opening daily price^^.

^^ See [7] for a complete description used in the simulation. In this paper we report only the parameters that we have changed.

222

Andrea Consiglio et al.

4.1 Price Dynamics Fig. 1 and 2 display the dynamics of daily prices generated using exogenously assigned target allocations, and optimal target allocations determined on the basis of the agents' perception about the joint distribution of retums^"^. In the upper (botttom) panel of fig. 2 we plot the price series corresponding to the case where agents assume that there exists a negative (positive) association structure among the A^" risky assets. The daily price series report for each day the last midquote registered during the trading day.

Fig. 1. Daily time series of midquote prices generated under randomly assigned target allocations Comparing Fig. 1 and Fig.2 we notice that the learning process changes significantly the price dynamics. Generally, under learning the price series present long run positive trends and become clearly non-stationary. The phenomenon is particularly evident in the case of learning with a negative association structure, where, additionally, the dynamics of price changes appears to be less volatile. In Table 1 we report the correlation among the three risky assets measured using the returns series generated under the three settings. In all cases asset returns are positively correlated, thus, at first inspection it seems that the assets' association structure assumed by the agents does not affect the sign of the realized correlation structure. What drives these results? What is the mechanics through which the assets' association structure affects portfolio choices?

^^ In all the simulations we maintain constant all the other parameters, and we use the same seed number

Learning and the Price Dynamics

223

8 250

Fig. 2. Daily time series of midquote prices for the different settings. In the upper panel we plot the series relative to the case of optimal allocations determined assuming negative correlations among the assets. In the bottom panel we show the series relative to the case of optimal allocations determined assuming positive correlations among the assets In Fig. 3 we report the optimal asset allocations of one group of agents, computed at one point in time, for different levels of the risk measure A under the assumption of a positive and a negative association structure. Recall that the agents' objective function is defined in terms of upside and downside movements of the portfolio returns with respect to a benchmark. When A is very small the optimal asset allocations are not influenced by the assumed association structure. Agents will invest all their wealth in the asset that, given their beliefs, has the higher expected returns. When A increases, agents become more sensible to the downside movements of the portfolio returns. When they assume that the assets

224

Andrea Consiglio et al.

Table 1. Correlation among the asset returns generated under randomly assigned target allocations (DIR), and under optimal target allocations assuming a positive (POS) and a negative (NEG) association structure. Model Pl2 p23 PlSl DIR 0.33 0.26 0.29 POS 0.17 0.44 0.22 NEG 0.25 0.25 0.20

Dhl*

Oh?

DhS*

• hi*

Dh?

DhS*

Fig. 3. Optimal asset allocations for one group of agents (conditional on the agents' subjective view about the joint distribution of returns) computed at one point in time, for different levels of A. In the upper panel we report the values relative to the case of optimal allocations determined assuming negative correlations among the assets. In the lower panel we plot the values corresponding to the case of optimal allocations determined assuming positive correlations among the assets are negatively correlated, they v^ill believe that they can reduce the risk of downside movements increasing diversification. The higher A, the higher is the proportion invested in the assets with lower expected returns, and the greater is the portfolio diversification. In particular, agents will invest more in the asset that has the lowest expected returns, but it is more negatively correlated with the others (asset 1). When agents assume that the assets are positively correlated, the optimal assets allocations will be mainly driven by the agents beliefs about the univariate marginal distribution of assets returns even for high levels of A. Agents have little scope for diversifying their portfolio, at high levels of A they will optimally choose to invest in asset 2 and 3, that are the assets with the lowest positive association. What are the consequences of the described agents' portfolio choices in term of market prices? As an example. Fig. 4 shows the behavior of the agents' optimal allocations for asset 3 across time. We plot the average and the standard deviation of the portfolio allocations across the six groups.

Learning and the Price Dynamics

225

Fig. 4. Average and standard deviation of the asset 3 optimal allocations for the six groups in the different settings. In the upper panel we report the values relative to the case of optimal allocations determined assuming negative correlations among the assets. In the lower panel we plot the values corresponding to the case of optimal allocations determined assuming positive correlations among the assets

Clearly, when agents assume a strong negative assets' association structure, they will tend to diversify as much as possible. Consequently asset's demands will, on average, tend to be the same for all the assets, and market prices will show a common trend. In addition, assets returns will tend to be equally positively correlated. Conversely, when agents assume that assets returns are strongly positively correlated, agents will diversify less and optimal asset allocations will be more dependent on agents beliefs about the marginal returns distributions. Assets' demands, and assets' prices, will be more volatile across agents and across time. Diversification will

226

Andrea Consiglio et al.

be mostly confined to the assets with the lowest positive correlation driving up the correlation of those assets. In our market, populated by portfolio traders, to generate negative correlations between assets returns we should have information shocks that change the agents' perceptions about the parameters of the univariate marginal returns distributions, and that induce agents to shift fi*om one subset of assets to another. But, once agents assume that assets returns are negatively correlated their optimal allocation strategy will generate positively correlated assets returns.

References 1. N. Barberis. Investing for the Long Run when Returns are Predictable. Journal of Finance, 55:225-264, 2000. 2. N. Barberis, M. Huang, and T. Santos. Prospect Theory and Asset Prices. Quarterly Journal of Economics, CXVI:l-52, 2001. 3. S. Benartzi and R.H. Thaler. Myopic Loss Aversion and the Equity Premium Puzzle. Quarterly Journal ofEconomics, 110(l):73-92, 1995. 4. P. Bossaerts. Learning-Induced Securities Price Volatility. Working Paper, California Institute of Technology, 1999. 5. M. Brennan and Xia Y. Stock Price Volatility and Equity Premium. Journal ofMonetary Economics, 47:249-283, 2001. 6. A. Consiglio, F. Cocco, and S.A. Zenios. www.Personal_Asset_Allocation. Interfaces, 34:287-302, 2004. 7. A. Consiglio, V. Lacagnina, and A. Russino. A Simulation Analysis of the Microstructure of an Order Driven Financial Market with Multiple Securities and Portfolio Choices. Quantitative Finance, 2005. Forthcoming. 8. R. Dembo and D. Rosen. The practice of portfolio replication, a practical overview of forward and inverse problems. Annals of Operations Research, 85(l):267-284, 1999. 9. K. Kahneman and A. Tversky. Prospec Theory: an Analisys of Decision under Risk. Econometrica, 47(2):263-291, 1979. 10. J. Lewellen and J. Shanken. Learning, Asset-Pricing Tests, and Market Efficiency. The Journal ofFinance, LVII: 1113-1145, 2002. 11. Y. Malevergne and D. Somette. Testing the Gaussian Copula Hypothesis for Financial Assets Dependencies. Quantitative Finance, 3:231-250, 2003. 12. R.B. Nelsen. An Introduction to Copulas. Springer-Verlag, New York, 1999. 13. Xia Y. Learning about Predictability: The Effect of Parameter Uncertainty on Dynamic Asset Allocation. The Journal of Finance, 56:205-246, 2001.

How Do the Differences Among Order Distributions Affect the Rate of Investment Returns and the Contract Rate Shingo Yamamoto^, Shihomi Wada^, and Toshiji Kawagoe^ ^ Future University-Hakodate, Department of Complex Systems. ^ Future University-Hakodate, Graduate School Systems Information Science. [email protected] ^ Future University-Hakodate, Department of Complex Systems, 116-2 Kameda-Nakanocho, Hakodate, Hokkaido, 041-8655, Japan. kawagoe@f u n . ac . j p Summary. Although the distribution of orders follows exponential distribution is reported in a market study, researchers rarely questioned about how differences among order distributions affect the rate of investment returns (RI) and contract rate (CR). In this study, we compared several order distributions (OD) using U-Mart (Unreal Market as an Artificial Research Testbed), an artificial market simulation system. We also controlled the type of time series of spot price data (PI) and the average of the distribution (m) in our experiment. We found that CR attains the maximum when m= 10, ODs were Constant and Pis were Down. The average RI attains maximum in most of the cases when m= 50, Pis were Up and ODs are Uniform and Constant. These results may suggest further study about that (1) which quantity management strategy is profitable for a trading agent and that (2) which kind of order distribution can improve market efficiency is needed.

1 Introduction According to Fukushima [3], it is reported that in JGB futures market, which is an order driven type market treated at the Tokyo Stock Exchange, the frequency of the order quantity follows an exponential distribution"^. In Fig.l, we show the data that Fukushima [3] reported. However, the reason why the order quantity follows exponential distribution is not explained. Moreover, it is also not clear how difference among order distributions affects the contract rate (CR) and the rate of investment returns (RI). To clarify these problems, the artificial market approach would be appropriate, because it is easy to ^ Fukushima [3] also points out that in the distribution of order quantity in the JGB futures market, a hundred million yen, which is a minimum unit, is most frequent, then a common multiple of five hundred million yen (10, 15, 20, 30 hundred million yen) is much more frequent.

228

Shingo Yamamoto et al. d.\J 1

1

1

1

1

-\

15 1 \ o c

-\

10 \\ L ]

CD CD

5 \\ r\ II 1

0

yyinJL.-__.n 10

20 30 Order size

40

Fig. 1. Fukushima's data [3] of order distribution in JGB futures market

control trade agent's strategies and market conditions (Deguchi et al. [2]). In this research, using an artificial market, we systematically investigate how the difference in trade agent (TA)'s order quantity determination algorithms is affect the trade results such as CR and RI. We used U-Mart (Unreal Market as an Artificial Research Testbed) developed by Nakajima et al. [8] in our experiment because the market institution currently dealt with in U-Mart is almost the same as that of the JGB futures market. In our experiment, except for order quantity determination algorithm, we used the standard agent set, which is provided by U-Mart system, to compare with the previous studies. In our experiment, we manipulated the following four factors; (1) time series of price index (PI), (2) the decision-maker (DM) of order quantity, (3) the distribution of order quantity (OD), and (4) average of order distributions (m). We analyzed how these factors affect RI and CR. The organization of the paper is as follows. In Section 2, experimental conditions and procedures are explained. In Section 3, the results of the experiment and their statistical analysis are shown. Conclusions and directions for the future research are given in the last section.

2 Experimental Design In this section, the outline of our experiment is given. First, we explain the details of TAs which were used in our study. Second, independent variables and dependent variables that we manipulated in our experiment are explained.

How Order Distributions Affect the RI and CR

229

2.1 Details of TAs We used U-Mart Version 2.0 (Nakajima et al. [8]) in our experiment. We modify the standard agent set (AS), which was provided by U-Mart system, by incorporating order quantity determination algorithms that will be explained later. As we considered that the number of market participants in the standard AS is too small, we multiplied them by five, so that the total number of TAs becomes 100. Table 1 shows the proportion of TAs in our experiment^.

Table 1. The contents of agent set name of agent number of agents S-RAND 15 F-RAND 5 TREND 10 ANTI-T 10 15 S-RSI 5 F-RSI S-MA 15 F-MA 5 SF-SPR 10 DAY-BS 10 Total

5

100

S-RAND and F-RAND determine a kind of order (buy or sell). The limit prices are randomly determined based on spot and futures prices respectively. TREND and ANTI-T determine a kind of order according to the trend in the latest future prices. Both TREND and ANTI-T determine the limit price by normal distribution whose average is the latest future price. S-RSI and F-SRI determine the limit price by RSI of spot and future prices respectively. RSI is a technical index which judges a turning point the market price from movement of them. Its value is obtained by RSI = {U/U -\- D). U is the sum of price increments during the latest n days, obtained by U = Un^=i max(pn — pio,0). D is the sum of price decrements during the latest n days, obtained by D = Un^i max(pio — Pn,0). pn is the closing price at n days before. RSI is used for determining the timing for order. RSI < 0.3 means buying, while RSI > 0.7 means selling. S-MA and F-MA determine the limit price by the moving average (MA) of spot and futures price respectively. MA is an average of the latest n terms' prices and its value is given by MA(n) = (l/n)i:;j=ipn. SF-SPR determines a kind of order by the spread index SPR. SPR is obtained by SPR = (pf — PS)IPS' PS is the latest spot price, and pf is the latest futures prices. When SPR > E, SF-SPR makes a selling order, and when SPR < —E makes a buying order, where -E is a threshold value for making order. DAY-BS makes a selling order, with the latest future prices plus a fixed amount, and makes a buying order with the latest futures price plus a fixed amount. As for these technical analyses, see Murphy [6, 7], and Tanaka [9].

230

Shingo Yamamoto et al.

2.2 Independent Variables In the previous studies of artificial market, it is much common to investigate the effect of modifying the TA's trade strategies and/or the proportions of TAs in a market, or the effect of changing the market institutions, on RI, CR, and volatility (Deguchi et al. [2], Fukushima [3], Nakajima et al. [8]). However, to the best of our knowledge, there are few studies concerning with the order determination algorithm. So we investigated the effects of changing order quantity determination algorithms on RI and CR in this research. First, for the decision-maker (DM), we consider the following two cases; (1) TA itself is a DM, (2) a meta TA is a DM. When TA itself is a DM, the TA determines order quantity according to a given probability distribution. When a meta TA is a DM, the meta TA allots a constant order quantity to each TA according to a given probability distribution. For the order quantity distribution (OD), we compare exponential distribution^, which is observed in real-world market, uniform distribution^,which is provided by U-Mart standard AS, normal distribution^, and a constant. For the size of order quantity, according to Fukushima [3], we used following four sets of average m and variance cr^, (m, a^) = (10, 20), (20, 90), (30, 210), (50, 600)^ For the price information (PI), we used J30 price information, from 29 December, 1989 to 29 November, 1999, which is given by U-Mart system. In our experiment, we divided this Pis into following four patterns of time series; "Up", "Down", "Bound", and "Oscillation" as in U-Mart International Experiment (UMIE) tournament determined by UMIE2004 System Operational Committee [10]. Time series of these spot prices are shown in Fig. 2. Thus, our experimental condition can be expressed by four-factors (DM, OD, m, PI). Table 2 summarizes these experimental conditions.^^

^ Probability density function is f{x) = Xe~^^, average is 1/A, and variance is 1/A^. ^ Probability density function is f{x) = 1/(6 — a) (if a < x < b ) , f{x) = 0 (otherwise), average is 1/(6 — a), and variance is (6 — a)^/12 Probability density function is f{x) = J - e 2^^, average is fi, and variance is a ^ If OD = const., a^ = 0. If OD is uniform distribution, for letting (m, cr^) same as those in other distributions as possible, we set {m,a^) as follows; (m,cr^)=(10, 19), (20, 91), (30, 217), (50, 602) ^° In other word, our experiment can be seen the following multiple linear regression model with dummy variables to investigate the effect of four factors on RI and CR. (For the detail of multiple liner regression model, see Greene [4]). First, D"^, Dt, Dt DI, D^, D ? , D J , D^, D^, D^ are dummy variables. If D"^ = 1, TA itself is a DM, IfD"^ = 0, meta TA is a DM. If D? = 1, OD is exponential distribution. IfDi = 1, OD is uniform distribution. If D^ = 1, OD is normal distribution. If Df = 0{i = 1,2, 3), OD is constant. lfDi = hm= 10. lfD^ = l,m = 20. IfD^, m = 30. If Dt = 0{i = l,2,3), m = 50. Then, If D^ = 1, PI is UP If D^ = 1, PI is Down, If Dg = 1, PI is Bound. If Df = 0{i = 1, 2,3), PI is Oscillation.

How Order Distributions Affect the RI and CR

3800

PI = Up

Pl = Down

§3800 >^ .2 2800 Q. -•—•

231

1

1

h

(^1800 = 3800

120 120 240 PI = Bound

§3800

120 120 240 PI = Oscillation

.2 2800 o Q. CO

1800

120

120

240

240

Fig. 2. Time series of the spot prices Table 2. Four factors in our experiment Factor DM OD

IfiTA itself Ifi Expotential 3fi Normal m((7^) Ifi 10 (20) 3fi30(210) Time series of Ifi Up spot price (PI) 3fi Bound

2fimetaTA 2fi Uniform 4fi Constant 2fi20(90) 4fi50(600) 2fiDown 4fi Oscillation

It seems that we should run 2 x 4 x 4 x 4 = 128 different experiments as shown in Table 2. However, when the OD = 4 (Constant), each TA's order quantity is the same regardless of who is a DM. So, it is enough to run 112 experiments. We run computer simulations 50 times for each of 112 experimental conditions as in UMIE (UMIE2004 System Operational Committee [10]).

2.3 Dependent Variables In our experiment, by manipulating experimental conditions explained in the previous subsection, we analyzed how changing order quantity distribution affects RI and CR. So, RI and CR are dependent variables in our experiment. RI is defined as RI = {LOT — ^o)/^o where UJQ is the initial amount of fund and LOT is the amount of money at the final round T. CR is defined as CR — Nc/Nt, Thus, y = a-\- PV^ + Yfi^ihiDf + 6iD? + SiDf) is our regression model, where y =RI or y =CR, and at, j3i,ji,6i, and Si (i = 1,2,3)) are coefficients to be estimated.

232

Shingo Yamamoto et al.

where Nt is the number of orders made at the day t and A^c is the number of order quantity that is contracted at the day t.

3 Results 3.1 The Distribution of Order Quantity Figure 3 shows the distributions of order quantity in the market as a whole when DM =TA, OD = Exp., m = 20, PI = all ^^

11

21

31

41

51

61

71

Fig. 3. OD in the market as a whole when each TA's OD is Exponential

At a glance, in the cases of exponential distribution, the distributions of order quantity in the market as a whole corresponded to each TAs' order distributions. For the other distributions, same results are hold.

3.2 The Analysis of Rl and CR We show the maximum value of average CR for each (DM, OD, PI) in Table 3. Average CR attains the maximum in all cases when m = 10. When ODs were Constant and m = 30, average value of CR also attains the maximum value. We also show the maximum value of average CR for each (DM, m, PI) in Table 4. From this table, average CR likely attains the maximum when ODs were Constant (13 out of 32 cases). Further, we show in Table 5, which shows the maximum value of average CR for each combination of (DM, OD, m), for over half of the cases average CR attains maximum when Pis were Down (18 out of 32 cases). The shape of a graph is almost same when m takes other values.

How Order Distributions Affect the RI and CR Table 3. The maximum value of average CR DM OD PI meta Exp Up meta Exp Down meta Exp Bound meta Exp Osc. meta Uni Up meta Uni Down meta Uni Bound meta Uni Osc. meta Norm Up meta Norm Down meta Norm Bound meta Norm Osc. meta Cons Up meta Cons Down meta Cons Bound meta Cons Osc.

m

10 10 10 10 10 10 10 10 10 10 10 10 30&10 30&10 30&10 30&10

]Max. value 1DM

0.4139 0.4130 0.4045 0.4119 0.4051 0.4109 0.4131 0.4030 0.3992 0.3973 0.4086 0.4228 0.4111 0.4025 0.4006 0.4014

OD PI TA Exp Up TA Exp Down TA Exp Bound TA Exp Osc. TA Uni Up TA Uni Down TA Uni Bound TA Uni Osc. TA Norm Up TA Norm Down TA Norm Bound TA Norm Osc. TA Cons Up TA Cons Down TA Cons Bound 1 TA Cons Osc.

m Max. value 1 10 0.4075 10 0.4130 10 0.4110 10 0.4082 10 0.4097 10 0.4135 10 0.4090 10 0.4111 10 0.4084 10 0.4085 10 0.4075 10 0.4112 30&10 0.4111 30&10 0.4025 30&10 0.4006 30&10 0.4014

Table 4. The maximum value of average CR DM m PI meta 10 Down meta 20 Down meta 30 Down meta 50 Down meta 10 Up meta 20 Up meta 30 Up meta 50 Up meta 10 Osc. meta 20 Osc. meta 30 Osc. meta 50 Osc. meta 10 Bound meta 20 Bound meta 30 Bound meta 50 Bound

OD metax. value | DM m PI OD metax. value Exp 0.4130 TA 10 Down Uni 0.4135 Exp TA 20 Down Exp 0.4053 0.4050 Cons 0.4025 TA 30 Down Cons 0.4123 Exp 0.3908 TA 50 Down Cons 0.3981 1 Exp 0.4139 TA 10 Up Cons 0.4114 Exp 0.4046 TA 20 Up Uni 0.4016 Cons 0.4011 TA 30 Up Cons 0.4114 Norm 0.3920 0.3967 TA 50 Up Cons Norm 0.4228 TA 10 Osc. Norm 0.4112 Norm 0.4104 TA 20 Osc. Norm 0.4047 Cons 0.4014 TA 30 Osc. Cons 0.4107 Exp 0.3896 0.3964 TA 50 Osc. Cons Uni 0.4131 TA 10 Bound Exp 0.4110 Uni 0.4044 TA 20 Bound Exp 0.3991 Cons 0.4006 TA 30 Bound Cons 0.4102 Uni 0.3890 1 TA 50 Bound Cons 0.3954

233

234

Shingo Yamamoto et al. Table 5. The maximum value of average CR DM OD m PI ]Vlax. value 1DM OD m PI Max. value meta Exp 10 Up 0.4139 TA Exp 10 Down 0.4130 meta Exp 20 Down 0.4053 TA Exp 20 Down 0.4050 meta Exp 30 Down 0.3999 TA Exp 30 Down 0.3979 meta Exp 50 Down 0.3908 TA Exp 50 Down 0.3872 meta Uni 10 Bound 0.4131 TA Uni 10 Down 0.4135 meta Uni 20 Bound 0.4044 TA Uni 20 Down 0.4030 0.3977 TA Uni 30 Up 0.3930 meta Uni 30 Osc. 0.3876 meta Uni 50 Bound 0.3890 TA Uni 50 Up meta Norm 10 Osc. 0.4228 TA Norm 10 Osc. 0.4112 0.4104 TA Norm 20 Osc. 0.4047 meta Norm 20 Osc. meta Norm 30 Up 0.3969 TA Norm 30 Osc. 0.3972 meta Norm 50 Up 0.3920 TA Norm 50 Down 0.3827 meta Cons 10 Down 0.4025 TA Cons 10 Down 0.4123 meta Cons 20 Down 0.3884 TA Cons 20 Down 0.3981 meta Cons 30 Down 0.4025 TA Cons 30 Down 0.4123 meta Cons 50 Down 0.3884 JjA Cons 50 Down 0.3981 Table 6. The maximum value of average RI DM OD meta Exp meta Exp meta Exp meta Exp meta Uni meta Uni meta Uni meta Uni meta Norm meta Norm meta Norm meta Norm meta Cons meta Cons meta Cons meta Cons

PI Up Down Bound Osc. Up Down Bound Osc. Up Down Bound Osc. Up Down Bound Osc.

m Max. value 1DM OD PI 50 0.9917 TA Exp Up 50 0.9910 TA Exp Down 50 0.9908 TA Exp Bound 10 0.9899 TA Exp Osc. 50 0.9916 TA Uni Up 50 0.9909 TA Uni Down 50 0.9907 TA Uni Bound 10 0.9899 TA Uni Osc. 50 0.9918 TA Norm Up 50 0.9911 TA Norm Down 50 0.9909 TA Norm Bound 10 0.9899 TA Norm Osc. 50 0.9919 TA Cons Up 50 0.9911 TA Cons Down 50 0.9910 TA Cons Bound 10 0.9899 1 TA Cons Osc.

m Max. value 50 0.9919 50 0.9910 50 0.9910 10 0.9899 50 0.9919 50 0.9911 50 0.9911 10 0.9899 50 0.9918 50 0.9910 50 0.9910 10 0.9899 50 0.9919 50 0.9911 50 0.9910 10 0.9899

How Order Distributions Affect the RI and CR Table 7. The maximum value of average RI DM m PI meta 10 Up meta 10 Dov^m meta 10 Bound meta 10 Osc. meta 20 Up meta 20 Dowm meta 20 Bound meta 20 Osc. meta 30 Up meta 30 Dowm meta 30 Bound meta 30 Osc. meta 50 Up meta 50 Dowm meta 50 Bound meta 50 Osc.

OD ]Max. value 1DM m PI OD Max. value Exp 0.9906 TA 10 Up Uni 0.9906 Cons 0.9903 TA 10 Dowm Exp 0.9903 TA 10 Bound Cons 0.9899 Exp 0.9900 Uni 0.9899 TA 10 Osc. Uni 0.9899 Cons 0.9913 TA 20 Up Cons 0.9913 TA 20 Dowm Exp 0.9907 1 Cons 0.9906 Exp 0.9901 TA 20 Bound Uni 0.9901 Exp 0.9899 TA 20 Osc. Exp 0.9899 TA 30 Up Cons 0.9917 Cons 0.9917 Cons 0.9908 TA 30 Dowm Uni 0.9909 Uni 0.9905 TA 30 Bound Norm 0.9904 TA 30 Osc. Uni Exp 0.9898 0.9899 TA 50 Up Cons 0.9919 Cons 0.9919 Cons 0.9911 TA 50 Dowm Cons 0.9911 Cons 0.9908 TA 50 Bound Uni 0.9911 Uni 0.9898 1 TA 50 Osc. Uni 0.9898

Table 8. The maximum average of RI DM OD m PI Max. value 1DM OD m PI Max. value meta Exp 10 Up 0.9906 TA Exp 10 Up 0.9905 meta Exp 20 Up 0.9911 TA Exp 20 Up 0.9912 meta Exp 30 Up 0.9913 TA Exp 30 Up 0.9915 meta Exp 50 Up 0.9917 TA Exp 50 Up 0.9919 meta Uni 10 Up 0.9906 TA Uni 10 Up 0.9906 meta Uni 20 Up 0.9911 TA Uni 20 Up 0.9912 meta Uni 30 Up 0.9914 TA Uni 30 Up 0.9916 meta Uni 50 Up 0.9916 TA Uni 50 Up 0.9919 meta Norm 10 Up 0.9906 TA Norm 10 Up 0.9905 meta Norm 20 Up 0.9912 TA Norm 20 Up 0.9912 meta Norm 30 Up 0.9915 TA Norm 30 Up 0.9915 meta Norm 50 Up 0.9918 TA Norm 50 Up 0.9918 meta Cons 10 Up 0.9905 TA Cons 10 Up 0.9905 meta Cons 20 Up 0.9913 TA Cons 20 Up 0.9913 meta Cons 30 Up 0.9917 TA Cons 30 Up 0.9917 meta Cons 50 Up 0.9919 1 TA Cons 50Up 0.9919

235

236

Shingo Yamamoto et al.

In Table 6, we show the maximum value of average RI for each combination of (DM, OD, PI). The average RI attains maximum in most of the cases when m = 50 (24 out of 32 cases). On the other hand, when Pis were Oscillation, RIs attains maximum when m = 10. From Table 7, where the maximum values of average RI for each combination of (DM, m, PI) are shown. Uniform and Constant outperformed in most of cases (10 out of 32 are Uniform, and 12 out of 32 are Constant). Finally, we summarize the maximum value of average RI for each combination of (DM, OD, m) in Table 8. We found that the average RI attained maximum when Pis were Up.

4 Conclusions As Kawagoe and Wada [5] point out, there are few studies about strategy determining orders quantity in an artificial market. In this research, although strategies we used were so limited, we investigated how differences of order quantity strategy affected RI and CR. In our results, we showed that the order distribution in the market as a whole always followed the distribution of each TA's order distribution. CR attains the maximum in all cases average when m = 10, and when ODs were Constant, and where Pis were Down. The average RI attains maximum in most of the cases when m = 50. Uniform and Constant cases in most of cases. The average RI also attained and where ODs were maximum when Pis were Up. For designing a successfiil TA, making a TA which can avoid bankruptcy is most difficult. Martingale may be a well-known order quantity strategy (Tanaka [9]) . However, Martingale is not practical strategy because it lead us to the bankruptcy unless we had large enough money. In the previous studies in U-Mart system, two typical cases of bankruptcy were reported as follows; (1) Because of the lack of money at the end of a day, TA could not clear the payment for buying and selling orders, and (2) TA did not anticipate the large cost when it carried over positive or negative positions at the final days. Though current standard AS set provided by U-Mart system can avoid bankruptcy of type (1), it doesn't have any position management function. So, there is no guarantee to avoid bankruptcy of type (2). Thus, in the future research, it would be worthwhile to incorporate more sophisticated agent into our consideration, who makes orders when current expected benefit derived from those orders is greater than the present discounted value of loss caused by the change in position for that agent.

References 1. Izumi K. (2003). Artificial Market, Morikita publishing (in Japanese). 2. Deguchi H, Izumi K, Shiozawa K, Takayasu Y, Terano H, Sato T, Kita H (1997). "Discussion Socially and scholarly meanings for studying artificial market" The Book of Artificial Interigence Society, 12(1), p. 1-8 (in Japanese). 3. Fukushima Y(2001). "The way of order matchings and price change in the JGB fixtures market." The Agency of Money Market, Working Paper Series, 2001-J-1 (in Japanese).

How Order Distributions Affect the RI and CR

237

4. Greene WH (2002). Econometric Analysis, Prentice Hall. 5. Kawagoe T, Wada S (2005). "A New Approach to Controlling Artificial Market Conditions", mimeo. 6. Murphy JJ (1986). Technical Analysis of the Future Markets, New York Insutitute of Finance. 7. Murphy JJ (1997). Study Guide for Technical Analysis of the Future Markets, New York Insutitute of Finance. 8. Nakajima Y, Ono I, Sato H, Mori N, Kita H, Matsui H, Taniguchi K, Deguchi H, Terano T, Shiozawa Y (2004). "Introducing Virtual Futures Market System "U-Mart"," Experiments in Economic Sciences - New Approaches to Solving Real-world Problems. 9. Tanaka K(1998). A Big Book of Technical Analysis, Sigma Base Capital (in Japanese). 10. UMIE2004 System Operational Committee (2004). "Overview of UMIE2004 — December 1th, 2003 Version" (Appendix of U-Mart developer kit).

/HFWXUH1RWHVLQ(FRQRPLFV DQG0DWKHPDWLFDO6\VWHPV )RULQIRUPDWLRQDERXW9ROV²  SOHDVHFRQWDFW\RXUERRNVHOOHURU6SULQJHU9HUODJ 9RO1+0:LOVRQ(G &RPSXWHU$LGHG7UDQVLW 6FKHGXOLQJ;,SDJHV

9RO  - =KX 0RGXODU 3ULFLQJ RI 2SWLRQV ;  SDJHV

9RO  -5 7\UDQ 0RQH\ ,OOXVLRQ DQG 6WUDWHJLF &RPSOHPHQWDULW\DV&DXVHVRI0RQHWDU\1RQ1HXWUDOLW\ ;SDJHV

9RO')UDQ]HQ'HVLJQRI0DVWHU$JUHHPHQWVIRU27& 'HULYDWLYHV9,,,SDJHV

9RO6+HOEHU3HUIRUPDQFH$QDO\VLVRI)ORZ/LQHV ZLWK1RQ/LQHDU)ORZRI0DWHULDO,;SDJHV 9RO86FKZDOEH7KH&RUHRI(FRQRPLHVZLWK$V\P PHWULF,QIRUPDWLRQ,;SDJHV 9RO/.DDV'\QDPLF0DFURHFRQRPLFVZLWK,PSHUIHFW &RPSHWLWLRQ;,SDJHV 9RO  5 'HPHO )LVFDO 3ROLF\ 3XEOLF 'HEW DQG WKH 7HUP6WUXFWXUHRI,QWHUHVW5DWHV;SDJHV 9RO07KpUD57LFKDWVFKNH(GV ,OOSRVHG9DUL DWLRQDO3UREOHPVDQG5HJXODUL]DWLRQ7HFKQLTXHV9,,, SDJHV

9RO  , .RQQRY &RPELQHG 5HOD[DWLRQ 0HWKRGV IRU 9DULDWLRQDO,QHTXDOLWLHV;,SDJHV 9RO3:HL‰8QHPSOR\PHQWLQ2SHQ(FRQRPLHV;,, SDJHV 9RO-,QNPDQQ&RQGLWLRQDO0RPHQW(VWLPDWLRQRI 1RQOLQHDU(TXDWLRQ6\VWHPV9,,,SDJHV 9RO  0 5HXWWHU$ 0DFURHFRQRPLF 0RGHO RI :HVW *HUPDQ8QHPSOR\PHQW;SDJHV 9RO$&DVDMXV)RFDO3RLQWVLQ)UDPHG*DPHV;, SDJHV 9RO  ) 1DUGLQL 7HFKQLFDO 3URJUHVV DQG (FRQRPLF *URZWK;9,,SDJHV

9RO6+DUWPDQQ3URMHFW6FKHGXOLQJXQGHU/LPLWHG 5HVRXUFHV;,,SDJHV

9RO0)OHLVFKPDQQ4XDQWLWDWLYH0RGHOVIRU5HYHUVH /RJLVWLFV;,SDJHV

9RO  / Y 7KDGGHQ 0RQH\ ,QÁDWLRQ DQG &DSLWDO )RUPDWLRQ,;SDJHV

9RO1+DGMLVDYYDV-(0DUWtQH]/HJD]-33HQRW (GV *HQHUDOL]HG&RQYH[LW\DQG*HQHUDOL]HG0RQRWRQLF LW\,;SDJHV

9RO0*UD]LD6SHUDQ]D36WlKO\(GV 1HZ7UHQGV LQ'LVWULEXWLRQ/RJLVWLFV;SDJHV 9RO9+1JX\HQ--6WURGLRW37RVVLQJV(GV  2SWLPDWLRQ,;SDJHV

9RO$.LUPDQ-%=LPPHUPDQQ(GV (FRQRP LFVZLWK+HWHURJHQRXV,QWHUDFWLQJ$JHQWV9,,SDJHV 

9RO : % =KDQJ$7KHRU\ RI ,QWHUQDWLRQDO7UDGH ;,SDJHV

9RO3<0RL[(G 7KH0HDVXUHPHQWRI0DUNHW5LVN ;,SDJHV

9RO0.|QLJVWHLQ(TXLW\(IÀFLHQF\DQG(YROXWLRQDU\ 6WDELOLW\LQ%DUJDLQLQJ*DPHVZLWK-RLQW3URGXFWLRQ;,, SDJHV

9RO  6 9R‰ - 5 'DGXQD (GV  &RPSXWHU$LGHG 6FKHGXOLQJRI3XEOLF7UDQVSRUW;,SDJHV

9RO''*DWWL0*DOOHJDWL$.LUPDQ,QWHUDFWLRQ DQG0DUNHW6WUXFWXUH9,SDJHV

9RO  % 3 .HOOHUKDOV )LQDQFLDO 3ULFLQJ 0RGHOV LQ &RQWLQXRXV7LPHDQG.DOPDQ)LOWHULQJ;,9SDJHV 

9RO$*DUQDHY6HDUFK*DPHVDQG2WKHU$SSOLFDWLRQV RI*DPH7KHRU\9,,,SDJHV

9RO0.RNVDODQ6=LRQWV0XOWLSOH&ULWHULD'HFLVLRQ 0DNLQJLQWKH1HZ0LOOHQLXP;,,SDJHV

9RO01HXJDUW1RQOLQHDU/DERU0DUNHW'\QDPLFV ;SDJHV

9RO.1HXPDQQ&6FKZLQGW-=LPPHUPDQQ3URMHFW 6FKHGXOLQJZLWK7LPH:LQGRZVDQG6FDUFH5HVRXUFHV;, SDJHV

9RO<<+DLPHV5(6WHXHU(GV 5HVHDUFKDQG 3UDFWLFH LQ 0XOWLSOH &ULWHULD 'HFLVLRQ 0DNLQJ ;9,, SDJHV

9RO  ' +RUQXQJ ,QYHVWPHQW 5 ' DQG /RQJ5XQ *URZWK;9,SDJHV

9RO  % 6FKPROFN 2PPLWWHG 9DULDEOH 7HVWV DQG '\QDPLF6SHFLÀFDWLRQ;SDJHV

9RO$67DQJLDQ&RQVWUXFWLQJDQG$SSO\LQJ2EMHF WLYH)XQFWLRQV;,,SDJHV

9RO76WHJHU7UDQVLWLRQDO'\QDPLFVDQG(FRQRPLF *URZWKLQ'HYHORSLQJ&RXQWULHV9,,,SDJHV

9RO  0 .OSPDQQ 6WRFN 0DUNHW 2YHUUHDFWLRQ DQG )XQGDPHQWDO9DOXDWLRQ,;SDJHV

9RO  6 0LQQHU 6WUDWHJLF 6DIHW\ 6WRFNV LQ 6XSSO\ &KDLQV;,SDJHV

9RO:%=KDQJ$Q(FRQRPLF7KHRU\RI&LWLHV;, SDJHV

9RO0(KUJRWW0XOWLFULWHULD2SWLPL]DWLRQ9,,, SDJHV

9RO  . 0DUWL 6WRFKDVWLF 2SWLPL]DWLRQ 7HFKQLTXHV 9,,,SDJHV

9RO73KDQ+X\&RQVWUDLQW3URSDJDWLRQLQ)OH[LEOH 0DQXIDFWXULQJ,;SDJHV

9RO6:DQJ<;LD3RUWIROLRDQG$VVHW3ULFLQJ;,, SDJHV

9RO*+HLVLJ3ODQQLQJ6WDELOLW\LQ0DWHULDO5HTXLUH PHQWV3ODQQLQJ6\VWHP;,,SDJHV

9RO  + .UDIW 2SWLPDO 3RUWIROLRV ZLWK 6WRFKDVWLF ,QWHUHVW5DWHVDQG'HIDXOWDEOH$VVHWV;SDJHV

9RO%6FKPLG3ULFLQJ&UHGLW/LQNHG)LQDQFLDO,QVWUX PHQWV;SDJHV

9RO*\&KHQ;+XDQJ;
9RO+,0HLQKDUGW&RRSHUDWLYH'HFLVLRQ0DNLQJLQ &RPPRQ3RRO6LWXDWLRQV9,,,SDJHV

9RO-/LQJHQV8QLRQ:DJH%DUJDLQLQJDQG(FRQRPLF *URZWK;,,,SDJHV

9RO61DSHO%LODWHUDO%DUJDLQLQJ9,,,SDJHV 

9RO  & %HQNHUW 'HIDXOW 5LVN LQ %RQG DQG &UHGLW 'HULYDWLYHV0DUNHWV,;SDJHV

9RO $ .ORVH * 6SHUDQ]D / 19DQ:DVVHQKRYH (GV  4XDQWLWDWLYH$SSURDFKHV WR 'LVWULEXWLRQ /RJLVWLFV DQG6XSSO\&KDLQ0DQDJHPHQW;,,,SDJHV

9RO%)OHLVFKPDQQ$.ORVH'LVWULEXWLRQ/RJLVWLFV ;SDJHV

9RO  % *ODVHU (IÀFLHQF\ YHUVXV 6XVWDLQDELOLW\ LQ '\QDPLF'HFLVLRQ0DNLQJ,;SDJHV 9RO5&RZDQ1-RQDUG(GV +HWHURJHQRXV$JHQWV ,QWHUDFWLRQVDQG(FRQRPLF3HUIRUPDQFH;,9SDJHV 

9RO5+DIQHU6WRFKDVWLF,PSOLHG9RODWLOLW\;, SDJHV 9RO'4XDGW/RW6L]LQJDQG6FKHGXOLQJIRU)OH[LEOH )ORZ/LQHV;9,,,SDJHV 9RO  0 :LOGL 6LJQDO ([WUDFWLRQ ;,  SDJHV 

9RO  & 1HII &RUSRUDWH )LQDQFH ,QQRYDWLRQ DQG 6WUDWHJLF&RPSHWLWLRQ,;SDJHV

9RO'.XKQ*HQHUDOL]HG%RXQGVIRU&RQYH[0XOWL VWDJH6WRFKDVWLF3URJUDPV;,SDJHV

9RO:%=KDQJ$7KHRU\RI,QWHUUHJLRQDO'\QDPLFV ;,SDJHV

9RO*1.ULHJ.DQEDQ&RQWUROOHG0DQXIDFWXULQJ 6\VWHPV,;SDJHV

9RO0)U|OLFK3URJUDPPH(YDOXDWLRQDQG7UHDWPHQW &KRLVH9,,,SDJHV

9RO  7 /X[ 6 5HLW] ( 6DPDQLGRX 1RQOLQHDU '\QDPLFV DQG +HWHURJHQHRXV ,QWHUDFWLQJ$JHQWV ;,,, SDJHV

9RO  6 6SLQOHU &DSDFLW\ 5HVHUYDWLRQ IRU &DSLWDO ,QWHQVLYH7HFKQRORJLHV;9,SDJHV 9RO&)'DJDQ]R$7KHRU\RI6XSSO\&KDLQV9,,, SDJHV 9RO&(0HW],QIRUPDWLRQ'LVVHPLQDWLRQLQ&XUUHQF\ &ULVHV;,SDJHV 9RO  5 6WROOHW] 3HUIRUPDQFH$QDO\VLV DQG 2SWL PL]DWLRQRI,QERXQG&DOO&HQWHUV;SDJHV 9RO:.UDEV6:3LFNO$QDO\VLV&RQWUROODELOLW\ DQG2SWLPL]DWLRQRI7LPH'LVFUHWH6\VWHPVDQG'\QDPLFDO *DPHV;,,SDJHV 9RO5:DSOHU8QHPSOR\PHQW0DUNHW6WUXFWXUHDQG *URZWK;;9,,SDJHV 9RO0*DOOHJDWL$.LUPDQ00DUVLOL(GV 7KH &RPSOH[ '\QDPLFV RI (FRQRPLF ,QWHUDFWLRQ ;9  SDJHV 9RO.0DUWL<(UPROLHY*3ÁXJ(GV '\QDPLF 6WRFKDVWLF2SWLPL]DWLRQ9,,,SDJHV

9RO-/HVNRZ03XFKHW$Q\XO/)3XQ]R1HZ 7RROVRI(FRQRPLF'\QDPLFV;,;SDJHV 9RO  & 6XHULH 7LPH &RQWLQXLW\ LQ 'LVFUHWH 7LPH 0RGHOV;9,,,SDJHV 9RO%0|QFK6WUDWHJLF7UDGLQJLQ,OOLTXLG0DUNHWV ;,,,SDJHV 9RO5)RHOOPL&RQVXPSWLRQ6WUXFWXUHDQG0DFUR HFRQRPLFV,;SDJHV 9RO-:HQ]HOEXUJHU/HDUQLQJLQ(FRQRPLF6\VWHPV ZLWK([SHFWDWLRQV)HHGEDFNSODQQHG  9RO  5 %UDQ]HL ' 'LPLWURY 6 7LMV 0RGHOV LQ &RRSHUDWLYH*DPH7KHRU\9,,,SDJHV 9RO6%DUEDUR(TXLW\DQG(IÀFLHQF\&RQVLGHUDWLRQV RI3XEOLF+LJHU(GXFDWLRQ;,,SDJHV 9RO0)DOLYD0*=RLD7RSLFVLQ'\QDPLF0RGHO $QDO\VLV;SDJHV 9RO06FKXOPHULFK5HDO2SWLRQV9DOXDWLRQ;9, SDJHV

9RO  * 'XGHN &ROODERUDWLYH 3ODQQLQJ LQ 6XSSO\ &KDLQV;SDJHV

9RO$YRQ6FKHPGH,QGH[DQG6WDELOLW\LQ%LPDWUL[ *DPHV;SDJHV

9RO05XQNHO(QYLURQPHQWDODQG5HVRXUFH3ROLF\ IRU&RQVXPHU'XUDEOHV;SDJHV

9RO+%RE]LQ3ULQFLSOHVRI1HWZRUN(FRQRPLFV;; SDJHVSODQQHG

9RO;*DQGLEOHX[06HYDX[.6|UHQVHQ97·NLQGW (GV 0HWDKHXULVWLFVIRU0XOWLREMHFWLYH2SWLPLVDWLRQ,; SDJHV

9RO7/DQJHQEHUJ6WDQGDUGL]DWLRQDQG([SHFWDWLRQV ,;SDJHVSODQQHG

9RO5%UJJHPDQQ0RGHO5HGXFWLRQ0HWKRGVIRU 9HFWRU$XWRUHJUHVVLYH3URFHVVHV;SDJHV 9RO$(VVHU3ULFLQJLQ,Q &RPSOHWH0DUNHWV;, SDJHV

9RO$6HHJHU5HFHQW$GYDQFHVLQ2SWLPL]DWLRQ;9 SDJHVSODQQHG 9RO  3 0DWKLHX % %HDXÀOV 2 %UDQGRX\ (GV  $UWLÀFLDO(FRQRPLFV;,,,SDJHV

9RO6.RNRW7KH(FRQRPHWULFVRI6HTXHQWLDO7UDGH 0RGHOV;,SDJHV

9RO : /HPNH7HUP 6WUXFWXUH 0RGHOLQJ DQG (VWL PDWLRQLQD6WDWH6SDFH)UDPHZRUN,;SDJHV SODQQHG

9RO1+DXWVFK0RGHOOLQJ,UUHJXODUO\6SDFHG)LQDQ FLDO'DWD;,,SDJHV

9RO0*HQVHU$6WUXFWXUDO)UDPHZRUNIRUWKH3ULFLQJ RI&RUSRUDWH6HFXULWLHV;,;SDJHV