ACT-R based memory models of iterated prisoner’s dilemma

Abstract

Iterated Prisoner’s Dilemma game is an important tool for studying cooperation in social, biological and artificial environments. Various behavioral and neuroscientific experiments point to complex decision making and memory processes for human subjects. This thesis proposes four distinct memory models of Iterated Prisoner’s Dilemma game that are built upon ACT-R cognitive architecture. This work aims to overcome the shortcomings of a previous ACT-R based memory model by Lebiere et al. (2000), by providing extensive exploration of the parameter space and analysis of simulation results for all data points. Moreover, in contrast to previus work, this study introduces distinct declarative memory modules for each player. Third, model behavior is analyzed for the cases where it plays the game not only against itself, but against basic condional and unconditional strategies as well. Finally, by implementation of three new memory models for Iterated Prisoner’s Dilemma, this study intends to attain cooperation against teaching strategies. In decision making process, all memory models evaluate expected payoffs of possible moves according to the most likely outcome making that move. First model records game history in terms of frequency and recency of possible outcomes. Second memory model records outcome patterns that are experienced in the course of the game. Third model has a two step decision process where expected payoff is calculated according to both types of information about game history. Forth model employs an association mechanism between goal and declarative modules which enable the model to record outcome history in relation to contextual information that is kept in goal module. After parameter setting, simulations are conducted for the cases where each model plays iterated game with itself and with basic game strategies. According to simulation results, all models were successful in exploiting and defending against unconditional strategies. Against teaching strategies, although they presented learning behavior, all models except third model have failed to attain cooperative equilibrium. First, second and forth models have adapted their behavior to exploit learning Pavlovian strategy and forgiving teaching strategies. All models exhibited learning behavior against basic strategies. For the cases where each model plays the iterated game against itself, all models have successfully attained cooperation in a significant portion of the games. Apart from second model, all models exhibited a learning pattern consistent with human subjects. Moreover, similar to human subjects, simulated agents can be classified into teaching and learning groups according to their behavioral patterns.