The Intersection of Moral Theory and Cognitive Science

1. 1 Moral Theory Meets Cognitive Science How the Cognitive Sciences Can Transform Traditional Debates Stephen Stich Dept. of Philosophy & Center for Cognitive Science Rutgers University sstich@ruccs.rutgers.edu Jean Nicod Lectures 2007

2. 2 Lecture 4 Stephen Stich Daniel Kelly Joshua Knobe Debunking Moral Intuition A Hodgepodge of Multipurpose Kludges Jean Nicod Lectures 2007

3. 3 Lecture 4 Stephen Stich Joshua Knobe Daniel Kelly Debunking Moral Intuition A Hodgepodge of Multipurpose Kludges Jean Nicod Lectures 2007

4. 4 Introduction psychological mechanisms & processes intuitive moral judgment Philosophers and more recently cognitive scientists have offered many accounts of the psychological mechanisms & processes underlying intuitive moral judgment sometimes moral intuitions not to be trusted Moral philosophers have always insisted that sometimes the outputs of those processes peoples moral intuitions are not to be trusted when skepticism is warranted though they disagree about when skepticism is warranted

5. 5 Introduction newly emerging perspective Our goal in this talk is to sketch a newly emerging perspective on the mechanisms underlying moral intuition implications whether and when intuitions should be relied on and to explore its implications for the hotly debated issue of whether and when intuitions should be relied on

6. 6 Introduction Philosophers have typically assumed that those mechanisms were well designed for something Philosophers have typically assumed that those mechanisms were well designed for something But we now have reasons to think that many of theses mechanisms are not well designed for ANYTHING But we now have reasons to think that many of theses mechanisms are not well designed for ANYTHING

7. 7 Introduction Moral Psychology is a Kludge Moral Psychology is a Kludge A hodgepodge of multipurpose kludges! A hodgepodge of multipurpose kludges!

8. 8 Introduction Before explaining and defending this claim it will be useful to consider some of the reasons that philosophers both classic & contemporary have offered for discounting moral intuitions Before explaining and defending this claim it will be useful to consider some of the reasons that philosophers both classic & contemporary have offered for discounting moral intuitions

9. 9 Philosophical Background When should we be skeptical about moral intuitions? When should we be skeptical about moral intuitions? Moral Sense Ideal Observer The Moral Sense & Ideal Observer traditions Reflective Equilibrium Reflective Equilibrium Evolutionary arguments Evolutionary arguments debunking intuition

10. 10 Philosophical Background Moral Sense Ideal Observer The Moral Sense & Ideal Observer traditions intuitions are correct when made under ideal conditions Ideal observer theorists maintain that our moral intuitions are correct (or justified) when made under ideal conditions not ideal intuitions are not to be trusted When conditions are not ideal e.g. when we have false beliefs about relevant non-moral matters, or we are irrational our intuitions are not to be trusted

11. 11 Philosophical Background Moral Sense Ideal Observer The Moral Sense & Ideal Observer traditions Hutcheson moral sense the Author of Nature For Hutcheson an important precursor of this tradition moral judgments are the product of a moral sense implanted in us by the Author of Nature can be relied upon Thus it can be relied upon when doing its job properly mislead unfavorable But, like other senses, it can mislead when conditions are unfavorable

12. 12 Philosophical Background Reflective Equilibrium Reflective Equilibrium Rawls Decision Procedure for Ethics (1951) Narrow Reflective Equilibrium Narrow Reflective Equilibrium Bring intuitions about particular cases moral principles into accord rejected To do this, sometimes an intuition about a particular case must be rejected

13. 13 Philosophical Background Wide Reflective Equilibrium Wide Reflective Equilibrium Bring intuitions about particular cases moral principles the rest of our beliefs into accord with the rest of our beliefs including beliefs about scientific matters, history, politics even metaphysics & semantics Even more of our intuitions rejected Even more of our intuitions about particular cases will have to be rejected

14. Philosophical Background Evolutionary arguments Evolutionary arguments debunking intuition Perhaps the most influential writer in this tradition is Peter Singer Updated in Ethics & Intuition (2005) The Expanding Circle Ethics and Sociobiology Peter Singer FARRAR, STRAUS & GIROUX New York 1981

15. 15 Philosophical Background nepotistic intuitions kin tribesmen In The Expanding Circle , Singer focuses on nepotistic intuitions which maintain that, in various domains, we ought to value the welfare of our kin and tribesmen more than the welfare of people outside these circles adaptive in ancestral environments The psychological processes leading to judgments of this sort were adaptive in ancestral environments (and perhaps they still are) why we have these nepotistic & tribal intuitions there is no good reason to use them But once we see why we have these nepotistic & tribal intuitions , Singer suggests, we can also see that there is no good reason to use them in a decision procedure for ethics

16. 16 Philosophical Background trolley problems In Ethics and Intuition (2005) Singer develops the argument by focusing on the sort of trolley problems that have loomed large in recent philosophical and empirical studies

17. 17 Philosophical Background neuroscientific evidence emotional reaction harm is caused by the sort of interaction that would have occurred in ancestral environments Singer (following Greene) maintains that the neuroscientific evidence suggests that intuitions about the footbridge case are the result of our emotional reaction to cases in which harm is caused by the sort of interaction that would have occurred in ancestral environments

18. 18 Philosophical Background But what is the moral salience of the fact that I have killed someone in a way that was possible a million years ago, rather than in a way that became possible only two hundred years ago? I would answer: none. The salient feature that explains our different intuitive judgments concerning the two cases is that the footbridge case is the kind of situation that was likely to arise during the eons of time over which we were evolving; whereas the standard trolley case describes a way of bringing about someones death that has only been possible in the past century or two. But what is the moral salience of the fact that I have killed someone in a way that was possible a million years ago, rather than in a way that became possible only two hundred years ago? I would answer: none.

19. 19 Philosophical Background At [a] more general level this casts serious doubt on the method of reflective equilibrium . There is little point in constructing a moral theory designed to match considered moral judgments that themselves stem from our evolved responses to the situations in which we and our ancestors lived during the period of our evolution as social mammals, primates, and finally, human beings. We should, with our current powers of reasoning and our rapidly changing circumstances, be able to do better than that. (348) At [a] more general level this casts serious doubt on the method of reflective equilibrium . There is little point in constructing a moral theory designed to match considered moral judgments that themselves stem from our evolved responses to the situations in which we and our ancestors lived during the period of our evolution as social mammals, primates, and finally, human beings. We should, with our current powers of reasoning and our rapidly changing circumstances, be able to do better than that. (348) undermine some conceptions of doing ethics Those conceptions of ethics tend to be too respectful of our intuitions. Our better understanding of ethics gives us grounds for being less respectful of them. What I am saying, in brief, is this. Advances in our understanding of ethics undermine some conceptions of doing ethics . Those conceptions of ethics tend to be too respectful of our intuitions. Our better understanding of ethics gives us grounds for being less respectful of them. (349)

20. 20 Philosophical Background We agree with Singers skepticism about intuition skepticism But we also think his skepticism is not radical enough! not radical enough!

21. 21 Philosophical Background Assumptions Singer the friends of intuition share Assumptions that Singer and the friends of intuition share : well designed The psychological system underlying our moral intuitions is well designed point reason for Thus there is some point to or reason for the intuitive moral judgments people make when the system is working properly dubious moral importance not to be taken seriously Though Singer (unlike the friends of intuition) insists that the function the system is designed for is of dubious moral importance , and thus that the intuitions are not to be taken seriously

22. 22 Philosophical Background is not well designed We believe that the engine of moral intuition is not well designed at all elegant machine kludge Far from being the sort of elegant machine celebrated in the writings of some evolutionary psychologists, we think that it is a kludge cobbled together rather awkwardly designed for functions that have noting to do with morality a cluster of mechanisms cobbled together rather awkwardly from bits of mental machinery most of which were designed for functions that have noting to do with morality

23. 23 Philosophical Background bricolage To use a term that may be more common in Paris, we maintain that the engine of moral intuition is the result of bricolage Franois Jacob Claude Lvi-Strauss

24. 24 Philosophical Background quirks This explains many of the quirks of moral intuition skeptical use in moral deliberation And provides yet another reason to be skeptical of their use in moral deliberation

25. 25 Overview of the Rest of the Talk

26. 26 Overview of the Rest of the Talk Two examples of the kludginess of the mechanisms underlying moral intuition Dan Kellys Moral Disgust Dan Kellys work on Moral Disgust Joshua Knobes intentionality unconscious moral judgments Joshua Knobes work on intentionality judgments & unconscious moral judgments From kludginess to skepticism From kludginess to skepticism

27. 27 Kelly on Disgust Kelly psychological mechanisms underlying the uniquely human disgust system how that system evolved Kelly has constructed a rich, nuanced, empirically supported account of the psychological mechanisms underlying the uniquely human disgust system and how that system evolved In this talk Ill only have time to for a brief sketch of two central themes Daniel Kelly

28. 28 Kelly on Disgust The Entanglement Thesis The Entanglement Thesis Disgust is itself a kludge Disgust is itself a kludge a uniquely human emotion produced by the merger of two distinct systems The Co-Optation Thesis The Co-Optation Thesis co-opted After the merger, disgust was co-opted by the norm system the norm system the ethnic boundary system the ethnic boundary system human ultra-sociality which were central elements in the emergence of human ultra-sociality

29. 29 Kelly on Disgust vast array of evidence Kelly assembles a vast array of evidence for these theses, drawn from neuroscience social psychology cognitive psychology developmental psychology evolutionary psychology gene-culture co-evolution theory the devil is in the details As usual, the devil is in the details So I join Paul Rozin in urging that you read the work as it appears in print

30. 30 Kelly on Disgust The Entanglement Thesis Disgust exhibits a puzzling array of elicitors elicitors which evoke an equally puzzling cluster of responses responses

31. 31 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects

32. 32 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects body Substances associated with the body : feces, vomit, spit Organic decay Organic decay illness People and objects associated with illness : a shirt once worn by a person with leprosy Sexual practices Sexual practices : necrophilia, incest moral transgressions ors Some moral transgressions & transgress ors : rape, torture, child molestation outgroups Members of low status outgroups : untouchables, Jews

33. 33 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects body Substances associated with the body : feces, vomit, spit Organic decay Organic decay illness People and objects associated with illness : a shirt once worn by a person with leprosy Sexual practices Sexual practices : necrophilia, incest moral transgressions Some moral transgressions & transgressors: rape, torture, child molestation outgroups Members of low status outgroups : untouchables, Jews Some elicitors are pan-cultural

34. 34 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects body Substances associated with the body : feces, vomit, spit Organic decay Organic decay illness People and objects associated with illness : a shirt once worn by a person with leprosy Sexual practices Sexual practices : necrophilia, incest moral transgressions Some moral transgressions & transgressors: rape, torture, child molestation outgroups Members of low status outgroups : untouchables, Jews Others are culturally local (or idiosyncratic)

35. 35 Kelly on Disgust The Entanglement Thesis The disgust response includes Gape face (occasionally accompanied by retching) Feeling of nausea Sense oral incorporation Quick withdrawal A more sustained & cognitive sense of offensiveness A more sustained & cognitive sense of contamination

36. 36 Kelly on Disgust The Entanglement Thesis connected How are all of these connected ? The Entanglement Thesis fusion The Entanglement Thesis maintains that the human emotion of disgust is the result of the fusion of two distinct mechanisms each of which has homologous counterparts in other species only in humans though they have combined only in humans

37. 37 Kelly on Disgust The Entanglement Thesis the poison avoidance mechanism digestion One mechanism ( the poison avoidance mechanism ) is directly linked to digestion regulate food intake poisonous or otherwise harmful It evolved to regulate food intake and protect the gut against ingested substances that are poisonous or otherwise harmful designed to expel substances It was designed to expel substances entering the gastro-intestinal system via the mouth new elicitors And to acquire new elicitors very quickly As John Garcia famously demonstrated, ingested substances that induce gut-based distress often generate acquired aversions

38. 38 Kelly on Disgust The Entanglement Thesis the parasite avoidance mechanism The other mechanism ( the parasite avoidance mechanism ) pathogens and parasites Evolved to protect against infection from pathogens and parasites , by avoiding them close physical proximity infectious agents Not specific to ingestion, but serves to guard against coming into close physical proximity with infectious agents visible pathogens and parasites places, substances and other organisms This involves avoiding not only visible pathogens and parasites , but also places, substances and other organisms that might be harboring them

39. 39 Kelly on Disgust The Entanglement Thesis The disgust response includes Gape face (occasionally accompanied by retching) Feeling of nausea Sense oral incorporation Quick withdrawal A more sustained & cognitive sense of offensiveness A more sustained & cognitive sense of contamination response These elements of the disgust response are traceable to the poison avoidance system

40. 40 Kelly on Disgust The Entanglement Thesis The disgust response includes Gape face (occasionally accompanied by retching) Feeling of nausea Sense oral incorporation Quick withdrawal A more sustained & cognitive sense of offensiveness A more sustained & cognitive sense of contamination and these are traceable to the parasite avoidance poison system

41. 41 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects body Substances associated with the body : feces, vomit, spit Organic decay Organic decay illness People and objects associated with illness : a shirt once worn by a person with leprosy Sexual practices Sexual practices : necrophilia, incest moral transgressions Some moral transgressions & transgressors: rape, torture, child molestation outgroups Members of low status outgroups : untouchables, Jews elicitors These elicitors are traceable to the poison avoidance system

42. 42 Kelly on Disgust The Entanglement Thesis Elicitors include Foods Foods : dog meat, grubs, insects body Substances associated with the body : feces, vomit, spit Organic decay Organic decay illness People and objects associated with illness : a shirt once worn by a person with leprosy Sexual practices Sexual practices : necrophilia, incest moral transgressions Some moral transgressions & transgressors: rape, torture, child molestation outgroups Members of low status outgroups : untouchables, Jews and these are traceable to the parasite avoidance system

43. 43 Kelly on Disgust The Entanglement Thesis different developmental schedules One bit of evidence supporting the Entanglement Thesis is that different components of that response are on different developmental schedules Distaste & gape are present within the first year of life Contamination sensitivity emerges significantly later nomological cluster Once the full system in in place, the components of the response are produced together they form a nomological cluster Any elicitor of disgust will reliably produce all or most of those clustered components

44. 44 Kelly on Disgust The Entanglement Thesis A puzzle A puzzle : Why should the sight of a festering sore or a person with leprosy evoke a gape face and a feeling of nausea? The solution kludge The solution : Disgust is a kludge ! ultrasociality But it is kludge with features that could be readily co- opted and put to other uses as humans began living in larger groups and human ultrasociality emerged

45. 45 Kelly on Disgust The Co-Optation Thesis

46. 46 Kelly on Disgust The Co-Optation Thesis The Gape Face as a Signal The Gape Face as a Signal perspicuous signal dangerous foods and risk of infectious disease As group size increased, there was an increasing need for a perspicuous signal warning of dangerous foods and risk of infectious disease In humans, the face and facial expressions provide a rich source of such social information gape face co-opted as a signal toxic foods parasites and contagious pathogens The gape face , which clearly has roots in the facial motions that accompany retching, was co-opted as a signal , warning others not just against toxic foods , but also against the presence of parasites and contagious pathogens

47. 47 Kelly on Disgust The Co-Optation Thesis Co-Optation by the Norm System Co-Optation by the Norm System social coordination As group size increased, there was increased need for complex social coordination norm system The norm system whose structure we considered briefly in the 2 nd Lecture played an important role in facilitating this co-ordination And the disgust system had features that made it an obvious candidate to be co-opted by the norm system as it evolved

48. 48 Kelly on Disgust The Co-Optation Thesis The S&S model suggests that compliance motivation & punitive motivation are linked to the emotion system

49. 49 Kelly on Disgust The Co-Optation Thesis Acquisition Mechanism Execution Mechanism identify norm implicating behavior infer contents of normative rules compliance motivation punitive motivation emotion system Rule-related reasoning capacity norm data base r 1 ---------- r 2 ---------- r 3 ---------- r n ---------- Proximal Cues in Environment judgment other emotion triggers beliefs explicit reasoning post-hoc justification

50. 50 Kelly on Disgust The Co-Optation Thesis several separate emotion systems But psychological & neurological evidence indicates that there are several separate emotion systems the disgust system being one of them

51. 51 Kelly on Disgust The Co-Optation Thesis Acquisition Mechanism Execution Mechanism identify norm implicating behavior infer contents of normative rules compliance motivation punitive motivation DISGUST Rule-related reasoning capacity norm data base r 1 ---------- r 2 ---------- r 3 ---------- r n ---------- Proximal Cues in Environment judgment other emotion triggers beliefs explicit reasoning post-hoc justification other emotions

52. 52 Kelly on Disgust The Co-Optation Thesis compliance & punitive motivation Disgust is a natural candidate to provide both compliance & punitive motivation for norms that involve intrinsically disgusting matters, like the disposal of corpses & bodily wastes, and other activities that are antecedently salient to the disgust system, like eating practices Compliance norm violating behavior disgusting Compliance is motivated by making norm violating behavior disgusting & thus aversive Punitive motivation violator is considered dirty and contaminated Punitive motivation is provided because the violator is considered dirty and contaminated and is avoided or shunned

53. 53 Acquisition Mechanism Execution Mechanism identify norm implicating behavior infer contents of normative rules compliance motivation punitive motivation DISGUST Rule-related reasoning capacity norm data base r 1 ---------- r 2 ---------- r 3 ---------- r n ---------- Proximal Cues in Environment judgment other emotion triggers beliefs explicit reasoning post-hoc justification other emotions Kelly on Disgust The Co-Optation Thesis

54. 54 Kelly on Disgust The Co-Optation Thesis kludge built with kludgy parts The norm system is thus a kludge built with kludgy parts quirky and disturbing behavior Not surprisingly, this can lead to some very quirky and disturbing behavior disgust system triggered nothing to do with norms Several recent studies have focused on the fact that the disgust system can be triggered by many things that have nothing to do with norms triggered by these non-moral items dramatic and persistent influence on a persons judgments about moral issues but even when triggered by these non-moral items , the disgust system can have dramatic and persistent influence on a persons judgments about moral issues

55. 55 Kelly on Disgust The Co-Optation Thesis Acquisition Mechanism Execution Mechanism identify norm implicating behavior infer contents of normative rules compliance motivation punitive motivation DISGUST Rule-related reasoning capacity norm data base r 1 ---------- r 2 ---------- r 3 ---------- r n ---------- Proximal Cues in Environment judgment other emotion triggers beliefs explicit reasoning post-hoc justification other emotions

56. 56 Kelly on Disgust The Co-Optation Thesis Wheatley & Haidt hypnotically induced disgust often Wheatley & Haidt have shown that when participants are hypnotically induced to feel a brief pang of disgust when they encounter the work often and then presented with the following scenario often Dan is a student council representative at his school. This semester he is in charge of scheduling discussions about academic issues. He often picks topics that appeal to both professors and students in order to stimulate discussion. something wrong many judge that Dan is doing something wrong !

57. 57 Kelly on Disgust The Co-Optation Thesis Schnall Schnall et al. have shown participants make more severe moral judgments when the judgments are made in a disgusting office: greasy pizza boxes sticky chair a dried up smoothie a chewed up pen

58. 58 Kelly on Disgust The Co-Optation Thesis downstream consequences Other studies have focused on prima facie irrational downstream consequences of the disgust system being triggered in moral deliberation

59. 59 Kelly on Disgust The Co-Optation Thesis Acquisition Mechanism Execution Mechanism identify norm implicating behavior infer contents of normative rules compliance motivation punitive motivation DISGUST Rule-related reasoning capacity norm data base r 1 ---------- r 2 ---------- r 3 ---------- r n ---------- Proximal Cues in Environment judgment other emotion triggers beliefs explicit reasoning post-hoc justification other emotions Downstream consequences

60. 60 Kelly on Disgust The Co-Optation Thesis The Lady Macbeth Effect The Lady Macbeth Effect Zhong & Liljenquist unethical deed Zhong & Liljenquist have shown that recalling an unethical deed increased the desire for products related to cleansing, like antiseptic wipes cleaning ones hands reduced moral emotions And that cleaning ones hands after describing a past unethical deed reduced moral emotions like guilt & shame reduced the likelihood that participants would volunteer to help and also reduced the likelihood that participants would volunteer to help a desperate graduate student!

61. 61 Kelly on Disgust The Co-Optation Thesis The Lady Macbeth Effect The Lady Macbeth Effect Schnall et al. Schnall et al. (unpublished) compared judgments about moral severity in two groups of participants cleansing One group had just used an alcohol-based cleansing gel on their hands non- cleansing The other group had just used an ordinary, non- cleansing hand cream cleansing gel significantly less severe! The moral judgments of those using the cleansing gel were significantly less severe!

62. 62 Kelly on Disgust The Co-Optation Thesis Ethnic Boundary Markers Ethnic Boundary Markers Boyd & Richerson Boyd & Richerson & their students have argued that another crucial step in the development of human ultra- sociality was the emergence of mechanisms which allow people to recognize members of their own tribe or ethnie norms facilitate coordination This is important because in-group members share beliefs & norms that facilitate coordination

63. 63 Kelly on Disgust The Co-Optation Thesis cuisines eating practices Since different cuisines & eating practices are one of the more visible correlates of ethnie membership, and since disgust is heavily involved in regulating food intake, disgust was a natural candidate to be co-opted by the emerging system of ethnic identification signs of out-group came to evoke disgust Eating practices of out-groups and other readily detectable signs of out-group membership came to evoke disgust motivation And disgust came to provided a significant part of the motivation to avoid out-group members

64. 64 Kelly on Disgust The Co-Optation Thesis kludgy solution Though the evolutionary function of the ethnic boundary marker system was to facilitate cooperation by keeping groups apart, the kludgy solution to this problem has some unfortunate consequences offensive & contaminating Out-group members are not simply avoided, they are also considered offensive & contaminating disgusting and sub-human! People who embrace different norms are often felt to be disgusting and sub-human!

65. 65 Kludge Meets Kass

66. 66 Kludge Meets Kass Leon Kass, M.D., Ph.D. Conservative bio-ethicist Chairman of the U. S. A. President's Council on Bioethics from 2002 to 2005

67. 67 Kludge Meets Kass The Wisdom of Repugnance In his book, Life, Liberty & the Defense of Dignity (2002), there is a chapter called The Wisdom of Repugnance Kass maintains that deep wisdom "in crucial cases...repugnance is the emotional expression of deep wisdom , beyond reason's power fully to articulate it. Shallow are the souls that have forgotten how to shudder In this age in which everything is held to be permissible so long as it is freely done, and in which our bodies are regarded as mere instruments of our autonomous rational will, repugnance may be the only voice left that speaks up to defend the core of our humanity. Shallow are the souls that have forgotten how to shudder ."

68. 68 Kludge Meets Kass human cloning The claims play a central role in Kass critique of human cloning abortion pornography same-sex marriage Others have adopted the idea to argue against abortion , pornography & same-sex marriage

69. 69 Kludge Meets Kass Some philosophers, most notably Martha Nussbaum, have challenged Kass, arguing that disgust should be discounted in moral & legal deliberation because (roughly) it reminds us of our animal origins

70. 70 Kludge Meets Kass I think Kellys work offers a far more plausible plausible & powerful powerful critique critique

71. 71 Kludge Meets Kass There is no reason to think there is wisdom in repugnance wisdom in repugnance because Disgust is a Kludge Disgust is a Kludge and the psychological system that bases moral judgments on disgust is a Kludge twice over! Kludge twice over!

72. 72 Kludge Meets Kass Anti-Jewish Nazi propaganda often invoked the imagery and language of disgust, purity, contamination & dehumanization very flagrantly A poster advertising the film The Eternal Jew Hitler described the Jew as a maggot in a festering abscess, hidden away inside the clean and healthy body of the nation

73. 73 Knobe on Norms and Intentional Action My second example draws some elegant and exciting work by Joshua Knobe which demonstrates the way in which unconscious moral judgments judgments which an agent may explicitly reject can nonetheless have significant impact on a range of morally relevant intuitions

74. 74 Knobe on Norms and Intentional Action Gary Marcus System 2 Processes In his new book, Kluge , Gary Marcus argues that more recently evolved, computationally slow and consciously accessible mental processes System 2 Processes in the currently fashionable jargon were grafted onto older (System 1) psychological systems designed for quite different purposes kludgy architecture quirks and shortcomings The resulting kludgy architecture accounts for many of the quirks and shortcomings that contemporary cognitive science has discovered

75. 75 Knobe on Norms and Intentional Action Knobes work in the moral domain I think that Knobes work provides an important & disquieting illustration of this phenomenon in the moral domain

76. 76 Knobe on Norms and Intentional Action the side effect effect The story begins with the side effect effect (aka the Knobe effect) one of best known and most surprising finding in the emerging field of experimental philosophy Knobe (2003) reports an experiment in which participants were presented with a pair of almost identical vignettes

77. 77 Knobe on Norms and Intentional Action harm help The vice-president of a company went to the chairman of the board and said, We are thinking of starting a new program. It will help us increase profits, but it will also harm [ help ] the environment. harming helping The chairman of the board answered, I dont care at all about harming [ helping ] the environment. I just want to make as much profit as I can. Lets start the new program. harmed helped They started the new program. Sure enough, the environment was harmed [ helped ].

78. 78 Knobe on Norms and Intentional Action harm blame intentionally In the harm case, participants were asked how much blame the chairman deserved (on a scale from 0 6) and whether he intentionally harmed the environment help praise intentionally In the help case, participants were asked how much praise the chairman deserved (on a scale from 0 6) and whether he intentionally helped the environment harm In the harm case, 82% said the chairman brought about the side-effect intentionally help did not In the help case, 77% said the chairman did not bring about the side-effect intentionally

79. 79 Knobe on Norms and Intentional Action initial hypothesis peoples moral assessment of the side-effect intentionally Knobes initial hypothesis was that peoples moral assessment of the side-effect plays a substantial role in determining whether they are willing to say that the side-effect was brought about intentionally the side-effect is morally bad intentional A judgment that the side-effect is morally bad makes it more likely that it will be judged to be intentional incompatible a purely factual matter an obvious rationale Though this seems incompatible with the widespread idea that judgments of intentionality are judgments about a purely factual matter , it does have an obvious rationale since judgments about whether an action is intentional play a central role in determining whether an agent deserves praise or blame

80. 80 Knobe on Norms and Intentional Action the effect of moral judgments that people consciously make mistaken Subsequent research showed that, if the hypothesis is understood as a claim about the effect of moral judgments that people consciously make , this hypothesis is mistaken The problem emerges clearly in study Knobe ran in collaboration with David Pizarro & Paul Bloom

81. 81 Knobe on Norms and Intentional Action Liberal university students Knobe-style vignettes side-effect Liberal university students were given Knobe-style vignettes in which an advertising executive approves an ad campaign which has the side-effect of encouraging interracial sex encouraging interracial sex or placing gardenias in ones office

82. 82 Knobe on Norms and Intentional Action None inter- racial sex morally wrong None of the participants judged that inter- racial sex (or placing gardenias) is morally wrong intentionally encouraged interracial sex But participants were much more inclined to say that the executive intentionally encouraged interracial sex Explicit moral judgments intention- ality Explicit moral judgments cannot explain the difference in judgments about the intention- ality of the side-effects

83. 83 Knobe on Norms and Intentional Action non-conscious normative judgments the behavior in question violates a norm explicitly reject However, (following Pizarro & Bloom) Knobe has recently proposed that perhaps participants were making non-conscious normative judgments that the behavior in question violates a norm that is made salient by the question or situation, even if it is a norm that they explicitly reject

84. 84 Knobe on Norms and Intentional Action The picture Knobe now proposes looks like this: conscious moral judgment In reaching a conscious moral judgment , we can consider a variety of different moral norms, weigh these norms against each other, perhaps even determine that some of the norms are themselves unjustified.

85. 85 Knobe on Norms and Intentional Action Non-conscious moral judgments Non-conscious moral judgments are formed through a much simpler (system-1 style) process extremely quickly involve very shallow processing They are formed extremely quickly and therefore involve very shallow processing the only norms we consider are the ones that first come to mind search weigh unjustified In generating a non-conscious moral judgment, the only norms we consider are the ones that first come to mind . We do not search for additional norms; we do not weigh norms against each other; we do not ask whether any of the norms might themselves be unjustified .

86. 86 Knobe on Norms and Intentional Action the behavior in question violates any of the norms Instead, we simply determine whether the behavior in question violates any of the norms in the very limited set we are considering transgression this judgment influences our intuitions about intentional If it does, we classify it as a transgression . It is this judgment as to whether or not the behavior is a transgression that then influences our intuitions about intentional action.

87. 87 Knobe on Norms and Intentional Action the most salient norms evoked by a given case The theory predicts that the most salient norms evoked by a given case will be the ones used to in making intentionality judgments, even if subsequent reflection leads the agent to think that there is nothing wrong with violating the norm or that doing so would be a very good thing. Here is a vignette that Knobe has recently used to test this idea

88. 88 Knobe on Norms and Intentional Action violating fulfilling violating fulfilling In Nazi Germany, there was a law called the racial identification law. The purpose of the law was to help identify people of certain races so that they could be rounded up and sent to concentration camps. Shortly after this law was passed, the CEO of a small corporation decided to make certain organizational changes. The Vice-President of the corporation said: By making those changes, youll definitely be increasing our profits. But youll also be violating [ fulfilling ] the requirements of the racial identification law. The CEO said: Look, I know that Ill be violating [ fulfilling ] the requirements of the law, but I dont care one bit about that. All I care about is making as much profit as I can. Lets make those organizational changes! As soon as the CEO gave this order, the corporation began making the organizational changes. 81% violate 30% fulfill 81% of subjects in the violate condition said that he violated the requirements intentionally; 30% of subjects in the fulfill condition said that he fulfilled the requirements intentionally.

89. 89 Knobe on Norms and Intentional Action not the last word how intentionality judgments are generated Knobes theory is certainly not the last word on how intentionality judgments are generated His work has inspired dozens of other researchers there are many studies I have not mentioned and many others are underway

90. 90 Knobe on Norms and Intentional Action IF intentionality judgments kludgy architecture However, IF Knobes theory is on the right track, then intentionality judgments are a product of a kludgy architecture which can be influenced by norms and judgments which the agent is not aware of is not aware of , and does not endorse does not endorse raises serious questions use moral deliberation law This raises serious questions about the use of those judgments in further moral deliberation , or in the law

91. 91 From Kludginess to Skepticism From Kludginess to Skepticism Both Kellys & Knobes work support the hypothesis that motivates this talk The psychological mechanism underlying moral intuition is A Hodgepodge of Multipurpose Kludges A Hodgepodge of Multipurpose Kludges

92. 92 From Kludginess to Skepticism From Kludginess to Skepticism What should we conclude about moral intuition? Suppose thats right. What should we conclude about moral intuition? NOT all moral intuition should be rejected The answer is NOT that all moral intuition should be rejected nor even that intuitions that are closely tied to kludgy features of the mind should be rejected admirable features cultural evolution of norms increased scope and acceptance of norms prohibiting physical harm For, as Shaun Nichols has argued, some of the most admirable features of the cultural evolution of norms including the increased scope and acceptance of norms prohibiting physical harm are the products of kludgy design

93. 93 From Kludginess to Skepticism From Kludginess to Skepticism ALL moral intuitions skepticism Rather, I suggest, the right conclusion to draw is that ALL moral intuitions should be viewed with a healthy dose of skepticism may not have been well designed to do anything The mechanisms that give rise to them may not have been well designed to do anything skeptical about moral intuitions skeptical of the output of a kludgy piece of computer software So we should be skeptical about moral intuitions for roughly the same reason that we should be skeptical of the output of a kludgy piece of computer software

94. 94 From Kludginess to Skepticism From Kludginess to Skepticism Compare and Contrast Compare and Contrast The friends of intuition well designed for morally admirable goals The friends of intuition (e.g. moral sense theorists) think the system producing them is well designed for morally admirable goals it can sometimes misfire though it can sometimes misfire when conditions are unfavorable Previous enemies of intuition well designed morally problematic goals Previous enemies of intuition (e.g. Singer) think the system producing them has been well designed for morally problematic goals kludge has not been well designed at all! We believe that the system producing them is a kludge much of it has not been well designed at all!

95. 95 From Kludginess to Skepticism From Kludginess to Skepticism how can we go about making moral decisions? But if we should be skeptical about all intuition, how can we go about making moral decisions? BIG question HARD one Thats a BIG question & a HARD one . Perhaps Ill be able to suggest an answer

96. 96 From Kludginess to Skepticism From Kludginess to Skepticism the next time I come to Paris