Conservatives are more receptive to climate science that involves green-tech innovation than those that entail restrictions (e.g. The spotlight here is on a fundamental question in political theory: who should get what from whom, when, how, and why? He is also the author of Expert Political Judgment and (with Aaron Belkin . Psychologist Peter T. Coleman experiments to learn how to reverse-engineer successful conversations between people about polarizing issues. In addition, the mission was based on mistaken assumptions about wildfiresthat immediate suppression was the optimal strategy. Posing questions and letting the other person draw their own conclusions is more powerful than trying to give them your answer. So too do different mental jobs. *Served Daily*. Tetlock, P.E., &Lebow, R.N. Additionally, companies can enroll in virtual workshops to boost their forecasting capabilities.14. (2004). Escalation of commitment is another (psychological factor). The interrogators would aggressively assault the subjects world-views (the goal was to mentally stress the participants). Taboo Cognition and Sacred Values BACK TO TOP Defining and Assessing Good Judgment My 2005 book, Expert Political Judgment: How Good Is It? Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Superforecasting by Penguin Random House. Tetlock, P. E., Visser, P., Singh, R., Polifroni, M., Elson, B., Mazzocco, P., &Rescober, P. (2007). Blind adherence to these tools can result in poor outcomes: inflexible overconfidence, bad decision-making, avoidable errors, and failures to learn and grow. Weak arguments dilute strong ones. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Do preach to the people that already believe in your mission. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Different physical jobs call for different tools. Political Psychology, 15, 509-530. Opening story: Columbia Universitys Difficult Conversations Lab. Present schooling still relies heavily on the lecture. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Different physical jobs call for different tools. The fundamental message: think. He was elected a Member of the American Philosophical Society in 2019. In Superforecasting, Tetlock and coauthor Dan Gardner offer a masterwork on prediction, drawing on decades of research and the results of a massive, government-funded forecasting tournament. So argues Wharton professor Adam Grant in a fascinating new interview. I was most interested in the ideas from Part 1 and wish he focused on those more. It was psychologist Philip Tetlock who demonstrated that, generally, the accuracy of our predictions is no better than chance, which means that flipping a coin is just as good as our best guess. Some smokejumpers held on to their equipment (as they were trained to do) despite the added weight (possibly) preventing them from surviving. How Can We Know? These habits of thought can be learned and cultivated by any intelligent, thoughtful, determined person., Tetlock, who was born in Canada, attended university in his native country, at the University of British Columbia, where he completed his undergraduate degree in 1975 and his Masters degree in 1976.8He went on to do his doctoral studies at Yale, where he obtained his Ph.D. in psychology in 1979.9Since then, Tetlock has taught courses in management, psychology, and political science at the University of California, Berkeley, the Ohio State University, and the University of Pennsylvania, where he is a current faculty member.10Broadly, his research focuses on the evaluation of good judgment and the criteria used to assess judgment, bias, and error.11, In describing how we think and decide, modern psychologists often deploy a dual-system model that partitions our mental universe into two domains. Opening story: International debate champion Harish Natarajan vs. Debra Jo Prectet (later revealed to be a computer AI). Home; About. The book also profiles several "superforecasters." 8 He went on to do his doctoral studies at Yale, where he obtained his Ph.D. in psychology in 1979. A rivalry exists whenever we reserve special animosity for a group we see as competing with us for resources or threatening our identities.. black and white) leads to polarization, but presenting issues as complex with many gradations of viewpoints leads to greater cooperation. As Prosecutor, we automatically attack any ideas that don't f Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Opening story: Orville and Wilbur Wright and the chemistry the two brothers had as intellectual partners. Experiments can inform our daily decisions.. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. modern and postmodern values. Jason Zweig ofThe Wall Street Journalcalls it the most important book on decision making since Daniel KahnemansThinking, Fast and Slow, which, in the area of behavioral economics, is very high praise indeed. If we want to gain alignment we have to understand where everyone is starting from. Focusing on results might be good for short-term performance, but it can be an obstacle to long-term learning.. Philip Tetlock: It virtually always influences how people make decisions, but it's not always good. Sunk costs are one explanation (an economic factor). Alternative view: intelligence is the ability to rethink and unlearn, i.e. After seeing Earth from above, their perspective changes and the see the commonality of our existence. Study: Typically, researchers report new findings in scholarly journals and Tetlock (1998, 1999) has done so for of some part of the findings of his study. System 2 is the familiar realm of conscious thought. Second thoughts on expert political judgment. [38][39] One consequence of the lack of ideological diversity in high-stakes, soft-science fields is frequent failures of what Tetlock calls turnabout tests.[40][41][42]. We can demonstrate openness by acknowledging where we agree with our critics and even what weve learned from them.. Forecasters with the biggest news media profiles were also especially bad. Great listeners are more interested in making their audiences feel smart., Part III: Collective Rethinking It now turns out there are some people who are spectacularly good at . For millennia, great thinkers and scholars have been working to understand the quirks of the human mind. Affirming the persons desire and ability to change. Make your next conversation a better one. Only one side can be right because there is only one truth. Such research, he says, can "deepen our understanding of how to generate realistic . Cons: The pattern of bookending every chapter with an anecdote gets tiresome. Their heated relationship came to a head in what became known as the "war of the currents Thinking like a politicianseeking to please otherscan lead us astray. Philip Tetlockin Superforecasting: The Art and Science of Prediction, Tetlocks career has been based on the assessment of good judgment. 29). In real-world debates over distributive justice, however, Tetlock argues it is virtually impossible to disentangle the factual assumptions that people are making about human beings from the value judgments people are making about end-state goals, such as equality and efficiency. It looks like the CFO was in Prosecutor mode - calling out the flaws in your reasoning, marshalling arguments to prove you wrong and win her case. As social beings, we are motivated to seek belonging and status. Tetlock has advanced variants of this argument in articles on the links between cognitive styles and ideology (the fine line between rigid and principled)[31][32] as well as on the challenges of assessing value-charged concepts like symbolic racism[33] and unconscious bias (is it possible to be a "Bayesian bigot"?). Superforecasting is both a fascinating leap into the art of decision making as well as a manual for thinking clearly in an increasingly uncertain world. How Can We Know? philip tetlock preacher, prosecutor, politician. Opening story: Daryl Davis is a musician and a Black man. Learning from experience: How do experts think about possible pasts (historical counterfactuals) and probable futures (conditional forecasts)? Tetlock aims to provide an answer by analyzing the predictive methodologies of leaders and researching those that are most successful at accurately forecasting future events. In Behind the Science of Intelligence Analysis, Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security, Division of Behavioral and Social Sciences and Education. Expert Political Judgment: How Good Is It? As if growing up is finite. During a spacewalk, Luca felt water in his helmet. Philip Tetlock of the University of Pennsylvania and author of Superforecasting talks with EconTalk host Russ Roberts about his work on assessing probabilities with teams of thoughtful a Show EconTalk, Ep Philip Tetlock on Superforecasting - Dec 20, 2015 In this hour-long interview, Tetlock offers insight into what people look for in a forecaster everything from reassurance to entertainment and what makes a good forecaster it requires more than just intelligence. the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores; the degree to which the best forecasters could learn to distinguish many degrees of uncertainty along the zero to 1.0 probability scale (many more distinctions than the traditional 7-point verbal scale used by the National Intelligence Council); the consistency of the performance of the elite forecasters (superforecasters) across time and categories of questions; the power of a log-odds extremizing aggregation algorithm to out-perform competitors; the apparent ability of GJP to generate probability estimates that were "reportedly 30% better than intelligence officers with access to actual classified information. The others might not agree with those arguments, but they are left defenseless and bitter. How can organization structure incentives and accountability procedures to check common cognitive biases such as belief perseverance and over-confidence? We dont just hesitate to rethink our answers. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Being persuaded is defeat. Present fewer reasons to support their case. The team was inserted into challenging conditions and the fire quickly overtook them. It consists of everything we choose to focus on. Since its original publication, Expert Political Judgment by New York Times bestselling author Philip Tetlock has established itself as a contemporary classic in the literature on evaluating expert opinion. Their conclusions are predetermined. [1] Deniers reject anything from the other side. They revert to preacher, prosecutor, and politician modes. Prosecutors: We attack the ideas of others, often to win an argument. We risk overemphasizing pleasure at the expense of purpose. Opening story: Smokejumpers and the Mann Gulch fire (Montana) of 1949. Grants solution is an idea he calls rethinking. Rethinking is the process of doubting what you know, being curious about what you dont know, and updating your thinking based on new evidence (in other words, the scientific method). Journal of Personality and Social Psychology 78 (2000):853-870. Being persuaded is defeat. Still, Tetlock has gone beyond journal articles, turning to a Harish uses a powerful combination of techniquescommon understandings, non-judgmental questions, flexible thinkingto win over some in the audience. the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals. In the most comprehensive analysis of expert prediction ever conducted, Philip Tetlock assembled a group of some 280 anonymous volunteerseconomists, political scientists, intelligence analysts . Preachers: We pontificate and promote our ideas (sometimes to defend our ideas from attack). Cognitive bias: Seeing what we want to see. We constantly rationalize and justify our beliefs. They too are prone to forgetting their professional tools. We identify with our group or tribe. Department of Psychology / Stephen A. Levin Building / 425 S. University Ave / Philadelphia, PA 19104-6018Phone: (215) 898-7300 / web@psych.upenn.edu, Welton Chang [Psychology Graduate Student], 2023 The Trustees of the University of Pennsylvania, https://psychology.sas.upenn.edu/system/files/Tetlock%20CV%20Updated%20feb%202%2. Tetlocks primary research interest, the question of what constitutes good judgment, is also his claim to fame. Insights and interesting reads delivered straight to your inbox. Tetlock, R.N. Use a steel man (instead of straw man) and consider your opponents strongest argument. Even criticize them. (2005). Terms in this set (50) freedom and equality. Changing your mind is a sign of moral weakness. Professionally, its all about setting the table and/or recognizing the table thats been set. Those who embraced flexible thinking did not. Opening story: Teacher Erin McCarthy assigned her 8th grade students a textbook from 1940 to see if they accepted the information without question or if they noticed any problematic anachronisms. This research argues that most people recoil from the specter of relativism: the notion that the deepest moral-political values are arbitrary inventions of mere mortals desperately trying to infuse moral meaning into an otherwise meaningless universe. Confident humility: An ideal wherein the individual has faith in their abilities but retains sufficient doubt and flexibility to recognize they could be wrong. The mission was aborted and Luca barely escaped drowning in his spacesuit due to a mechanical failure that wasnt properly diagnosed. Politician mode seeks the approval of others and has little conviction for the truth. [3] The original aim of the tournament was to improve geo-political and geo-economic forecasting. Status is gained by holding the purest expression of these views. Tetlock is also co-principal investigator of The Good Judgment Project, a multi-year study of the feasibility of improving the accuracy of probability judgments of high-stakes, real-world events. Visit www . What might happen if its wrong? We can strategize from there and know which mode, preacher, prosecutor, or politician, to operate from and to. Professor Tetlock, who's based at the University of Pennsylvania, famously did a 20-year study of political predictions involving more than 280 experts, and found that on balance their rate of . Rational models of human behavior aim to predict, possibly control, humans. Psychological safety is not a matter of relaxing standardsits fostering a climate of respect, trust, and opennessits the foundation of a learning culture.. Tetlock, P. E. (1994). Its easy to notice when others need to change their opinions, but difficult for us to develop the same habit for ourselves. How Can We Know?,[2] Tetlock conducted a set of small scale forecasting tournaments between 1984 and 2003. Relationship conflict: Personal feuds and arguments (e.g. You wouldn't use a hammer to try to cut down a tree, and try to use an axe to drive nails and you're likely to lose a finger. Psychologically unsafe settings hide errors to avoid penalties. Its not a matter of having low self-confidence. Just a few more efforts at rethinking can move the needle.. Open their mind to the possibility they might be wrong and let them work their way to the solution. The purpose of learning isnt to affirm our beliefs; its to evolve our beliefs., The rethinking cycle: Humility => Doubt => Curiosity => Discovery, The overconfidence cycle: Pride => Conviction => Confirmation and Desirability Biases => Validation, Chapter 2: The Armchair Quarterback and the Imposter. Nuance is not rewarded by the attention economy. Chapter 5: Dances with Foes. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. In practice, they often diverge.. Interrogate information instead of simply consuming it. He has served on the faculty of the University of California, Berkeley (19791995, assistant professor), the Ohio State University (the Burtt Endowed Chair in Psychology and Political Science, 19962001) and again at the University of California Berkeley (the Mitchell Endowed Chair at the Haas School of Business, 20022010). on government, politics, or international affairs and the Robert E. Lane Award for best book in political psychology. PHILIP E. TETLOCK is Annenberg University Professor at the University of Pennsylvania (School of Arts and Sciences and Wharton School). . 1988-1995 Director, Institute of Personality and Social Research, University of California, Berkeley. [17][18] Tetlock uses the phrase "intuitive politician research program" to describe this line of work. It requires us to admit that the facts may have changed, that what was once right may now be wrong.. Structuring accountability systems in organizations: Key trade-offs and critical unknowns. Armchair quarterback syndrome: Phenomenon where confidence exceeds competence. The sender of information is often not its source. All Rights Reserved, Fostering holistic wellness through science and design, Unlocking product's potential through behavioral design, Generating low cost, high impact interventions in public policy, Unlocking every classroom's potential using behavioral science, Bringing about a sustainable future through nudging, Empowering people to take control of their finances, Designing positive choice environments for consumers and brands, Translating good intentions into scalable progress, Helping investors avoid bias and grow their impact, Thought pieces on how behavioral science creates positive impact, Conversations with some of the world's most influential voices, A practical guide on how our minds understand the world around us, How some of the world's most influential thinkers got there, Foundational concepts to help you understand decision science, From Theory to Frameworks: Putting Behavioral Science to Work, Management in the WFH World with Jean-Nicolas Reyt, We're on a mission of empowerment through evidence based choice, Join us on our mission to help the world make better decisions, Diverse perspectives, brought together by a passion for impact, Thought leadership from the front lines of behavioral science, A look at some of our most impactful work.