Don’t overestimate your role in successful outcomes


, ,

Jonathan Bernstein draws on Bill James to offer advice to both the Republican and Democratic parties following last week’s election:

I can start not with wisdom from political science, but from the great baseball analyst Bill James, who had useful observations about both winners and losers that I think are worth learning from in the political context, even though it’s obviously quite different.

About winners, Bill James noted their tendency to invariably keep everyone.  That means not only keeping the good players who would help win the next year, but also the bad players that made winning harder (because you can always remember at least one positive contribution, and you’re going to be far more aware of the positives than the negatives); it also meant keeping the players who were useful now, but unlikely to be helpful in the future.  Translated to politics, this suggests that Republicans need to be wary about assuming that everything they did was successful just because they had a good year.  James counsels baseball teams to be ruthless in their self-assessments against the tendency to settle for what worked last time.  Republicans now should do the same.

For losers, James identified the tendency of fans, and sometimes management, to focus most of their criticism on the stars, on the perceived (or even real) flaws of the very best players.  In the baseball context, this is usually insanely self-destructive.  Does the same thing happen in politics?  Yup.  Is it self-destructive?  Yup.

When we succeed, whether in business or politics, we tend to attribute the bulk of that success directly to our efforts.  The Republicans won a significant electoral victory last week, but there is great danger in over assigning causality to their own efforts and under assigning causality to situational and structural factors that may not be in place come 2012 (as I previously discussed here).  Successful organizations must be “ruthless in their self-assessments” so as to avoid confirmation bias and ensure that future strategies match up with future structural environments.  Any event will be the product of agents’ actions and structural factors.  The trick is to untangle as best one can the relationship between and relative weight of both to understand the outcome.

Structural explanations are not always sexy or gratifying, but they typically explain a lot


, ,

In the days after the US midterm elections cable news outlets, radio programs, political pundits, newspapers, and activists on both sides of the ideological spectrum have exerted a great deal of blood and sweat to explain the nationwide drubbing of the Democrats. Democrats are predictably covering their behinds—conceding voter anger, but cautioning that the country has not lurched to the right in just two years. Republicans are claiming validation of their position and a greater ideological alignment with the American people. Activists and enthusiasts of all stripes are weaving narratives that use the election results to validate their personal political perspective. The question, of course, is whether any of this is correct or meaningful. Was this election a mass repudiation of Democratic policies? Was it a validation of the Republican platform and/or Tea Party-style conservatives?

Elections are like Rorschach bots—everyone sees something different, and often times what they see is what they want to see. Particularly with elections, people like to place causation in the hands of people—agents—whose efforts, words, thoughts, etc, drive the outcome. And to be sure, individual agents can and do wield a great deal of influence on events. But an overemphasis on agents can lead to spurious conclusions about why something happens. You must also look at structural or environmental factors. Over at the Monkey Cage, John Snides has a great piece precisely along these lines. Snides and his colleagues looked at which factors where the best predictors of voter choice:

If you had one thing, and one thing only, to predict which Democratic House incumbents would lose their seats in 2010, what would you take? The amount of money they raised? Their TARP vote? Their health care vote? Whether they had a Tea Party opponent? A Nazi reenactor opponent?

Not surprisingly, it’s none of those. Continue reading

Book Review: Codes of the Underworld


, ,

I recently finished Diego Gambetta’s Codes of the Underworld: How Criminals Communicate.  For those looking for a more academic take on signaling (particularly from a sociological point of view) it’s a great find.  As I previously mentioned, Gambetta uses the extreme case of cooperation amongst criminals to tease out more general dynamics of trust, signaling, and communication.  The Mafia can be considered a “hard-case” for theories of signaling trust; given the extreme incentives for criminals to lie and the lack of credibility they wield given the very fact that they are criminals, how is it that criminals manage to coordinate their actions and trust each other at all?  By understanding how trust works in this harsh environment we learn something about how to signal trustworthiness in broader, less restrictive environments.  As Gambetta notes:

Studying criminal communication problems, precisely because they are the magnified extreme versions of problems that we normally solve by means of institutions, can teach us something about how we might communicate, or even should communicate, when we find ourselves in difficult situations, when, say, we desperately want to be believed or keep our messages secret.

The book is a great example of studying deviant cases or outliers, particularly when the area of study is not well worn.  This is a valuable general methodological lesson.  We are typically taught to avoid outliers as they skew analysis.  However, they can be of great value in at least two circumstances: 1) Generating hypotheses in areas that have not been well studied and 2) Testing hypotheses in small-N research designs, where hard cases can establish potential effect and generalizability and easy cases suggest minimal plausibility. Continue reading

Most Viewed Posts: October 2010

Here are the most viewed posts during October.

Remember, you can follow Signal/Noise by RSS feedemail, or by liking the the Facebook page.

As always, thanks for reading!

  1. Visualizing my Social Network
  2. Visualizing the 2011 US Budget
  3. “Statistics is the New Grammar”
  4. Leveraging Social Networks in the Workplace
  5. The True Size of Africa
  6. Has revenue sharing impacted the competitive balance in major league baseball
  7. Counter-signaling in the Luxury Brand Market
  8. Higher Education ROI: How good was your investment?
  9. And the AL Cy Young Award Should Go To..
  10. In Praise of Falsification

Support Ulysses “Seen”


I’ve previously mentioned the Ulysses “Seen” project on this site.  As it turns out, they’ve recently launched a Kickstarter campaign to raise some funds.  For those that are interested in supporting this project (and getting some cool stuff in return) here’s your chance:

With your support, Rob Berry can draw the comic full time over the next several months rather than interrupt his drawing with waiting tables; we can defray the basic costs of production; and we can devote more resources to developing the site as web-based community. We’ve figured out how much we need to pay rent on the studio and keep the web site up for another year: $6,300. We’ve created a great set of rewards for you–but the best one is being a part of a project that is making history.

As of October 30, we have added a new EXCLUSIVE reward for contributors at the $35 level and above. Robert Berry has been creating new character sketches of random Dubliners to be used in crowd scenes going forward. We don’t want to alter our existing rewards structure, but we feel strongly about these new sketches that we want to offer the original pieces as rewards. The order in which the contributions have been received will directly determine the order in which contributors are able to choose. These sketches can be found on the blog (at the first link below) and on the ULYSSES “SEEN” Facebook page (at the second link below). There will be at least 69 sketches at the end of the process, so look for new images to be added to these galleries every day or two.

To support this truly cool endevour and obtain the accompanying warm-fuzzies, click here.

Book Review: Proofiness


, ,

Charles Seife’s Proofiness is an accessible and entertaining look at the many ways numbers can be used (more to the point, abused) in order to win an argument.  Seife spends the early part of the book outlining his typology for numerical abuses.  For instance, “disestimation” is the act of taking a number too literally, understating or ignoring the uncertainty that surrounds it.  This is often done when some kind of data is presented without taking into account that its calculation contains a great deal of measurement error (think of polling or the US Census).  Seife also shows how visualization can be used to manipulate the meaning of data–what he terms “apple-polishing”.  A classic example is portraying longitudinal data in a graph where the y-axis is truncated instead of starting at zero.  Even a small change over time will be magnified by such a presentation, as you can see from the two graphs below.

Apple-polished Y-axis

Correct Y-axis

The book is packed with great examples.  However, Seife spends a bit too much time on some cases.  More variety would have made the book better.  Seife also tends to focus on the intentional manipulation of data while ignoring the unintentional instances.  There is no doubt that people use many of the tricks he describes to bend data to their advantage, but often times misleading data is the result of people simply making bad calculations rather than purposeful manipulation.  Additionally, Seife’s suggestion as to how to combat proofiness, mathematical sophistication, doesn’t seem capable of solving the problem on its own.  While I agree that the public could benefit from a more robust understanding of numbers and their manipulation, Seife basically ignores the issue of perceptual bias.  Even the most sophisticated consumers of data are subject to fundamental perceptual biases.  Given that we are “predictably irrational”, to quote Dan Ariely, any solution must also take into account that we are hardwired in many ways to be manipulated by proofiness.

The book is not for deep subject matter experts in mathematics or statistics, but it is a fantastic primer for the lay person.

In Praise of Falsification



For those that have not read it yet, The Atlantic recently featured an article profiling Dr. John Ioannidis who has made a career out of falsifying many of the findings of medical research that guides clinical practice.  Ioannidis’ research should cause us all to appreciate the various bias we may bring to our own work:

[C]an any medical-research studies be trusted?

That question has been central to Ioannidis’s career. He’s what’s known as a meta-researcher, and he’s become one of the world’s foremost experts on the credibility of medical research. He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences. Given this exposure, and the fact that his work broadly targets everyone else’s work in medicine, as well as everything that physicians do and all the health advice we get, Ioannidis may be one of the most influential scientists alive. Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem. [my emphasis]

Unlike most famous researchers, Ioannidis is not famous for a positive discovery or finding (unless you count his mathematical proof that predicts error rates for different methodologically-framed studies).  Instead, his status has been obtained because of his ability to falsify the work of others–to take their hypotheses and empirical research and show that they are wrong.

This is highly unusual, not only in the area of medical research, but in most academic disciplines. Continue reading

Book Review: The Rational Optimist



A few weeks ago I wrote an initial reaction to Matt Ridley’s The Rational Optimist.  Ridley’s thesis is that the key to prosperity lies in economic growth, and that economic growth is caused primarily by innovation fueled by the exchange and mating of ideas.  As long as ideas are allowed to move freely they will inevitably mate, mutate, and produce innovations that will drive economic growth and improve the quality of life:

The perpetual innovation machine that drives the modern economy owes its existence not mainly to science (which is its beneficiary more than its benefactor); nor to money (which is not always a limiting factor); nor to patents (which often get in the way); nor to government (which is bad at innovation). It is not a top-down process at all. Instead, I am going to try now to persuade you that one word will suffice to explain this conundrum: exchange. It is the ever-increasing exchange of ideas that causes the ever-increasing rate of innovation in the modern world.

Why is Ridley a “rational optimist”?  Because, in his words, he has “arrived at optimism not through temperament or instinct, but by looking at the evidence”.  As Ridley sees it, throughout history, when the free exchange of ideas has been hampered by natural disasters, disease, war, wrong-headed government policy and monopolistic business practices, the result has been a reverse in development and a regression of health and happiness.  Despite these periods of regression, Ridley remains optimistic that the world will continue to get better as long as we ensure that society remains open.  Recent developments in communication technology have only accelerated what Ridley sees as an already robust atmosphere for ideational mating.  This, combined with the progress already achieved in medicine, agriculture, and human rights gives him more hope than not that as a global society we will avoid the many catastrophic predictions making the rounds.

Overall, the book was quite enjoyable.  Ridley spares no sacred cows and marshalls logical and empirical evidence to make his point–whether he is eviscerating the organic farming and climate change movements or patent rights.  He traces over 200,000 years of human history to make the case that we are, on balance, far better today than we have ever been.  This isn’t to gloss over the many problems that persist.  Rather, Ridley freely admits the issues we face while arguing that the past should give us confidence that these issues will also be overcome.  History has shown us that when ideas are allowed to freely move about and reproduce with each other, society is able to drastically improve itself and solve seemingly intractable problems.  I would have liked to see more explicit discussion of ideational “mating”, rather than it’s sprinkling through various chapters, but overall the point is well taken.  My biggest peave with the book is Ridley’s treatment of government.

Continue reading

Friday Signaling Roundup



Here are a few quick signaling items for your perusal.  I will try to do a similar roundup each Friday if I’ve stumbled on enough items throughout the week.  Enjoy!

  • How to Signal That You Are Marrying for Love? It’s tougher than you might think.  Some suggest using a pre-nuptial agreement to signal one’s love and affection instead of their love of money.  If one is truly marrying for love and not money they should have no problem signing a pre-nup if they are the less-wealthy of the pair.  However, the pre-nup may act as a signal from the wealthier of the two parties that they have reason to believe that the marriage will not last.  Therefore, pre-nups are likely only an optimal signal when they are suggested at first by the least wealthy member of the couple. (via Cheap Talk)
  • Tyler Cowen asks the questions “Which ingredient most signals a quality dish?”:  I can’t think of one off the top of my head.  Scallions is noted in the post, and that’s a pretty good one.  I’d think that ingredients that are financially costly and/or time consuming to prepare would also signal quality.  So, higher quality cuts of meat or dishes that are slow roasted or smoked, etc.  A friend of mine once remarked, “Ah, Bean salad.  If you’ve got bean salad then you know there is going to be great desert.”  He was using the quality of an earlier dish to predict the quality of a later one.  (via Marginal Revolution)
  • Can Cheap Talk Deter (PDF)? Potentially in an entry-deterrence situation, according to a draft paper by Dustin Tingley and Barbara Walter.  Tingley and Walter find that in an experimental setting, contra the expectations of their formal model, when participants were able to make a verbal threat to the first potential market entrant it decreased the instances of conflict from 83% (where communication wasn’t allowed) to 38%.  This is interesting, since the verbal threats by the defender where by definition costless (since they wouldn’t not face the challenger again and additional challengers would not know if they followed through on the threat)–meaning, they shouldn’t have revealed any additional information to the challenger.  My first thought is that in an experimental setting subjects might be revealing information through their body language or micro-expressions (which can’t be captured by a formal model) and that these signals conveyed additional information to the challenger.  But defenders where only allowed to communicate their threats to challengers through email.  The authors offer some potential reasons for the discrepant results, such as the unexpected success of early round costless threats actually signals that the defender is a savvy player and understands the game (i.e. fighting early in early rounds to deter future entrants makes sense, and therefore they are likely to follow through on the threat since future entrants will see that they fought).

Happy 75th Anniversary to the Gallup Poll



A unique anniversary is upon us. Seventy-five years ago today — Oct. 20, 1935 — the Gallup Poll published its first official release of public opinion data.

Here we are three-quarters of a century later, still working to fulfill the mission laid out in that first release: providing scientific, nonpartisan assessment of American public opinion.

The subject of that first release? Well, given the fact that 1935 was smack dab in the middle of the Depression, it may come as no surprise that the topic focused on public opinion about “relief and recovery,” or in other words, welfare. President Franklin Delano Roosevelt was at that time heavily involved in creating a number of relief, recovery, and work programs designed to help people whose lives were being affected by the Depression. Figuring out what the public thought about all of this became Dr. George Gallup’s first official poll question.

You can read Frank Newport’s full write up of the first poll here.