What is causing the Decline Effect?

The Decline Effect, which seems to have appeared in almost every area of science, is a phenomenon scientists have noticed only recently in which the results which seemed initially to be very strong get weaker and weaker as studies replicate the original research.

Since nobody seems to be gaining from it, undue carelessness in carrying out the original research or even deliberate fraud do not seem to explain why the decline effect is happening.  .  The effect was initially even identified by someone trying to replicate his own research.

There might still, however, be a weakness in the way the scientific method is being implemented by the scientific community.  The problem might be that we have inadvertently made it less and less worthwhile to publish research that does not show that whatever we are studying has some positive relationship to some apparent effect or result.  For example, suppose an article studying Vitamin B12 supplements concludes that it can make a difference in reducing memory loss in old age.  That’s good news.  We can do something about some of that distressing memory loss.  But if, on the other hand, the research finds no discernible differences, there is always the possibility that different research might still turn up something.  Different doses, different combinations of vitamins, different populations, different measurements of cognitive functioning – all of them might turn up some result.  In the meantime, the message is that there’s no new advice for dealing with our forgetfulness.  Not nearly as many people are going to be interested in this headline information.

To put it in technical terms, it is difficult to get research published in professional journals that does not demonstrate what are called “statistically significant” results.

The essence of statistical analysis is deciding whether some event probably happened by chance or not.  Although analyses have become extraordinarily sophisticated with the increasing computer power in the modern world, the principle remains the same.

For instance, walking down the street, there is one chance in 365 that the next person I see will have been born on the same day of the year as I was.  There is one chance in two that if I flip a penny, it will come up heads.  Science uses statistical analysis to decide if the effect connected to some variable happened by chance or is what is called “statistically significant.”

Did this group of people, for instance, who took an aspirin every day for the last ten years have lower levels of cancer than a similar group of people who didn’t regularly take aspirin?  And if the first group did have lower levels of cancer, was that merely a chance difference or were the differences so great that the probability that it was chance are miniscule?

What the decline effect is suggesting is that results that at first look as if could almost certainly not have happened by chance gradually seem to look more and more like chance with repeated replication.

The problem is that if science is tending more and more to publish only those results that are statistically significant, then research which suggests that some variable does not have any effect does not tend to get known by the scientific community at large.  If research showing that eating less red meat is associated with lower levels of cancer gets published, but if research showing that red meat consumption does not seem to be related to incidence of cancer does not, then there is going to be a bias in the publicly accessible research.  So the fact that in reality the association between red meat and cancer might be either very small or not exist at all is going to take much longer to become evident.

The almost universally accepted level of probability that results are not caused by chance is 5%.  In other words, to be statistically significant, there has to be less than 5 chances out of a 100 that it was a fluke outcome caused by chance.  But that means that at least 5% of all research reporting a positive result is probably a result of chance and not a real effect at all.

Traditionally, the way to catch these errors has been through replication of research studies by a variety of different researchers.  But if studies that are not statistically significant are difficult to get published, then these chance errors are not going to be found in the natural course of study.

Imagine you are a young, ambitious scientist eager to make a mark.  Are you going to deliberately put your professional time and energy into doing research that is going to refute research which has already been published and is quite possibly lauded by the professional community?  Is it the wisest thing to begin by pitting your findings against received wisdom?  Wouldn’t it be better to strike out and find out something new and positive and statistically significant?

My strong suspicion is that the Decline Effect is the result of a widespread but indeliberate failure to adequately replicate initial research that reports some supposedly significant finding.   And so it is taking us longer to sort out the wheat from the chaff, to identify those findings that are robust, and those that were based merely on chance.

The one thing science cannot afford to do is to let go of real, robust replication – including studies that report that they have been unable to find anything significant at all.

Advertisements

About Terry Sissons

Terry Sissons is the author of The Big Bang to Now: All of Time in Six Chunks, and this blog is a dialogue about the universe and what’s happened in the last 13 billions years.
This entry was posted in how do we know that? and tagged . Bookmark the permalink.

14 Responses to What is causing the Decline Effect?

  1. chandler richardson says:

    Well i believe the big bang as we call it is an everso expanding wave causing the universe to expand at an exponentional rate meaning at one point or another it will reverb causing an implosion at the end of the universe which then bounce back off itself recreating a world such as ours yet slightly different in some way unknown though this is just a theory of mine if you are curious or want to know more follow me on facebook the username is chanthe man i know ghastly name but i will change it when i have the time please do leave a comment id love to here what you have to say.good day to you all.

    • chandler richardson says:

      I forgot to mention i call this the richardson theorum already patented from this post as i said before good day

      • chandler richardson says:

        So nobody cares this is ingeniuos people

      • chandler richardson says:

        Sorry for the post were i say my theory is ingeniuos that is my friend who didnt see the comment mr sisson made wich i thank him for the time it took him to write it

    • Thank you for what, really is an extremely interesting comment on the Big Bang and the direction of what is certainly an ongoing process. The Richardson theorum does not impress me as being all unfeasible. What annoys me no end is that we’re not going to be able to test it. It’s even unlikely that you and I will be around to observe the full consequences of our man-made global warming. Though if it kills us all, we won’t know it. I doubt even the last man standing would be aware of that desolate fact.

      I would be interested to know why you made your comment following the post on the Decline Effect, which doesn’t seem to be directly related to your theorum. Was it because you couldn’t see any more relevant place to post it?

      Chanthe man isn’t ghastly. When I saw your name, I actually thought it was rather clever. It left me wondering if I couldn’t think up an equally clever name for Terry Herman Sissons. (No! I’m not going to use Terrier Hiss — that really is ghastly.)

      • chandler richardson says:

        Well terry if it is ok to call you that the reason i posted it on the decline effect is not because i couldnt find a better place but yet it is the fact this is my favorite post on this site i to agree it does irratate me slightly that it wont be tested but that changes when people such as yourself post likes or dislikes about my theory it truly makes my day to get comments such as yours i love them now the consiquence is that we may not live to see it for there is another reason its on the decline effect i believe the earth wil not be able to support living organisms as it does today yet the human race may become extinct before this happens and i pray there wont be it would be unbearable to witness or be in the implosion of the universe i imagine the pain from that would make even god shed tears for all who fall on that day please do ask me more on facebook were i can type faster then i can on my tablet

      • chandler richardson says:

        The reason i say thank you is the favt it common courtesy for the fact people such as yourself to time to read my theory also i thank you for writing your comment.

      • chandler richardson says:

        I do hope to hear more from you truly and good day to you

      • chandler richardson says:

        I do also enjoy your book and website i find them just exilarating excuse any inncorrect spelling the buttons on my tablet arent very big

  2. theotheri says:

    Chandler – Please forgive my delay in responding to your invitation to follow up our discussion on Facebook. The problem is that my Facebook account was compromised and I’ve not yet decided how to deal with it. In the meantime, I’ve simply been avoiding the issue mostly because I’m trying to meet a publishing deadline that is looming.

    Your ideas and concerns sound very much like my own, however, and I would hope that we don’t lose contact. Especially if it’s my fault!

    Terry

    • chandler richardson says:

      Oh it is really no big deal now if your page has been compromised and its happened to me a great deal as well i just change my password to something complicated and very hard to comprihend srry for any misspelling on this matter though i really hope to continue any discussion with you at any such date

  3. chandler richardson says:

    Also we can continue our conversation on twitter under lost avenger 15

    • Dear Avenger,

      I appreciate your tenacity and am looking forward to continuing our dialogue on facebook. However, there are a number of personal and professional demands that at the moment are unusually demanding. I hope your patience can hold out possibly until August. I know this is no way to run a blog, but life has a way of making its own demands, doesn’t it?

      Thank you. And I hope you understand. Terry Sissons

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s