Subscribe To Our Newsletter

Get tips and tools to tell your data story better.

No, thanks

 In DataBlog

As we saw last week, RCTs can be very effective in attributing impact or causality to a program or treatment.  However, there are many issues, controversies and hidden problems with actually conducting RCTs in real-life settings.

 

I have seen a lot of studies claiming to be based on RCT design, which in reality are not.  In the complex real world, when dealing with human beings who are subject to many simultaneous influences, achieving true random selection and assignment to treatment is extremely rare.  And even when it does happen for many of the participants, there is usually a sub-population of people who are getting special treatment for one reason or the other – often unwittingly or even accidentally.

 

Furthermore, even if the subjects are assigned to groups randomly, very rarely are participants treated exactly the same way during the data collection process.

 

A scientist and philosopher, Nancy Cartwright has offered us a wonderful article proposing answers to many of these questions about RTCs.

 

“Are RCT the gold standard?”

http://www.lse.ac.uk/cpnss/projects/coreresearchprojects/contingencydissentinscience/dp/cartwright.pdf

 

“The answer to the title question, I shall argue, is ‘no’. There is no gold standard; no
 universally best method. Gold methods are whatever methods will provide a) the
 information you need, b) reliably, c) from what you can do and from what you can
 know on the occasion. Often Randomized Controlled Trials (RCTs) are very bad at
 this and other methods very good. What method best provides the information you
 want reliably will differ from case to case, depending primarily on what you already
 know or can come to know.”

 

So, if you have the money, resources and time to carry out an RTC, then it might be your best bet.  But most researchers and projects do not.  Which does not mean that all is lost.  If you cannot meet the strict assumptions necessary for an RTC, there are many other options for quasi-experimental research designs as well as methodologies to explore impact or causality with non-random data.  More on this next week.

[dismissable_alert class=”alert-info” ]…and take time right now to:

  • Sign up for monthly data science and visualization resources.
  • Take a look at some of the work we have done.
  • Friend us on Facebook for a personal connection with what we do.
  • Follow us on Twitter for the latest news.
  • Get inspired with us on Pinterest.[/dismissable_alert]
Recommended Posts

Start typing and press Enter to search