En Route to Open Science Research

In March, Dr. Gilad Feldman (HKU) gave HKUST researchers a very engaging webinar on his journey to open science. This post summarizes his talk and best practices on how you can embark on your own.

Get Started on a Journey to Open Science

Open science has many facets, and one can easily get bogged down by this ambitious yet broad term when we wanted to apply open science principles to our day-to-day research work. In the Library’s Researchers’ Series talk held on March 16, guest speaker Gilad pointed out that there is more than one way that one can take actions in making research workflow more transparent and reproducible, and we can start with simple steps.

Gilad’s journey on open science began in 2016 when he was doing post-doc work in Sweden. Reflecting on an undergoing of replication/reproducibility crisis in psychological science and in order to drive for a science reform for credible, reproducible and replicable research, Gilad decided to prioritize pre-registered replications and pre-registered meta-analyses in his future work, with a focus on the realm of judgment and decision making.

Among the many stories and projects he shared, here we highlight one example to demonstrate how one can start on conducting a replication study over a short period of time:

What’s Next: Large-Scale Replication Studies

Since 2017 Gilad has made a big push on scaling up replication studies and he started to collaborate with early career researchers (ECRs) and supervise students on conducting mass replications & extensions as well as meta-analyses in classic findings in judgement and decision making and social psychology.

Over the past four years, the team has completed around 80 pre-registrations and 40 Registered Report Stage 1, most of those with extensions, making it one of the largest replication efforts in social psychology. You may find the preprints from this table, and replication project codes and data on OSF.

The impact that this project brings is far-reaching and multifold. First and foremost, students not only gain a more in-depth understanding of the knowledge taught in classes, they are also mobilized to gain tremendous hands-on research experience by conducting the studies, as well as having publications that can build up their CVs. And of course, the existing scientific literature body will become more robust and credible by having more replication studies carried towards them. It is worth noting that because of one of the replication projects, one journal recently issued an expression of concern over a paper by one of the world’s most prominent behavioural psychologists.

Key Takeaways – Over to You to Embrace Open Science

The webinar recording is available on YouTube and the slides are available at https://osf.io/p9e84/ . Here are a few key takeaways on open science from his talk, which can be beneficial to students and ECRs in all fields.

Image: Screen capture of the slide “Summary of points made”

• Embrace transparency in resources, materials, data, and code (e.g. in OSF)
  • Sharing all materials, data, code in a way that would allow others to easily follow, understand, rerun, and reach the same conclusions
  • Sharing entire process starting from the moment of idea conception, including failures.
  • Reporting all research decisions (exclusions, conditions & variables), including null findings
• How to do replication study
  • Pick a very simple effect in your domain
  • Can be run online
  • Vignette study
  • Simple methodology (main effect or simple interaction)
• Conduct a replication of the effect
  • Write a pre-registration, simulate some data, prepare the code
  • Conduct the pre-registration
  • Run the data collection (e.g., online labor market MTurk)
  • Write up the manuscript
• Join an open-science community/conversation online
  • Professional associations, for example FORRT (Framework for Open and Reproducible Research Training) 
  • Find collaborations/trends via Twitter

More Information and Resources

You may follow Gilad via Twitter (@giladfeldman) or visit his group page for more information. I would also like to highlight a few more talks of his. Check them out on YouTube.

1. Collaborative “credibility revolution” open-science & meta-research in management, 2021
2. Pre-registrations and Registered Reports, 2020

– By Jennifer Gu, Library

Hits: 306