Alan Finkel: How to eradicate bad science

  Last updated September 13, 2019 at 4:01 pm

Topics:  

Science needs to take a lesson from the Banking Royal Commission on how to improve research.


scientific research_alan finkel_chief scientist

Australia’s Chief Scientist Alan Finkel. Credit: AAP




Why This Matters: If Australia wants to stay a world leader, we need to get rid of the bad science.




In the main, Australia produces high-quality research that is rigorous and reproducible, and makes a significant contribution towards scientific progress. But that doesn’t mean we can’t do it better.


In the case of the research sector here and abroad, we need to acknowledge that as good as the research system is, there is a problem.


There are a significant number of papers that are of poor quality, and should never have made it through to publication. In considering why this might be the case, I have found myself reflecting on the role of incentives in the research sector.


Because incentives matter, as we have seen through the findings of the Royal Commission into the banking sector led by Kenneth Hayne.


The commission shone a light on how the sector incentivises its employees. And there are some incentives in the research community that, in my view, need to be looked at.


We may be inadvertently encouraging poor behaviour. And to ensure research remains high-quality and trustworthy, we need to get the incentives right.


Lessons from the banking Royal Commission


scientific research_NAB_banking

Parallels can be drawn between inappropriate incentives in both the banking sector and research sector, Dr Finkel says. Credit: AAP


The commission showed that over the past decade or two, the banking sector moved from salary-based to bonus-based remuneration. But those bonuses have been mapping to the wrong values: to sales and profit instead of compliance with the law and net benefit to customers.


Similarly in the research sector, we can’t ignore that there are many incentives pushing some to cut corners and lower their standards.


The competition for funding is fierce and is increasing every day. The temptation to judge a researcher’s performance through simple metrics, such as the number of published research papers, is strong. These metrics are incentives that drive behaviour – not all of it good.


We all know of instances of poor research practice. Selective publication of results to support a hypothesis. HARKing: hypothesising after results are known. Manipulating data and research methods to achieve statistical significance.




Also: Anisa Rowhani-Farid – Towards a culture of open science and data sharing in health and medical research




If we can focus on improving the quality of scientific research in general, we can achieve broad and long-lasting benefits. And I think the best way to do this is to look at the incentives.


Quality should trump quantity


Publication is a principal criterion for career advancement in the research sector. And I don’t want to change that. However, the institutionalisation of performance metrics has created incentives for researchers to publish as many papers as possible.


There shouldn’t be an incentive for a researcher to salami-slice their results into three or four separate publications, rather than one meaningful publication. If the purpose of publication is to share your results in a way that can be built on by other researchers, this kind of practice completely defeats that purpose.


scientific research_science research_australian science

Perverse incentives can encourage the manipulation of research results, Dr Finkel says. Credit: AAP


One model that places the focus on quality over quantity is the “Rule of Five”. With this rule, a researcher’s performance for grant funding or promotion is judged on their best five publications over a five year period, accompanied by a description of its impact and the researcher’s individual contribution.


The exact number of publications or years isn’t important, as long as it is less than ten.


Of course, there are disciplinary differences that may need to be taken into account. But what matters is the emphasis on the significance of the research.


Researchers must undergo integrity training


Unlike other professions, there are no national competencies and no national recognition of education and training in scientific research integrity. While many institutions in Australia do provide training programs for their PhD students, these programs vary in quality, content and reach.


And, to the best of my knowledge, no Australian institutions have a training requirement for their existing research workforce.


I strongly believe the overall quality of research in Australia would be strengthened by research integrity training for all researchers.


Training puts a spotlight on expectations for the whole community and encourages consistent behaviour. It also removes that old chestnut of plausible deniability: “Honest, officer, I didn’t know it was wrong!”




Deeper: What kind of society do we want to be?




The training must be accredited, and must be high quality. It should not be a “tick the box” exercise. And if we circle back to incentives, the best way to encourage researchers to undertake the training is to make proof of training a requirement for obtaining a grant.


To those naysayers who say it will never happen, let me tell you that it already has. The Irish Health Research Board has recently implemented such a scheme.


‘Predatory, evil’ scholarly journals


Finally, I am concerned that the incentives in the research system are not just driving bad behaviour for researchers, but are also creating a market for criminals to enter scholarly publishing.


What is motivating the crooks is the pay-per-page system that has come with the introduction of open access publishing.


Now, open access publishing has many benefits and I support the move towards it. But I remain concerned that it has opened the door for predatory, evil, crooked journals.


It is just too easy to set up a journal and a website with a highfalutin title, and appropriate the biographies of leading researchers for the editorial board – without their knowledge or permission. Before you know it, huge numbers of papers are being published without any rigour.


And there are researchers who are knowingly paying to publish in journals that have no peer review, even though they claim to. Journals that have no ethics. Not even an editorial team to consider the submitted paper.


scientific research_alan finkel_chief scientist

Finkel: a new publishing standard should be introduced to weed out unscrupulous research journals. Credit: AAP


These researchers might just be naïve, but we have to acknowledge that the current incentives reward this behaviour.


While this is not a major problem in Australia, emerging research nations are really struggling with this.


In my conversations with senior research leaders around the world, they are looking for ways to improve performance metrics in a way that does not drive their researchers to these journals.


I propose a rigorous quality assurance system, designed to inform stakeholders that a particular journal’s processes for assessing a paper meets agreed publishing standards. I like to call it Publication Process Quality Assurance, or PPQA.


Compliance with PPQA would indicate to researchers, research institutions, libraries and granting agencies that the journal follows internationally accepted guidelines for the publication process.


Granting agencies are best placed to provide the incentive for researchers to only publish in PPQA-compliant journals by enforcing it through their grant application process.


Follow the money


You might have picked up by now a common thread; that in each of my three recommendations, I am looking to take the responsibility back to the granting agencies. It’s a concept referred to by others as “follow the money”.


If the granting agencies put in place these measures, they will ripple through into the research institutions and mitigate the ongoing risks of poor quality research.


It will change the culture and ensure that last century’s academic rigour continues for the 21st century research workforce and beyond.


This is an edited version of the speech Dr Finkel delivered in Melbourne on September 12 at the Peter Doherty Institute for Infection and Immunity, to mark its fifth anniversary.


This article is republished from The Conversation under a Creative Commons License. Read the original article


More Like This


A letter to a Year 10 student from Australia’s Chief Scientist


Getting science into Australia’s culture




About the Author

Alan Finkel
Dr Finkel is a neuroscientist, entrepreneur and philanthropist. He invented a commercially successful device which substantially speeds up drug research. Dr Finkel is the President of the Australian Academy of Technological Sciences and Engineering (ex-officio) and Chancellor of Monash University. He founded the science magazines Cosmos and G: The Green Lifestyle Magazine, contributes towards a number of research institutes and is Patron of the Australian Science Media Centre. In 2016, he was appointed Australia’s Chief Scientist.

Published By

Science and technology is as much a part of our cultural fabric as art, music, theatre and literature. They play a significant role in our daily lives, yet, in a world dependent on science, we often take them for granted. Australia’s Science Channel believes every citizen has a right, and a responsibility, to be informed, and our mission is to create programs to bring that about.


Featured Videos

Placeholder
Bringing Japan's breakthrough cancer technology to Australia
Placeholder
Columbia: NASA blew it
Placeholder
The Face of a Stranger
Placeholder
Where Does Space Begin?
Placeholder
The Rarest Drug on Earth
Placeholder
Why is blue so rare in nature
Placeholder
Ant Sisters
Placeholder
Jeremy the Lefty Snail and Other Asymmetrical Animals
Placeholder
Tracking Snow
Placeholder
Smart Slime?
Placeholder
Good in the machine
Placeholder
Kessler Syndrome: What happens when satellites collide?
Placeholder
Why This Skateboarding Trick Should Be Impossible
Placeholder
Charles Camarda on becoming an astronaut
Placeholder
Alan Duffy on what it took to get humans to the Moon
Placeholder
Do aliens exist? Brian Cox explains
Placeholder
From Apollo to Pulsars: Parkes still dishing out the discoveries
Placeholder
Brian Cox on black holes
Placeholder
Australia's 60,000 years of space history
Placeholder
In Class With… Jane Goodall
Placeholder
Etienne Rastoin-Laplane - What's fishy about the Galapagos?
Placeholder
Kit Prendergast - Flowers to keep native bees buzzing
Placeholder
Rebecca Wellard - Eavesdropping on killer whales
Placeholder
Hossein Tavassoli - Mending broken hearts
Placeholder
Dilan Seckiner - Forensic gait analysis
Placeholder
Samuel Bladwell - A new spin on electronics
Placeholder
Sathana Dushyanthen - The double-edged cancer sword
Placeholder
Dwan Price - Nuts and Guts
Placeholder
Catriona Nguyen-Robertson - Exercise takes your immune system for a ride
Placeholder
Thimo Ruethers - The deadly danger of crocs on a plate
Placeholder
Amanda Tauber - Slamming the brakes on metastatic cancer
Placeholder
Hayley Teasdale – The ball that prevents falls (FameLab Australia 2019 Runner-up)
Placeholder
In the Shadow of a Black Hole
Placeholder
In Class With... Monica Gagliano
Placeholder
In Class With... Brian Cox
Placeholder
Start your FameLab 2019 journey now
Placeholder
Nural Cokcetin - It all starts with FameLab
Placeholder
Erinn Fagan-Jeffries - It all starts with FameLab
Placeholder
Ronald Yu - It all starts with FameLab
Placeholder
Noushin Nasiri - It all starts with FameLab
Placeholder
Grassroots
Placeholder
What is machine learning?
Placeholder
Mythbusting artificial intelligence with expert Anton van den Hengel
Placeholder
Using machine learning to predict medical outcomes
Placeholder
KCLOC
Placeholder
Nature Calls
Placeholder
Mexican Fishing Bats
Placeholder
Bittersweet
Placeholder
Timelapse
Placeholder
Invisible Blanket
Placeholder
Look
Placeholder
The Anomalies: Venom Race
Placeholder
Science Meets Making
Placeholder
Spiral
Placeholder
Looking Out There
Placeholder
Protectors of the Penguins
Placeholder
Astroturf
Placeholder
Virtual Humans
Placeholder
Rancheros del Jaguar
Placeholder
Searching For Dark Matter
Placeholder
Finding prehistoric mega-shark fossils on Victoria's coast
Placeholder
The Grandfather of computers
Placeholder
James Cameron talks science
Placeholder
In Class With.....David Suzuki - The Environment
Placeholder
In Class With.....David Suzuki - Career
Placeholder
Sustainable water use with Doug Green
Placeholder
Why is Indigenous science important?
Placeholder
Vanessa Pirotta - Using drones to collect whale snot (FameLab Australia 2018 Winner)
Placeholder
Toby Hendy - Poking Plants (FameLab Australia 2018 Runner-Up)
Placeholder
Muthu Vignesh Vellayappan - Groovy Patches (FameLab Australia 2018 Audience Choice)
Placeholder
Taryn Laubenstein - The Tail of Two Fishes
Placeholder
Richard Charlesworth - Coeliac disease diagnosis can be a pain in the posterior
Placeholder
Pegah Maasoumi - Solar Windows
Placeholder
James Wong - Breathing while you hop: How do kangaroos do it?
Placeholder
Ben McAllister - The ORGAN Experiment: Shining a light on dark matter
Placeholder
Mortaza Rezae - Empowering beautiful minds
Placeholder
Zane Stromberga - Can allergy drugs beat bladder disease?
Placeholder
Working In.....Art - Astrophotography
Placeholder
What's the best way to move - springs or muscles?
Placeholder
FameLab Australia Semi-Final Highlights
Placeholder
Saving lives with platypus milk
Placeholder
Australian astronomers witness death throes of a cocooned star
Placeholder
How Australia's politicians see our future in space
Placeholder
Keeping satellites in the loop
Placeholder
Tim Flannery talks about COP
Placeholder
Tim Jarvis & Tim Flannery talk Climate Change
Placeholder
Andy's Week in Science - robo baby, university rankings, and cancer on circadian rhythms
Placeholder
From chocolate factory to surgery - the milliDelta robot
Placeholder
Andy's Week in Science: video games, low tech transition windows and a new CRISPR technique
Placeholder
Science lessons useful in Art Restoration career
Placeholder
Are drones the future of racing?
Placeholder
The future of esports according to the experts
Placeholder
Seeing is believing with artist Eugenie Lee
Placeholder
The human impact of Art Science collaboration
Placeholder
Follow your Interests in Robotics
Placeholder
Zoz on 3D Printing
Placeholder
Flavia Tata Nardini on women in engineering
Placeholder
Flavia Tata Nardini on the future of the internet
Placeholder
Explore the ocean floor and Antarctic biodiversity
Placeholder
Follow your interests in Medical Research
Placeholder
Artists on Science
Placeholder
What is Space Archaeology?
Placeholder
Follow your Interests
Placeholder
Scientists on Art
Placeholder
3D Printing in Medical Research
Placeholder
Ethical Issues
Placeholder
Problem Solving - Robotics at Dermatec
Placeholder
Problem Solving with CSI
Placeholder
Tamarah King - Earthquake Geologist
Placeholder
True or False with Bajo and Rad BONUS ROUND
Placeholder
True or False with Bajo & Rad
Placeholder
Andy's Week in Science - Cats vs Dogs
Placeholder
FameLab 2018 - Get Involved!
Placeholder
Nural Cokcetin - How FameLab changed my life
Placeholder
Erinn Fagan-Jeffries - How FameLab changed my life
Placeholder
Noushin Nasiri - How FameLab changed my life
Placeholder
Ronald Yu - How FameLab changed my life
Placeholder
Alan Duffy's Top 5 Science Communication Tips
Placeholder
A Judge's Top Tips for FameLab Australia
Placeholder
Brain Candy - Why, Why, Why Michael Stevens?
Placeholder
The Past, Present, and Future of Malaria
Placeholder
This is a video of poo pills being made!
Placeholder
Mind Games - Sports Psychology
Placeholder
Fuel to Win - Sports Nutrition
Placeholder
Fifty years since Australia beat the world to space
Placeholder
ECR Network: Talk Your Science with Alan Duffy
Placeholder
Andy's Week in Science - chimps, klompen, and clouds
Placeholder
Our robot medicine future - heart huggers and micro biohybrids
Placeholder
Six Awkward Cancer Questions
Placeholder
How do you tell if a whale is left-handed?
Placeholder
She Flies - Turning Girls into Drone Pilots
Placeholder
Andy's Week in Science - Magnetic Fabric, Cancer Treatments, and Echolocation
Placeholder
The Science of Sexuality
Placeholder
Sailing Through Space with Bill Nye
Placeholder
Using Sports Science to Help Olympic Athletes
Placeholder
Three and a Half Minutes of Top Shelf Career Advice
Placeholder
New Space Tech with Andrea Boyd
Placeholder
Kelly Meets the Mars Curiosity Rover
Placeholder
Hearts, Opera, and Tough Conversations - Andy's Week in Science
Placeholder
Bill Nye on Science, Girls, and Saving the World!
Placeholder
2017 Prime Minister's Prizes for Science Part 2
Placeholder
2017 Prime Minister's Prizes for Science Part 1
Placeholder
Who Decides the Law in Space?
Placeholder
Scientists Watch Collision That Created Gravitational Waves
Placeholder
Getting Cold Feet Leads to a Whole New Career
Placeholder
ECR Network - Why Every Scientist Should Be on Twitter - The Benefits
Placeholder
ECR Network - Why Every Scientist Should Be On Twitter - The Fears
Placeholder
Live Podcast - Life Vs Science
Placeholder
Origami Robots, Babies, and Kidneys - Andy's Week in Science
Placeholder
Namira Salim and the Zero-G Peace Summit
Placeholder
Elon Musk's Mars Plan: Expert Analysis
Placeholder
SPACE AF - Thursday
Placeholder
My Time in Space
Placeholder
IAC TV Daily Broadcast - Wednesday
Placeholder
SPACE AF - Wednesday
Placeholder
IAC TV Daily broadcast - Tuesday
Placeholder
SPACE AF - Tuesday
Placeholder
IAC TV daily broadcast - Monday
01:00:41
Placeholder
SPACE AF - Monday
Placeholder
Live from IAC 2017
Placeholder
Brew Ha Ha - Technology Rewrites History
Placeholder
Methamphetamine - Gateway Drug to Parkinson's Disease
Placeholder
Concussion, 3D BioPrinting, and The Universe - Andy's Week in Science
Placeholder
Pulsars, Clearwigs, and Pacemakers - Andy's Week in Science
Placeholder
Revolutions - The Quest to Transform HPV Racing
Placeholder
Brew Ha Ha - Hurricane Irma Blows Away Tesla's Rip Off
Placeholder
Brew Ha Ha - The Limit of Your Lifespan
Placeholder
The Recipient
Placeholder
Think Like a Scientist: Natural Selection in an Outbreak
Placeholder
The End of Snow
Placeholder
The Next Rembrandt
Placeholder
The Discarded
Placeholder
The Spectators
Placeholder
Test Tube Babes
Placeholder
Pangolins in Peril- A Story of Rare Scales
Placeholder
Rock Art Project
Placeholder
Pork.0
Placeholder
OWSIA (Darkened Water)
Placeholder
Nex
Placeholder
Northern Quolls
Placeholder
Dish Life
Placeholder
At Street Level
Placeholder
Custom Love
Placeholder
Adrift
Placeholder
A Story from Space
Placeholder
Brew Ha Ha - The Most Dangerous Thing in Boxing May Be the Gloves
Placeholder
ECR Network 2017 – Get Interdisciplinary!
01:27:00
Placeholder
Chris Hadfield: The Future of Space Exploration
Placeholder
Chris Hadfield: Life After Space
Placeholder
Chris Hadfield: Life in Space
Placeholder
Brew Ha Ha - Let's Make Algae Australian of the Year
Placeholder
Brew Ha Ha - Australia's Energy Showdown
Placeholder
Nine Awkward Astrophysicist Questions
Placeholder
Brew Ha Ha - There's No Such Thing as an Exercise Pill
Placeholder
National Science Week Awards Show
Placeholder
Brew Ha Ha - Your 5 Step Asteroid Success Plan
Placeholder
National Science Week Forecast
Placeholder
Open Doors. Open Future. Open Day.
Placeholder
Brew Ha Ha - Lose a Little to Gain Millions
Placeholder
Brew Ha Ha - Crowd Sourcing Origami Astronaut Protection
Placeholder
Brew Ha Ha - T-Rex's Prehistoric Power Walk
Placeholder
True or False with Kale Brock
Placeholder
The Grandfather Paradox
Placeholder
Brew Ha Ha - The Hidden Heroes Tackling Mozzies
Placeholder
Brew Ha Ha - Emergency AI Assistance
Placeholder
Brew Ha Ha - Frogs Forever, Dinosaurs Never!
Placeholder
Brew Ha Ha - Australia, Let's Go To Space
Placeholder
Brew Ha Ha - Welcome to the Microbiome, Archaea!
Placeholder
Brew Ha Ha - Roos Blindside Driverless Cars
Placeholder
Biodiversity of Antarctica Under Threat From Increase In Ice-Free Areas
Placeholder
Brew Ha Ha - The Future of the Census
Placeholder
Tell Me! Brian Cox
Placeholder
Crash, Burn, Tweak, Repeat
Placeholder
Brew Ha Ha - Humans Just Got Older and Wiser
Placeholder
Brew Ha Ha - Cheers to Brain Health?
Placeholder
Gene Therapy Could Cure Allergies
Placeholder
Brew Ha Ha - iHeart Hacking
Placeholder
Ridiculology - New Hubble
Placeholder
Brew Ha Ha - Trees Alone Can't Save Us
Placeholder
Brew Ha Ha - Earth's Accidental Force Field
Placeholder
Dinosaurs on the Big Screen
Placeholder
Brew Ha Ha - Farewell MP3
Placeholder
Kids Beat Grown-ups on Pneumonia Vaccines
Placeholder
Brew Ha Ha - The Booger Conspiracy
Placeholder
FameLab 2017 National Final - Part 2
Placeholder
FameLab 2017 National Final - Part 1
Placeholder
2017 Budget Response
Placeholder
What Are Animal Weapons?
Placeholder
If You Love Both Art and Science, Be a Scientific Illustrator
Placeholder
Getting Personal With Skinks
Placeholder
Brew Ha Ha - CSIRO Email Leaks
Placeholder
FameLab 2017 Western Australia Semi-Final Highlights
Placeholder
Brew Ha Ha - New Hope for Premmies
Placeholder
Brew Ha Ha - Britain Goes Coal-Free
Placeholder
Brew Ha Ha - Naked Mole-Rats (SFW)
Placeholder
Brew Ha Ha - Easter Reminders
Placeholder
Meet Andrea Boyd - Space Flight Controller
Placeholder
Brew Ha Ha - Sperm Drug Smugglers
Placeholder
FameLab 2017 New South Wales Semi-Final Highlights
Placeholder
The Science of Fiction
Placeholder
Brew Ha Ha - Liquorice Poisoning
Placeholder
Brew Ha Ha - Crowdsourcing Science
Placeholder
FameLab 2017 Queensland Semi-Final Highlights
Placeholder
Brew Ha Ha - SpaceX Preps for Relaunch
Placeholder
Poly Cystic Ovary Syndrome Breakthrough
Placeholder
Brew Ha Ha - Surviving a Media Storm
Placeholder
Will This Aussie Team Win the Race to Create the Ultimate Malaria Vaccine?