What's Peer Review - START HERE
Posted: Fri May 07, 2021 4:00 am
Not Always Brutal But...
Peer review is a good thing. Earlier we spoke about the scientific method. Peer review is the part at the end, right before the circle begins again. Once a researcher has finished their experiment and written up a corresponding paper, that paper is submitted to the most appropriate scientific journal. Scientific journals are the Reader's Digest of the scientific/academic world. The paper must be meaningful, relevant, and well done, before a (reputable) journal will publish the paper. Then when that edition of the journal comes out (monthly or quarterly), other scientists will comment on the paper.
Commentary is not whether the other scientists (peers) personally liked the paper or study. The point of peer review is to thoughtfully CATCH MISTAKES, or anything that might have been missed or done better, and offer suggestions on what the next steps should be.
This is the way science advances. It's a good process.
However, woe is the researcher who does shoddy work. Scientists can turn into a clique of mean girls, around bad science, quicker than a ghost hunter can film a jump-scare. For real, anyone who has listened to skeptics has heard the laser-sharp roasting that goes on.
What can you do, though, let bad science go without rebuke? No.
Peer review can be brutal but that is no reason for the legitimate researcher to fear. The process is meant to be helpful and polite. Where it gets ugly is when unprofessional scientists put out terribly flawed material. And yeah, that does need to be called out.
Wikipedia - List of Scientific Journals
https://en.wikipedia.org/wiki/List_of_s ... c_journals
PAY-TO-PLAY
A term heard in skeptical critique of pseudo-science articles is "pay-to-play." The journals listed above, most if not all of them are exclusive. Like a letter to the editor of a newspaper, the editor has discretion on whether to print the submission.
Journals are very picky, to put it plainly. Reputable journals, those that are taken seriously as the custodians of mankind's collective knowledge, do not publish things like APP's paper or the yahoo selling "medicinal" clam juice on late night infomercials. They publish papers from the top scientists in the world, on discoveries that will change the world: golden rice. gravitational waves, organ transplants, etc.
Why is it important for you to know this?
Because some journals don't operate with such high standards.
APP's little studies on paranormal experiences could not get into Nature or The Journal of Applied Physics, whose recent headline read:
Read the abstract (summary) here. Really, look at it rite quick.
So you see, they would not accept our paper. (And that's okay because truthfully we aren't at that level.)
However, APP's 150-page People Appear Distressed by "Paranormal" Events could easily get published in a "pay-to-play" journal. All it would take is the hundred dollar, or whatever, fee to the editors. Anyone can get published in certain journals, for a fee.
In between the best and worst are journals that do have high standards even if they are not top-tier.
Redemption for Pay-To-Play?
With all the disorder of 2020 and 2021, there's been discussion of misinformation. The situation with pay-to-play journals pre-dates the current state of public confusion. Pseudo-science has existed longer than "good" science. After all, good science came along to point out the difference between what can be proven and what can't.
If any of those type journals want to begin on a path to good reputation, they would start by inviting peer review from established scientists. Some skeptics say we shouldn't even try to reach those people. But let us be apologists for those up and coming scientists who are, unfortunately, not quite there yet. Fees could be increased to hire a review board. CONSTRUCTIVE feedback could be given.
Yes, of course that already happens in a few paid journals, but not enough. They are not doing the authors any favors by letting them think ANYONE will take them seriously. All they'll get in return is mockery, not helpful advice on how to do better next time.
.
The most important part of peer review is integrity.
Reputation is everything in science and academia. A lot of scientists won't get within arm's reach of a paranormal topic because anyone who does will lose their credibility. All of their work on (for example) molecular biology - past, present, and future - will be viewed with the disclaimer that the guy ALSO believes in Bigfoot. Why is it that way...? Because science holds that if you are so easily fooled with "woo" (that's the skeptic term for paranormal and pseudoscience) they they must not be a good scientist in the first place.
Reputation is everything in science and academia.
Scientists must be able to show their work to their peers and be able to defend it as basically flawless.
For that reason, paranormal participants of APP should respect the participating scientists and skeptics. Try to think through your posts, questions, and evidence before posting. Let's give them something substantial.
As with the previous section, replication and the replication crisis, there's a pretty low ceiling before APP's expertise runs dry. If you are knowledgeable about academic journals, please consider sharing with us by adding your reply below. Thank you!
The end.
NOTE: The material in the "APP's Answers" posts sometimes seem elementary and simplistic yet at the same time long and drawn-out. Yes. It's the beginning foundation for the topic. If you already know about these things, by all means add your suggestions to the appropriate forums. Or just scroll on. There can be a steep learning curve for those not already familiar with these things. Our answers are a starting point.
If you're already fairly knowledgeable, still scan each section and SKIP down to the RESOURCES listed. Readers don't have to take our opinion. Each part has good external sources/links, so you can get more in-depth and intricate answers from real experts.
QUICK LINKS - Good Science
MAIN / Good Science
MAIN / OUR ANSWER 1. Short Answer
MAIN / OUR ANSWER 2. Critical Thinking
MAIN / OUR ANSWER 3. CT - Next Level
MAIN / OUR ANSWER 4. Vetting Sources
MAIN / OUR ANSWER 5. "Do Your Own Research"
MAIN / OUR ANSWER 6. The Scientific Method
MAIN / OUR ANSWER 7. Replication - Or it Didn't Happen
MAIN / OUR ANSWER 8. Peer Review Can Be Brutal
MAIN / OUR ANSWER 9. Methodology is Key
MAIN / What's Your Opinion?
Home
FEEDBACK & DISCUSSION
Share your knowledge. Don't be shy if you have a quality contribution. (But please be polite, and proofread.)
For comment about the post you're reading, just reply to it. OR - To add a new post of your own, use the "NEW TOPIC" button located at the bottom of every forum's list of discussions. The button has BRIGHT PURPLE text. (Generally hit the back button from any post you're reading, to get back to the main forum for that subject.)
If you have a nice chunk of GOOD INFORMATION, please choose to add a new post rather than a comment-reply. That way you'll generate comments of your own.
TIPS FOR POLLS: Polls appear on some topics, if the post's author has created one. They're open to ALL registered users. All APP-written polls are set to be ongoing and allow you to change your vote later. Feel free to comment your suggestions for improving any poll, ours or others', just be kind and helpful.
CREATING A POLL: When writing any post (not comment replies) the option for "Poll Creation" is in a tab below the "submit" button. You can always edit the poll later via editing the post itself and navigating back to that tab, but that sort of defeats the purpose if you change it after people have answered, right? To start, try to think through the answers and provide an option for all likely points-of-view. Lastly, the poll will display below the post after it's published. In "preview" mode it shows on top.
.
.
Peer review is a good thing. Earlier we spoke about the scientific method. Peer review is the part at the end, right before the circle begins again. Once a researcher has finished their experiment and written up a corresponding paper, that paper is submitted to the most appropriate scientific journal. Scientific journals are the Reader's Digest of the scientific/academic world. The paper must be meaningful, relevant, and well done, before a (reputable) journal will publish the paper. Then when that edition of the journal comes out (monthly or quarterly), other scientists will comment on the paper.
Commentary is not whether the other scientists (peers) personally liked the paper or study. The point of peer review is to thoughtfully CATCH MISTAKES, or anything that might have been missed or done better, and offer suggestions on what the next steps should be.
This is the way science advances. It's a good process.
However, woe is the researcher who does shoddy work. Scientists can turn into a clique of mean girls, around bad science, quicker than a ghost hunter can film a jump-scare. For real, anyone who has listened to skeptics has heard the laser-sharp roasting that goes on.
What can you do, though, let bad science go without rebuke? No.
Peer review can be brutal but that is no reason for the legitimate researcher to fear. The process is meant to be helpful and polite. Where it gets ugly is when unprofessional scientists put out terribly flawed material. And yeah, that does need to be called out.
Wikipedia - List of Scientific Journals
https://en.wikipedia.org/wiki/List_of_s ... c_journals
PAY-TO-PLAY
A term heard in skeptical critique of pseudo-science articles is "pay-to-play." The journals listed above, most if not all of them are exclusive. Like a letter to the editor of a newspaper, the editor has discretion on whether to print the submission.
Journals are very picky, to put it plainly. Reputable journals, those that are taken seriously as the custodians of mankind's collective knowledge, do not publish things like APP's paper or the yahoo selling "medicinal" clam juice on late night infomercials. They publish papers from the top scientists in the world, on discoveries that will change the world: golden rice. gravitational waves, organ transplants, etc.
Why is it important for you to know this?
Because some journals don't operate with such high standards.
APP's little studies on paranormal experiences could not get into Nature or The Journal of Applied Physics, whose recent headline read:
(What does that even say? I mean, it LOOKS like English... sort of...) .Transitional flaw size sensitivity of amorphous silica nanostructures analyzed by ReaxFF/SiO based molecular dynamics
Read the abstract (summary) here. Really, look at it rite quick.
So you see, they would not accept our paper. (And that's okay because truthfully we aren't at that level.)
However, APP's 150-page People Appear Distressed by "Paranormal" Events could easily get published in a "pay-to-play" journal. All it would take is the hundred dollar, or whatever, fee to the editors. Anyone can get published in certain journals, for a fee.
In between the best and worst are journals that do have high standards even if they are not top-tier.
Redemption for Pay-To-Play?
With all the disorder of 2020 and 2021, there's been discussion of misinformation. The situation with pay-to-play journals pre-dates the current state of public confusion. Pseudo-science has existed longer than "good" science. After all, good science came along to point out the difference between what can be proven and what can't.
If any of those type journals want to begin on a path to good reputation, they would start by inviting peer review from established scientists. Some skeptics say we shouldn't even try to reach those people. But let us be apologists for those up and coming scientists who are, unfortunately, not quite there yet. Fees could be increased to hire a review board. CONSTRUCTIVE feedback could be given.
Yes, of course that already happens in a few paid journals, but not enough. They are not doing the authors any favors by letting them think ANYONE will take them seriously. All they'll get in return is mockery, not helpful advice on how to do better next time.
.
. APP Steps .
The paper we just mentioned, that behemoth 150-page, seven-appendix monstrosity, People Appear Distressed by "Paranormal" Events is open for peer review. We have created a forum dedicated to that, and hope to get some good suggestions for methodology tweaks before processing the remaining 90% of the study queue.
Perhaps for APP's second paper, once we get some peer review from self-publishing here on this website, maybe we could get a mention in a middle-tier publication.
But NO, under no circumstances would we pay to play. Our credibility already starts at zero (or less depending on the skeptic you ask) but we hope to earn respect as a junior contender. Buying into an academic publication is considered bad form right now. So at this time,
.
..
.
The most important part of peer review is integrity.
Reputation is everything in science and academia. A lot of scientists won't get within arm's reach of a paranormal topic because anyone who does will lose their credibility. All of their work on (for example) molecular biology - past, present, and future - will be viewed with the disclaimer that the guy ALSO believes in Bigfoot. Why is it that way...? Because science holds that if you are so easily fooled with "woo" (that's the skeptic term for paranormal and pseudoscience) they they must not be a good scientist in the first place.
Reputation is everything in science and academia.
Scientists must be able to show their work to their peers and be able to defend it as basically flawless.
For that reason, paranormal participants of APP should respect the participating scientists and skeptics. Try to think through your posts, questions, and evidence before posting. Let's give them something substantial.
As with the previous section, replication and the replication crisis, there's a pretty low ceiling before APP's expertise runs dry. If you are knowledgeable about academic journals, please consider sharing with us by adding your reply below. Thank you!
The end.
NOTE: The material in the "APP's Answers" posts sometimes seem elementary and simplistic yet at the same time long and drawn-out. Yes. It's the beginning foundation for the topic. If you already know about these things, by all means add your suggestions to the appropriate forums. Or just scroll on. There can be a steep learning curve for those not already familiar with these things. Our answers are a starting point.
If you're already fairly knowledgeable, still scan each section and SKIP down to the RESOURCES listed. Readers don't have to take our opinion. Each part has good external sources/links, so you can get more in-depth and intricate answers from real experts.
QUICK LINKS - Good Science
MAIN / Good Science
MAIN / OUR ANSWER 1. Short Answer
MAIN / OUR ANSWER 2. Critical Thinking
MAIN / OUR ANSWER 3. CT - Next Level
MAIN / OUR ANSWER 4. Vetting Sources
MAIN / OUR ANSWER 5. "Do Your Own Research"
MAIN / OUR ANSWER 6. The Scientific Method
MAIN / OUR ANSWER 7. Replication - Or it Didn't Happen
MAIN / OUR ANSWER 8. Peer Review Can Be Brutal
MAIN / OUR ANSWER 9. Methodology is Key
MAIN / What's Your Opinion?
Home
FEEDBACK & DISCUSSION
Share your knowledge. Don't be shy if you have a quality contribution. (But please be polite, and proofread.)
For comment about the post you're reading, just reply to it. OR - To add a new post of your own, use the "NEW TOPIC" button located at the bottom of every forum's list of discussions. The button has BRIGHT PURPLE text. (Generally hit the back button from any post you're reading, to get back to the main forum for that subject.)
If you have a nice chunk of GOOD INFORMATION, please choose to add a new post rather than a comment-reply. That way you'll generate comments of your own.
TIPS FOR POLLS: Polls appear on some topics, if the post's author has created one. They're open to ALL registered users. All APP-written polls are set to be ongoing and allow you to change your vote later. Feel free to comment your suggestions for improving any poll, ours or others', just be kind and helpful.
CREATING A POLL: When writing any post (not comment replies) the option for "Poll Creation" is in a tab below the "submit" button. You can always edit the poll later via editing the post itself and navigating back to that tab, but that sort of defeats the purpose if you change it after people have answered, right? To start, try to think through the answers and provide an option for all likely points-of-view. Lastly, the poll will display below the post after it's published. In "preview" mode it shows on top.
.
.