is scientific publication a discipline or a racket?

by el gato malo | Jun 6, 2024

as any system gets established and later ossified, those in and around it learn to game it. they optimize their behavior to drive success within the system and, many times, this has deleterious effects on the system itself, breaking it into something more akin to algorithm races as opposed to serving any useful purpose.

SEO (search engine optimization) is a classic example. google achieved dominance in the web search space and the web responded by making all their web pages into bizarre monstrosities targeted not so much at readers as at the google algo. it turns the search result increasingly into gibberish or worse, into annoying advertising forests full of aggravating content that simply will NOT tell you what you want to know. it keeps answers out of thumbnails, foregrounds sensationalist clickbait headlines, and buries even simple things like “recipe for lasagna” deep in stupid stories that have 9 paragraphs of text before the story. (useful fact: you can fix this by adding “reddit” to your search terms for recipes. try it. the results are striking.)

and this is why google is going to lose the search space. AI will eat them alive and it’s becoming manifestly clear that the bizarre, woke, untrained AI’s the googleplex are putting out are un-useful save as parody.

the results speak for themselves.

but this is the internet, the internet routes around damage, and it’s going to ultimately fix itself, albeit perhaps after a somewhat awkward adolescence.

but what of science publishing? can these same trends be of help to us there? because it sure seems like the space could use the help.

the great avenue for abuse in sci-pub has been “peer review.” this once well intentioned and perhaps well functioning practice has taken on weird totemic fixation among partisans and zealots.

they hold “peer review” forth like some sort of talisman, proof that whatever possibly religious documents you are about to wave around are the true and validated word of the almighty and constitute “the science which must be trusted.”

obviously, this was always a foolish notion and a grave misapprehension of the purpose of peer review. it was never meant to prove anything, it was just a first pass screen to see if methods and findings seems basically sound and thus ought be published in a journal or other such means of dissemination and validation. these publications were highly conscious of their reputations and thus sought to ensure that the content they put out was of high quality.

but none of them were claiming “peer review = true.” if that were the case, why publish at all? why have journals and competing, adversarial assessment and ideas? just put it in the textbooks and be done with it. once, the journals WERE the debate. they were stodgy things read by only a few folks in the field and everyone (mostly) understood that this was where we sought and hashed out truth. it was fuel for debate, not doctrine for dissemination.

but then the optimizers came and publication became both doctrinaire and big business. many journals were “captured” by ideologues who turned peer review into a form of ideological purity test and this found confluence with reliance upon government grants to perform the studies in the first place to the extent that entire fields became feudal fiefdoms working backwards from forgone conclusions to find ways to “prove” them because that’s what the money wanted. discovery became dogma.

and “peer review” became a potent tool to gatekeep this phenomenon. “we, the experts will only allow publication of that which agrees with the theory we have previously put forth” is where science goes to die. it’s stultification and stagnation. but it’s also VERY politically useful to a certain sort of technocrat seeking to gin up validation for policy choices. and so of course they seek, by sleight of hand, to make “peer review” synonymous with “proof” and every time someone on the internet demands “was it peer reviewed!?!” as though anything not so sifted could not plausibly have value, it simply digs the pseudoscientific pit of guild systems defending doctrine deeper.

past a point, it essentially substitutes “have the political commissars approved this?” for “scientific method” and alas, very few people seem to notice.

and bad things tend to happen to those who do.

this trend also gives rise to the awkward cousin of “politicization” this which is “total grift” and the same over-reliance and over-faith in peer review creates a business opportunity for if peer review is validation and validation is a valuable commodity, then someone is going to show up to sell it and, like bond rating agencies beholden not to those who buy bonds but rather to those who issue them (because, after all, they are the ones paying the fees) the incentives to ensure standards are perverse.

and as the very same AI that makes it so easy to catch university presidents for plagiarism spreads, the tide is going out and quite a lot of people are turning out not to have any pants on.

many lament this, but i cannot see why.

honestly, my first reaction was:

but once i got done with that, i got to thinking: “hey, this seems like a good thing!”

seriously, what even is the countercase? this is not science dying, this is science returning, science being restored and reinstituted. falsification IS science and if whole bodies and journals and industries and paradigms are garbage, then the sooner we take it out, the sooner we can get back to work.

link

meaningful parts of this space have become spam because spam paid. now we develop ad blockers and stop pretending that most peer review journals are currently more than adware viruses or that the imprimatur of “peer reviewed” matters a whit without knowing who the peers were. the peers of conmen and scammers are not exactly sterling references and the backrub circles of “you review mine and i’ll do hers and she’ll do yours” are mostly another type of circle that this is too polite a publication to name explicitly.

and this is hardly a wiley only issue.

the whole industry and vocation is under fire.

good.

this is the reputation economy at work and like most things, it should be a market, not an oligopoly and certainly not a dictatorship.

science that cannot be criticized can never be sound.

how would you even know?

if the claims stand up, then challenge makes them stronger.

if they don’t, then overturn them and move on.

adversarial debate is the only way and systems that evict you or refuse to fund you if you challenge then are the antithesis of scientific method.

peer review is, at best, a starting point, and in practice often a screening tool to ensure that we run in predetermined directions which turns science into mere doctrine.

that cannot be the way.

this is a good rock to have gotten rolling.

let’s help it pick up speed.