I still don't grasp why reproducing an experiment elsewhere is considered both a waste of time and is completely unpublishable. Reproducing an experiment in your laboratory at least once should be an absolute minimum requirement for publishing, but in many cases a single success out of tens or hundreds attempts leads to a publication. Madness.
While I don't agree with it, it's because if you do it again and get different results, you shooting yourself in the foot. The system rather you not repeat it, be wrong, and yet still get the paper. If you repeat it and get the opposite results, do you throw out the new data, throw out the hypothesis, or go against ethics and publish it anyway. Truthfully the university doesn't care if you are wrong, only that you publishing and contributing to the benefit of the University. You know how many papers are out there where the authors have told me in passing that they haven't been able to recreate their own data since? It's a number way higher than you would even guess. And frankly no one is going to repeat the experiments unless they tenured or have a really good reason to do so as part of another project. No one wants to be second place in Academia. It's one thing to be beaten to the punch. It's an entirely different thing to set out knowing what you doing has already been 'solved'. You are allocating resources to something which will not land you an impactful heat metric, and from a career standpoint, it' can be suicide.
I was unclear in my post. I understand why scientists do this on an individual level, but what I don't comprehend is why nothing is done to encourage reproducibility/reproducing on the collective level. The way science works now means that we likely have enormous amount of wrong and fake, as discussed in the video, data and results produced and published daily. And then people wonder why science is losing its authority...
Reproducibility is the cornerstone of hard sciences, because, well if you're here then you already know. But there are occasions where reproducibility isnt easy enough that one would allocate resources for, like astrophysics, so much of the time they criticize the data and come up with alternatives, then sometimes someone else tests those if not the original team. That's also a valid method where reproducing something might be difficult. However, something the recent room temp super conductor is a great example of reproducibility and verification, because there are hundreds of labs set up exploring the same technology, and it was easy to test.
This is how journal works. Publication is all about citation for its scientific merit which will increase journal citation index to justify its cost for publication which fund the journal. Repeat an existed study does not make any such merit for the journal.
not an academic, but i work in a STEM government job. the restructuring is absolutely partly by design. the other part is administrators and higher ups feeling the need to "make their mark" so to speak - it's simply not enough for them to steer the ship and ensure things continue running smoothly, they now have to enact large sweeping changes and leave a legacy. and if they screw it up, no big deal - they just move on to the next company/university.
The same goes for governments these days - only intent on making their statue in the square, not cleaning the streets or dealing with the piled up rubbish.
Nailed it. Admin mainly spends their time coming up with ways to waste faculty time. I need admin to collect tuition, pay the light bill, and leave me the $^ alone. Instead, I've got a thousand administrators above me, each coming up with brilliant schemes to take 10 minutes of my time. Oh, and buying new carpet to replace still-fine carpet.
Absolutely, they always need a complete redesign. It is near completion when the next lot arrive and off we go again. So many stuff-ups and so little accountability.
I am a severely depressed PhD student and most of the stress comes from my supervisor constantly putting me down for not being good enough and not publishing in Nature and whatever big journal there is. I can't wait to leave this hell. On the last point of your video, I have tried so many times to reproduce many studies in my field and half the time the articles do not even have the correct versions of the software they used to analyze the data, in some of them they are missing altogether. It is just disgusting, whenever I read a paper now, I cannot decide if to believe it or not. Whenever I see some gap in their logic I just stop... it is not worth the time.
Similar situation here, but in a different field (econ). Today, after years, I decided to drop out and pursue a career outside academia. Of course it stings my pride, but I'm sick of being miserable and borderline suicidal. I hope that things turn out for the better for you, and that you finish soon and move on to better things that make you happy!
I mean no offense, but is there even such thing as an economic science? I thought most of it was straight up propaganda with scientific wording? By science, I just mean describe the word and make measurable predictions that are used to further refine theories.
@@al8-.W No offense taken, it's a valid question! By your definition of science, then yes, since you define it so broadly and there is plenty in econ to study both empirically and conceptually. If we want to define it more strictly, it is a SOCIAL science, so fair arguments can be made about it not being scientific enough to qualify. Either way, we can't just collectively NOT study economics (my work specifically is in understanding economic regulation, which has provided me with amazing job opportunities that continue even after I decided to leave the program), so I've personally never been particularly hung up on whether it's seen as a science or not. It's knowledge, it's useful, and it's incredibly fun.
@@feefoofuu One day I made an argument that social sciences should be produced with our best minds available (probably physicists but you get the idea, I'm not targeting one field in particular, just a mindset and sheer degree of abstraction capacity). To me, social science has the potential to be at least as complicated as particle physics. Some of the reasons why I think this way: -Problems formulation can be daunting, especially if you're a human being and your employer has an agenda. -It's analogical to doing particle physics with broken instruments anyway because any feature you study is painfully extracted through a very noisy process (and the selection of metrics available to economists follow an agenda because the politics have an intuition of what you're going to measure and they want to be able to manipulate that so you can't point fingers at them). -The systems economists study are an emerging property from a complicated combination of what people tend to be naturally and their immediate environment. It's like trying to describe laws of physics in a word where every atom has a personality and are short lived. The data drift can be as extreme as people's habits changing just by the fact of your theory being out in the wild and altering beliefs about the world. That's without mentioning measures straight up becoming an objective, like gdp, etc... It's like trying to probe a quantum system without knowing it is quantum, plus the system itself is somewhat incentivized to fool you. I could go on, I have a very productive mind when it comes to epistemology.
So true, at least at the institutional level. Stanford and Harvard were smart for quickly acting on the research fraud allegations their people faced. Duke, by contrast, is looking pretty bad right now, with D@n Ari3ly’s situation-with so much evidence coming from all sorts of sources, including his alleged collaborators, possibly to put the entire body of his work into question. I hope they will reconsider their approach.
I’m currently working on a doctoral degree (not phd) and another way I see academia needing to move forward is in the area of styling. This ridiculous system that forces students to spend hours drooling over the correctness of their reference formatting versus leveraging that time to bolster the content is antiquated. It IS possible to give the same level of authorship credit without forcing students to jump through so many hoops.
I work with undergraduate and graduate writing. The abject fear students have of a misplaced comma on the reference page, when the emphasis should be on the quality of their writing/research, is unnecessary. Again, academia places the emphasis on things of little to no importance.
While journals' citations and styling requirements are often ridiculous, students don't need to jump through much (in terms of styling). Installing zotero and pressing APA7 once does not take hours. Similarly, requirements tend to be "Use almost default MS word settings". Also, it is of very little importance anyway and the "official styling guides" are regularly ignored by all parties.
Ai can already help with that, just send it the raw references and ask it to format it according to ur preferred citation style. I've been doing that and everything seems smooth@@joggerino3284
thanks for speaking the truth, Andy. I am a 3rd year PhD nursing research student at a large, well-known university on the West Coast in the U.S. Everything you’re sharing are things I’ve been intuiting the whole time in my program. I am by far the least ambitious student in our program and completely uninterested in the ego treadmill. My work is high quality, but I’m not willing to burn myself into the ground to play the game. Once I’m finished with my PhD I’m going into industry. Life is too short to deal with this bullshit. I say this with confidence as a hospice nurse as well.
Oh Boy, I worked for over a decade at a university: you're right. Upheaval and transformation were constant. We went back and forth: centralisation/decentralisation, grouping/regrouping of disciplines. Terrible insecurity, morale and a poisonous workplace. There was one constant: every change would involve a higher workload for staff, more work done by casuals and a decrease in value to the students. We would complete one radical reorganisation just in time to start another. I felt it was probably because the new executives needed to be seen to be acting upon the university in a dynamic, CEO kind of way. There were none with a matching desire to inform themselves or consult corporate memory, so they never realised that we'd all been here ten times before!
What you are saying is very important. I've had to resign from pursuing my PhD as my mental health suffered greatly, and I was ashamed and felt invalid that I didn't manage to achieve my academic goals. I still have regrets about it sometimes. It feels better when I see that the problem was systemic, and I wasn't alone who had bad experience with academia. Your message is so important, and I hope that future generations of scientist will have better experience with their scientific careers than I had.
At this point, I really wonder if there are any jobs that aren't shit. I'm someone who dropped out of my masters because of medical and financial reasons, and is slowly making progress with my health. But I'm disillusioned. Everything seems to have major drawbacks that can fuck you up.
Farmer in a third world country that never got to the point of farmers getting in debt to buy gmos, pesticides and a fleet of trucks bigger than the ship fleets you see in star wars movies seems decent enough. As long as you don't starve of course.
I agree. Andrew paints a very bleak picture of academia and there is good reason for this. However, I think other jobs are not without their problems. When I was young, people spoke about computers and AI freeing us from all of this work. In the future - and the year 2000 even was the distance future for me at this point (there was a TV programme called Space 1999 that was super futuristic!) - we would also work three-day weeks. But the future's not what it used to be. It didn't turn out that way. I think if you speak to people in the NHS (UK healthcare) or in sales, for example, you will hear similar stories. When email first came out I remember the excited of the 'ping'. You've got mail!! Fast forward a few decades and my list of more than 10,000 unread and opened emails is a dreadful and depressing (and constant) reminder that I cannot do everything I should be doing. There are just not enough hours in the day. So, yes, there are a lot of problems with academia. But it is still a job that I enjoy (though I am now a full professor and I appreciate the pressure on PhD students can be extreme). Other jobs have their issues so before people leap out of academic they should be careful that they are not leaping out of the frying pan into the fire.
That's how I feel. Every field I try to get into is horrifically flawed and has the most toxic people running it. It makes you just want to work for yourself.
It’s not just for grant research either. If you don’t meet research thresholds you will be given more teaching to do. This makes it more difficult to perform appropriate research and the pressure and downward spiral gets worse!
I would say, play stupid games win stupid prizes, but for some reason value is perceived in being a researcher at a top tier research university, even though it is quite clear what is actually required to achieve success. Some guy who has bought his way into the system or cranked out hundreds of useless papers is considered credible for giving scientific, medical, or consulting advice that sets the priorities of further research. Those that benefit from the system aren't going to reform it.
Exactly! Its the same logic that says a comment from a "student" is not very serious because theyre only a student - they couldnt POSSIBLY have enough background knowledge to ask very sensible and ultimately questions that are challenging the validity of their new claims.... Im 45 years old, in the classroom, (did explain my background) and a lecturer made a statement that I knew categorically to be now factually false, and when i questioned it and said "I thought that (insert xyz new facts here) was actually the understanding" being careful not to add the word "now" at the end... I didnt want to display his ignorance in front of all those soft first year students, just YET anyhow.... I was diplomatic. I heard he did fact check himself later on, but he never brought it up again.
I agree with everything you said. I would just say remember there are a few legit top tier researchers who do deserve to be there. Just not as many as are actually there. The ratio is probably 10/1 fraudsters to legit geniuses. For the record, I am neither, so I toil at a decidedly middle-tier institution, which is probably where I belong.
Another way of putting this: academia has to stop being competitive. Science isn't a competition. The idea that some people are inherently better than others at making scientific finding (assuming they received the necessary professional training) is very spurious. But even if it were true, you don't get the best people by creating a "competitive" bottleneck that only one or two in a hundred can pass through. Instead, you'll get those who game the metrics the best, or, in the slightly less troubling scenario, have some kind of head start (children of academics, for instance). Independent-minded young scholars, in particular, stand no chance in such a "competition". You don't get points for challenging the big shots in the field, do you?
The "Publish or perish" reward is not optimal. The researchers agents will naturally publish as much as possible without quality. The reward should be the total society impact and total new quality knowledge obtained. But measuring the true value of a good paper is hard I guess, to provide a reward based on that. But definitely there should be a better reward than number of papers.
This is a reinforcement learning problem. There are 3 ways that I know of: -discounted rewards : making sure earlier achievements are more valued than late achievements, to account for the compounding benefits of getting access to results earlier. -reward shaping : by introducing a proxy reward, we make it easier for the agent to learn while trying to stay true to our original goal. (this is the type of incentive we are trying to redesign there -if not eliminate altogether, I guess. Number of papers and citations is just a proxy that is user as a reward but it's just a veey lazy proxy) -advantage estimation. This is a more interesting kind of reward where the agent meeds to estimate the advamtage of a situation even in presence of underperforming "rewards", in anticipation for better rewards. I guess it's more aligned with long term tasks. If you ask me, abnegation should be the human answer but people need incentives in a world that is designed to erase you and make you poorer into oblivion. Even people who start out as decent people end up taking the same shortcuts, hoping that they can step out as soon as they got what they need (most of the time, enough money to live a decent life. Sadly, you simply cant save up money anymore so you just end up doing it until you retire at an old age)
One big problem is there is an underlying assumption that if a paper is valuable, it will be valuable immediately. It drives this sort of academic analogue to corporate next quarter earnings where a paper has a couple of years to take off or it's a bust, when the reality is that the true value of a discovery or a method sometimes doesn't become apparent until one or more other developments happen down the road, sometimes way down the road, and there's not much anyone can do to accelerate that
@@ClickBeetleTV The system must reward researchers to seek the long term gains! It is similar to politicians that get rewarded only for short term gain.
One of the largest problems is that academics often do not perceive themselves as part of the working class. The general administrative logic in academia aims to promote highly individualistic, even narcissistic, actors. In academia, you can find egregious examples of middlemen and administrative stupidity disguised as "innovation," which govern and make decisions that adversely affect us. While academics are smart, they can also be dumb. Their ultra-competitive egos make them easy targets for manipulation in an economic system where solidarity, collaboration, and articulation are not valued. The incentive system is generally flawed, and academics seem to be more interested in resenting their colleagues than in organizing (this can be more noticed in the Humanities). This issue is not just about complaining or who has the right to speak up; it's a matter of collective action. Waiting for some external intervention makes no sense at all. Social solidarity and collective defense of group interests are the only realistic ways to resolve this mess.
Great summary of our condition, but I'm pessimistic. "We can't ignore" except that we are. I think we will ignore these issues until the system collapses. Admin is focused on creating the illusion of academia - look at our first year program, look at our Olympic pool and hotel-style dorms, look at our cool logo. Whatever you do, don't look at our 4-year graduation rates, ROI, and for the love of the flying spaghetti monster don't look at any measure of learning! Until businesses understand that we are producing graduates with little to no skill and plenty of cognitive distortions that hobble them mentally, this will continue because the college premium on income will continue. I do think businesses are starting to wake up, though. It's my only hope.
It's really weird to me that in Academia hiring, they don't care who that person is, do they have leadership and management skills (they are literally leading a lab?), do they actually know how to train people to be independent researchers.
As you said, any metric can and will be exploited if that somehow secures your livelihood. That being said, I always thought that the number of citations your work gets is a better "long tail" indicator than the published articles per year. P.S.: Your channel is great. I have followed you for nearly half a year and appreciate your insights.
I am in the last year of my PhD studies. Due to scholarship deadline, I had to write my research article and my thesis in 5 months. I have to say, these have arguably been the worst 5 months of my life. I was traumatized by the immense pressure and the problematic communication with my supervisor. Everything, from the writing itself, the several rounds of corrections (usually just time-consuming minor figure and table adjustments), the paper submission and review, had a very huge impact on my mental health. I believe I will need months, even years to recover from that. It was a grueling process I never want participate in ever again, even though I really enjoyed doing the actual research (experiments and data analysis).
14 years in academia - 12 in clinical burnout (to differing degrees). Every day starts with terror - what grenades are lurking in my inbox? I’ve been very successful- lots of actual impact and strong industry relationships, 2 research fellowships and I Cannot Wait to resign!! It’s so toxic and I have it better than most.
After I earned my PhD, I have been an associate professor. However, I am considering leaving. The problem is, I am not sure where to take my behavioral science stuff (outside academia).
We just need a decentralized university. If no one does it I'll do it when I have the resources. Still need to grind my engineering career a little bit to keep myself off the streets in case my later projects go wrong 😅
I have been a non-academic researcher for years, you just have to take matters in your own hands, the less administrative limits the more you will achieve
@@TheOmegagoldfish Indeed, my recent (and brief) exploration of the possibilities has also led to that as the primary option. I have a genuine interest for research, but, I have little enthusiasm for teaching and overseeing classes, but it's a requirement.
@@CrucialFlowResearch Interesting. May I ask you two questions? How did you transition out of academia? Do you have any advice for non-academic researchers regarding where to explore opportunities?
Topic relevance that leads to solutions to problems or addresses gaps in knowledge or understanding of a research problem. During my lit review for my dissertation, I came across papers in "journals" that had grammatical and spelling errors. Having also gone through the research article peer-review process, I know the rigors of revision, and appeasing the referees. But the production of papers for sheer volume purposes, and the appearance of being active to keep tenure or funding is pretty silly. I read a quote on the weekend: scholars are the architects of the world. The question is: are they adding a new layer to knowledge, or are they fluffing the pillows of their own beds?
Yes it's possible to see all these problems in academia as part of a larger process of what the insightful folks at Juice Media call "enshittification."
Oh good God what does "capitalism" have to do with anything here? Do you think this would work better in the glorious communist utopia? You sound like some lazy, know nothing undergrad student trying to sound sophisticated. Capitalism is just a lazy way of saying "human nature sucks" @@AlexIncarnate911
I was a bit depressed last week, because I read the article titled "The natural selection of bad science", 2016. They first have a review which, this is my interpretation, concludes that academia currently is on the quantity side of the quantity - quality trade-off. Then they construct a model with the following assumptions: 1 Evaluation by quantity based metrics 2 All actors are ethical 3 All replication attempts are published The disheartening conclusion (as I understand it) is that a lower research quality/less rigor also applies to replication. Because if replication attempts are of the same standard as the original research, which is a reasonable assumption since it would correspond to a standard in a field, the failed replication attempts can be (validly) dismissed as being not rigorous. This of course ignores that the original research also was not rigorous. The only way to resolve the issue is a shift away from quantity based metrics to quality (and no metrics). But how to do that, I don't know. Edit: That is not to say that replication studies do not matter. They are crucial. It is just that that is not a silver bullet. Other measures also have to be taken.
Everything you mentioned in this video is correct, but you are ultimately wrong when naming the underlying cause of why the change doesn't and won't happen. It's not that we as a local or global community don't have the time to take to figure this shit out, it's that once one has climbed up the ladder and secured tenure, any and all incentive to change the system vanishes. They are now in position that the rest of the system works for them, because the academia is one huge pyramidal scheme and it makes me sick.
This competition amongst researcher really surprises me... Isn't the PHD prestigious eonugh? Doesn't that means you are already amongst the top 0.1% smartest people on earth, in your particular field? Olympics have a similar issues where the silver / bronze medals often go back home crying, because after all the effort you put in, getting anything less than number 1 means you failed.
Ever since I got serious about generative AI, I got a very disturbing, ever growing fear that deep fakes are going to populate an already saturated papers space. Even casually reading papers can be daunting for an engineer that only needs to quickly replicate results in a field supposedly eaaily replicable such as computer science (easy enoug, one would think, just download the code and the data, eh?). Well, not as much. With credible fake papers were facing a systemic risk here, more than the mere inconvenience we had to deal with in the past. We're moving to a world were scientists won't be able to trust each other and will have to reinvent the wheel every time because they won't be able to discern the new, genuine ideas. We're literally wasting decades of scientific progress while still paying the price for top notch research, multiplied by the number of labs in existence.
I am going to watch this in a bit, but I will say-in the US at least-the tenure system needs to be extensively reformed, if not completely abolished. That simple 😹 change would go a long way toward remediating some of the more egregious issues.
I wonder whether some restructuring happens because we can see what is suboptimal with the current structure, but not with the alternative. So we change it, gaining some efficiency, but losing some as well. The planned merger of Uni of Adelaide and UniSA - There will be some areas that improve, some that get worse, some that save expense, some that add them. I fail to see how one massive institution would be significantly better than the two.
Spot on. I realized this 20 years ago. I was actively involved in my own research and wanted to write it up. I already knew how, having advised PhD students, but no I had to complete a masters first and then a doctorate. I got 80% through the masters program when I ditched it because the quality of research, of the supervision and even access to the journals was very limited. Since then, I’ve been doing my own research privately and applying it commercially.
I'm an exhigh school teacher who was unreasonably canceled in 2008. I then released the years of pressure cooker strain by getting into trucking while the FIRST Robotics team I founded became a national power house. So, I find hearing your description of administrative systemic failure fascinating. It is point by point parallel to the delusional system I faced and battled for years while it seemed everyone else were cowed into silence. It could take a book full of irony to describe, but I can boil it down into a sentence. This principal violated my speration agreement so that he could of crow over my defeat at the start of school general meeting as warning to terrorize the remaining staff. This was how I accidentally stumbled upon a potential metric for admin disfunction... Investigate the system's decimation rate.
As long as we're trying to survive under capitalism, Academia cannot be any different. Like all the other structures of current society. Love your vids, the keep me sane! ❤❤❤
I have frequently seen two ways to pad one's publication list. At least one big-name classical historian I know published the same article in three or four different languages - German, French, Italian, English - and listed them separately, though the actual work done and the actual theories presented were the exact same. Then, a few years later, came revisions of the same article that at best take the content forward, but that really should not count as new work. The man in question has a publication list in three figures,and constantly rising. I do not say he would not be an eminent scholar anyway; he certainly knows his field and at least some of his views are original. But he games the system so shamelessly, it's a wonder there hasn't been any kind of exposure or scandal. As for large disruptive change, a close relative of mine who is a senior academic who should by now be secure in her work, given her very real achievements, her international reputation, and her long term of teaching, has been kept under pressure for years, along with her whole institution, by a hideous sequence of reorganizations and occasional threatened mass sackings that may have lasted about ten years. As for "we cannot rely on internal investigations," boy, have you hit a painful subject. I don't want to say more than that this kind of behaviour destroyed the career I hoped to have in history.
I completely agree, As an italian PhD, postdoc and now researcher I have experimented the same. But how shall we change this or at least make Europe or our own ministry of the University aware of that? my feeling is that it is impossible to change it locally because the wide system is so well sedimented
I couldn't agree more. Making a video of a process or sharing the data leads to losing the idea and someone will take the idea from you and ending in publishing results, which happened several times... the system is corrupted unfortuanately!
I would propose an S-index: an index not of citations but attributions of research to the betterment of society. Example: a researcher writes a paper that has some tangible impact on society as evidenced by its use by decision-makers. This would change the imperative from simply getting an ego boost by an H-factor to pointing to the usefulness of research that has an impact on society.
Great channel. One aspect you might consider covering is the nexus between politicians and academia. Nepotism, bum kissing, political interference for purposeful obfuscation and avoidance of investigation, "honorary" "fellowships", "doctorates", "professorships".....the revolving door. Not typically STEM, but these ex-pollies become VERY senior, extraordinarily highly paid staff members. In the US, often university presidents.
Sad fact is that the ABC in Australia is a problem. James Cook University Reef Research got questioned for systemic method issues and in less than about 2 months was an authoritive article source again...
I would love to have gone in to research. I had my first paper published with just a bachelors degree. But I decided to leave academia even without a masters because I can see how inefficient and under compensated the system is. I would be surprised if academia doesn’t have a people problem soon younger people are slowly realising the reality of it. In a system with a similar amount of fairness to industry it would have been a dream come true for to become an academic. I know that I will end up in a better place personally now I am an engineer in industry
Hi Andy. Ay Have Just Finished my PhD in neuroscience! I am applying to postdoctoral positions… and I am also applying to the industry in data science and data analytics… the cumulative information I have found in your channel and throughout my experience has convinced me that the best path for me is not in the academia anymore. Thanks a lot. Yes I agree academia is completely broken and you can’t imagine how broken it is in developing countries (I have work and studied in both in the US and in South America, both are bad)
Again, brilliant esp. restructuring. I am an academic at a top UK university and yep, just after suffering the shock of COVID and working to the bone to put our degree programs online, and then revert back again to some sort form of blended horror in 2022 forwards....what happens? A complete restructure and merging of faculties.
I think there's real value in using shared governance models for academia. Academic timetables are very long, and making significant structural changes every couple years does more harm than good. Shared governance should encourage greater consensus-based decision making which should be less susceptible to vacillating between objectives and goals. There are certainly situations where dramatic changes are necessary but these should be very rare events.
I know it's a different setting, the same has happened in schools in the UK - the kids and the communities are so far down the priority list - ofsted, grade numbers, shoving facts down the kids' throats, Trust managers' salaries then the kids ! As soon as the motivation is a figure you lose whatever it was we were aiming for in the first place.
Gosh - a depressing picture Andy. I agree with much of your analysis though. Certainly academic life has become a lot more stressful. I have become a full professor in a decent UK university. But I often say to people that were I, say, 25 years old now I am not sure I could succeed in the system. The demands of the job have changed so much over the decades. I did my PhD in the 1980s. The h-index did not exist. I am not sure citations existed. Certainly nobody spoke about them. I was under no pressure to publish during my PhD and I didn't publish - though I did publish a paper from my PhD a few years later. How things have changed! There are many problems with the h-index; not least that it doesn't work very well in the arts and humanities where journals are often not even the primary place where academics publish. However, the research assessment exercise (the UK government runs this about every 7 years) becomes increasingly reliant on metrics with each cycle. My understanding is that this is, in part, because alternative systems are too expensive/onerous. But we do need a way to evaluate academics. So what would you propose as an alternative method? Have you produced a video about that? Or, if not, would you consider making one?
Left Academia with a Masters to a 2.000 employees pharma company. In my team, there's a girl with "only" a Bachelors, another guy with a Masters and a guy with a PhD. We all do the same stuff and get paid the same base wage with a factor that increases with job experience. The organigram does not show the titles. That's the future, folks.
There's a guy I work with who's got an engineering masters, literally wrote the book on his discipline. Gets paid less than a project manager with a bachelors in English literature. We work for an engineering firm. In meetings it's clear she knows fuck all about engineering or construction. The simplest things have to be explained to her multiple times. She just delegates everything and arranges meetings. But she is paid far more than all the incredibly competent engineers because she belongs to the project management discipline who are on a different pay structure. Its all meaningless, just do what makes you happy.
Thank you for making these videos. I droped out of my masters due to the poor quality of 'per-reviewed' papers. For exsample. I was doing data analytis and I noticed papers use a complcated looking furmula in stead of computor code. Then claim this furmula is what they did. It Kinda reads like they are setting them selfs up for. 'OO you did'nt the same result as me. Well clerly you did'nt.....'.
The easy steps towards reproducibility are: - Make all datasets publically available - Make scripts for the data analysis publically available and well-documented Unfortunately, even these two simple steps cannot be applied equally to all the fields and types of studies. For example, many papers use huge datasets which should be stored and downloaded by reviewers. Some other papers use restricted data like some satellite products. Similarly, papers using commercial software cannot be checked in a similar manner. I would also understand why some developers might be hesitant to share all their work for a single publication. Yet, we can continue in this direction.
I was told not to have a fancy time on my first day at university. I thought we are going to have a conversation about research, grants, teaching etc. I was totally puzzled. What kind of people work in academia ?
Its mental to me that unis hire people per project, for every project on contracts that last only as long as that leg of the contract lasts. They dont just have a pool of people (that people come and go from) which gets distributed between projects as projects come and go. Its such a waste of people.
Well, Mr. squeaky wheel, after more than 40 years of this it's time to actually do something about it. I don't suggest making a video. As Buckminster Fuller used to say, "Reform the Environment". “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” ― Buckminster Fuller 1988: In Mendel Sach's book, "Einstein vs. Bohr",Joseph Agassi wrote about graduate students having nervous breakdowns before they could adjust to being research scientists rather than scholars. 1991: Robert Root-Bernstein mentions the paper race in his book, "Discovering: Inventing and Solving Problems at the Frontiers of Scientific Knowledge" In the 1990's there was a lot of talk, talk, talk about peer review and the grant application process. Dr. David Hestenes, PhD wrote an interesting essay on peer review. Robert O. Becker wrote a book stating that he wanted the public to know about what was actually going on in the medical research field.
the culture of quantity over quality is this way because universities are run by businessmen. They're basically half-privatized, and it took hard fighting to prevent it from being even more privatized. read dark academia by peter fleming if you havent
Great video! You speak directly out of my heart! Thanks Dr. Stapelton. Keep doing these great videos! I really love your content and you helped me alot during my first phase of my current PostDoc!
Built on exploitation of students, as well. As a Physics major, it became apparent teaching never meant that much. but we were told how important that field of study was so vital. If so, the entire curriculum would have been altered decades ago....
It is all so sad..... We should just create a platform when everyone can publish their paper for free. And to publish you need to review 3 papers. That would take care of the academic who refuse to review. And would make science accessible. If everyone stops publish in these journals then they won't hold any meaning or power. Next, put administration that can actually understand research in university so that they can hire professors based on their competence rather than on the number of publication. Is there anything else ? Let me know.
It needs to be changed from the bottom up, not the top down. It will not change until students stop publishing with these journals. They use our tax dollars to fund the research, the students pay tuition, fees, and more, and then literally give up the copyright to their work because the publisher owns the work because they plublished it. Its such a stupid system. Copyright is absolutely broken. Earn prestige for actually accomplishing real work, not because you published with a "reputable publisher" for whatever rationale has been argued for. Its against everyones interest. Its in the interest of science, society, and individuals to have open access which is what it was supposed to be before businesses like elsevier, springer, and others created a mob like moat around it all.
From my own experience as a researcher, my coleagues were to complient with the situation. They allways refused to complain even to the minor things not upset the 'bosses'. They were a bunch of selfish 'yes men' and didn't hesitate to bring their fellow researchers down. If there is to be any change, researchers themselves have to modify their posture and become more combative. BTW, you yourself are too soft and nice! At the root of academia's problems is capitalism. Someone has to have the courage to admit the obvious and face the elephant in the room.
the non yes-man get filtered out, with so much competition, it is possible to just throw the others out and still have way to many interested in the same position.
Hi Andy, I absolutely love your channel and can relate to at least large parts of it . I am (was?) a teaching lecturer in the social sciences at a Russell group university (one of the best the university had in pure metrics), which no-one ever seemed to care about, and I got made redundant anyway in what was essentially a departmental restructure. I really enjoy and relate to your videos but notice you don't really ever talk about teaching, which is kind of half (at least in theory), of what a university is intended to do. I would be very interested to hear a little bit more of your thoughts on teaching in general in academia.
The Academics... so smug and sure, with over inflated egos, peer reviewing each other with pomp and spite, living in an echo chamber of supreme intellect... where else can such a person thrive but in the campus of greatness known as Academia?
I liked the ideas at 10:00 about reproducibility! It’s so true that we could be showing videos of the processes used. That would take the field in a different, transparent direction!
You mention "impact factor", from the dictionary 'c. 1600, "press closely into something,"' from Latin impactus'. I just happened to look it up previously because I started to notice all our political and media BSers were using the term. Everything in Britain apparently has an impact. Truly lala land.
This reminds me very much of the IT Managed Services industry, where engineers are explicitly made to compete with each other, and performance is judged by number of tickets closed, regardless of whether or not the issue was actually resolved. It's set up in such a way that it actively encourages corner-cutting, discourages competence and discernment, and creates a toxic culture of backstabbing and mutual distrust.
Working with low quantity proteins for blots is so frustrating when we run dozens of tests to try and find a reliable way to detect the protein without 10x the cost just for people to publish fake data and unreproducable garbage is frustrating
There is dire need for improvement. The trickle down effects of this is more than we could imagine. I just don't see academia succeeding in the future without a complete disconnect of financial incentive.
Papers are hugely important, they are the science; the problem is not with using them as metrics, the problem is using metrics; and individual performance reviews; and artificial scarcity to make academia a battlefield
Andy, honestly, I'm already getting triggered by watching some of your videos... I'm getting discouraged with the research, I've worked hard this year and it feels like I'm a thousand years behind other researchers.
Why there is not a system to value the actual quality of papers? Like another index much more valuable than H-index taking into account a lot of variables in science life
How do you value the actual quality of paper without checking if what they claim is reproducible in the first place? I totally agree with you though, such an index would be much more useful than then outdated H-index... Currently, the peer review process values the quality of the papers, and if some reviewers are serious about their job they will run your code and check if your software works (in my field at least), however in molecular biology for instance this is impossible, they just have to resort looking at gel images who are often faked and so on. What I would like to see (at least in huge journals) is trying to reproduce the experiments (at least once independently) before even allowing a paper to be published. Many of my molecular biologist colleagues are unable to replicate certain results they see in published journals and they always shrug it off, saying probably doesn't work in our hands, I wonder if the reason is something deeper.
@@scienceguys9317 yeah I agree with you since I am becoming a Molecular Biology. However we need to figure out a way to change the system. It can't be so corrupted that scientists try to fake data in order to publish... also the money on the top are a big part of the problem
That paper mill site (which says for $5,000 they’ll put you as first author on a paper published by a major journal) is 100% guaranteed to be a catfish site! They’ll take your $5,000 and… give you nothing in return. What are you going to do, complain? To who?
Well, a good move would be to record all the necessary proofs and make a RUclips video out of it. If the paper gets published, you get to be a whistle blower who can possibly build a reputation for being a very honest person. If the paper doesn't get published you still get some credit for exposing a scam. Either way you have just lost 5k, what were you thinking, silly ? 😑
1. Publish as many papers as you can, no matter what quality so long as it is published. Or whether it actually important. 2. Get an R01 3. Go back to step 1. It stops being about the science.
By definition science has to be independent of the observer/experimentor. How come we got used to publishing reaserch with authors names at all? It seems to me this "personalization" is the root of all evil in academia.
The system is not perfect, but it works. You cannot deny the overall progress. If you have a plan to make it better, lots of people (including the funders) would be very excited to hear about it. Just adding more bureaucracy is not a viable solution. Allowing academics just do what they want at their pace, without evaluation, cannot a solution either. It is a zero-sum game after all. True research is a lot like gold-digging, where hard work is often not enough. You need some luck (and a good project , so choose wisely). You - an expert in your field - are never really sure what you will get; now imagine the funder's predicament...
Last year reviewed a paper for a journal that required the respective author’s contributions be listed. Its level of English was appalling, and there were no conclusions 🤔 Yet, one of the co-authors (who I know professionally) is a very senior Professor and institute head and native English speaker/writer - listed SPECIFICALLY therein as “submission review and proofreading”. The last OBVIOUSLY false. Indeed, I discovered this person sat on the grant review body that funded this work (in a foreign country). So they (a) steered “independently reviewed” grant money to a group they worked with, then (b) lied about their work on the publication. Obviously they just asked their name be tacked-on, and as they are well known, & the co-authors went along. But, it was, and is, clearly a lie & attempted academic fraud. But every single person reading this knows it happens ALL the time. Thank f*ck I left academia years ago. And I certainly sympathise with those still dwelling in its swamp. As you indicate, it is seriously warped.
I still don't grasp why reproducing an experiment elsewhere is considered both a waste of time and is completely unpublishable. Reproducing an experiment in your laboratory at least once should be an absolute minimum requirement for publishing, but in many cases a single success out of tens or hundreds attempts leads to a publication. Madness.
While I don't agree with it, it's because if you do it again and get different results, you shooting yourself in the foot. The system rather you not repeat it, be wrong, and yet still get the paper. If you repeat it and get the opposite results, do you throw out the new data, throw out the hypothesis, or go against ethics and publish it anyway. Truthfully the university doesn't care if you are wrong, only that you publishing and contributing to the benefit of the University. You know how many papers are out there where the authors have told me in passing that they haven't been able to recreate their own data since? It's a number way higher than you would even guess. And frankly no one is going to repeat the experiments unless they tenured or have a really good reason to do so as part of another project. No one wants to be second place in Academia. It's one thing to be beaten to the punch. It's an entirely different thing to set out knowing what you doing has already been 'solved'. You are allocating resources to something which will not land you an impactful heat metric, and from a career standpoint, it' can be suicide.
I was unclear in my post. I understand why scientists do this on an individual level, but what I don't comprehend is why nothing is done to encourage reproducibility/reproducing on the collective level. The way science works now means that we likely have enormous amount of wrong and fake, as discussed in the video, data and results produced and published daily. And then people wonder why science is losing its authority...
Reproducibility is the cornerstone of hard sciences, because, well if you're here then you already know.
But there are occasions where reproducibility isnt easy enough that one would allocate resources for, like astrophysics, so much of the time they criticize the data and come up with alternatives, then sometimes someone else tests those if not the original team. That's also a valid method where reproducing something might be difficult.
However, something the recent room temp super conductor is a great example of reproducibility and verification, because there are hundreds of labs set up exploring the same technology, and it was easy to test.
This is how journal works. Publication is all about citation for its scientific merit which will increase journal citation index to justify its cost for publication which fund the journal. Repeat an existed study does not make any such merit for the journal.
In physics it is a bit more rigorous. They say 5 sigma.
Whenever a measure becomes a target, it ceases to be a useful measure.
I've been looking for this saying for a while now! I remember reading it years ago (though it was worded a little differently).
Exactly.... change the that and you change the system... Papermill exist because of the incentives... change those and you will fix a lot of things...
not an academic, but i work in a STEM government job. the restructuring is absolutely partly by design. the other part is administrators and higher ups feeling the need to "make their mark" so to speak - it's simply not enough for them to steer the ship and ensure things continue running smoothly, they now have to enact large sweeping changes and leave a legacy. and if they screw it up, no big deal - they just move on to the next company/university.
The same goes for governments these days - only intent on making their statue in the square, not cleaning the streets or dealing with the piled up rubbish.
Nailed it. Admin mainly spends their time coming up with ways to waste faculty time. I need admin to collect tuition, pay the light bill, and leave me the $^ alone. Instead, I've got a thousand administrators above me, each coming up with brilliant schemes to take 10 minutes of my time. Oh, and buying new carpet to replace still-fine carpet.
Absolutely, they always need a complete redesign. It is near completion when the next lot arrive and off we go again. So many stuff-ups and so little accountability.
We need to reduce admin-HR, useless deans, sales, etc. they should not be the focus of researchers.
@@statisticaldemystic6817Haha. Loved the punchline!
I am a severely depressed PhD student and most of the stress comes from my supervisor constantly putting me down for not being good enough and not publishing in Nature and whatever big journal there is. I can't wait to leave this hell.
On the last point of your video, I have tried so many times to reproduce many studies in my field and half the time the articles do not even have the correct versions of the software they used to analyze the data, in some of them they are missing altogether. It is just disgusting, whenever I read a paper now, I cannot decide if to believe it or not. Whenever I see some gap in their logic I just stop... it is not worth the time.
Similar situation here, but in a different field (econ). Today, after years, I decided to drop out and pursue a career outside academia. Of course it stings my pride, but I'm sick of being miserable and borderline suicidal. I hope that things turn out for the better for you, and that you finish soon and move on to better things that make you happy!
I mean no offense, but is there even such thing as an economic science? I thought most of it was straight up propaganda with scientific wording? By science, I just mean describe the word and make measurable predictions that are used to further refine theories.
@@al8-.W No offense taken, it's a valid question! By your definition of science, then yes, since you define it so broadly and there is plenty in econ to study both empirically and conceptually. If we want to define it more strictly, it is a SOCIAL science, so fair arguments can be made about it not being scientific enough to qualify. Either way, we can't just collectively NOT study economics (my work specifically is in understanding economic regulation, which has provided me with amazing job opportunities that continue even after I decided to leave the program), so I've personally never been particularly hung up on whether it's seen as a science or not. It's knowledge, it's useful, and it's incredibly fun.
@@feefoofuu One day I made an argument that social sciences should be produced with our best minds available (probably physicists but you get the idea, I'm not targeting one field in particular, just a mindset and sheer degree of abstraction capacity). To me, social science has the potential to be at least as complicated as particle physics. Some of the reasons why I think this way:
-Problems formulation can be daunting, especially if you're a human being and your employer has an agenda.
-It's analogical to doing particle physics with broken instruments anyway because any feature you study is painfully extracted through a very noisy process (and the selection of metrics available to economists follow an agenda because the politics have an intuition of what you're going to measure and they want to be able to manipulate that so you can't point fingers at them).
-The systems economists study are an emerging property from a complicated combination of what people tend to be naturally and their immediate environment. It's like trying to describe laws of physics in a word where every atom has a personality and are short lived. The data drift can be as extreme as people's habits changing just by the fact of your theory being out in the wild and altering beliefs about the world. That's without mentioning measures straight up becoming an objective, like gdp, etc... It's like trying to probe a quantum system without knowing it is quantum, plus the system itself is somewhat incentivized to fool you.
I could go on, I have a very productive mind when it comes to epistemology.
Well… your supervisor is shit. He is the one not good enough, not you.
If an art museum admits to having a forged painting it calls into question their entire collection. Academia seems to have the same problem.
So true, at least at the institutional level. Stanford and Harvard were smart for quickly acting on the research fraud allegations their people faced. Duke, by contrast, is looking pretty bad right now, with D@n Ari3ly’s situation-with so much evidence coming from all sorts of sources, including his alleged collaborators, possibly to put the entire body of his work into question. I hope they will reconsider their approach.
Academia is a circus, these paper mills are actually proving it!
I’m currently working on a doctoral degree (not phd) and another way I see academia needing to move forward is in the area of styling. This ridiculous system that forces students to spend hours drooling over the correctness of their reference formatting versus leveraging that time to bolster the content is antiquated. It IS possible to give the same level of authorship credit without forcing students to jump through so many hoops.
I work with undergraduate and graduate writing. The abject fear students have of a misplaced comma on the reference page, when the emphasis should be on the quality of their writing/research, is unnecessary. Again, academia places the emphasis on things of little to no importance.
Hopefully this can be done by AI in future.
@@joggerino3284It has been done by reference management systems for years already
While journals' citations and styling requirements are often ridiculous, students don't need to jump through much (in terms of styling). Installing zotero and pressing APA7 once does not take hours. Similarly, requirements tend to be "Use almost default MS word settings". Also, it is of very little importance anyway and the "official styling guides" are regularly ignored by all parties.
Ai can already help with that, just send it the raw references and ask it to format it according to ur preferred citation style. I've been doing that and everything seems smooth@@joggerino3284
thanks for speaking the truth, Andy. I am a 3rd year PhD nursing research student at a large, well-known university on the West Coast in the U.S. Everything you’re sharing are things I’ve been intuiting the whole time in my program. I am by far the least ambitious student in our program and completely uninterested in the ego treadmill. My work is high quality, but I’m not willing to burn myself into the ground to play the game. Once I’m finished with my PhD I’m going into industry. Life is too short to deal with this bullshit. I say this with confidence as a hospice nurse as well.
Oh Boy, I worked for over a decade at a university: you're right. Upheaval and transformation were constant. We went back and forth: centralisation/decentralisation, grouping/regrouping of disciplines. Terrible insecurity, morale and a poisonous workplace.
There was one constant: every change would involve a higher workload for staff, more work done by casuals and a decrease in value to the students. We would complete one radical reorganisation just in time to start another. I felt it was probably because the new executives needed to be seen to be acting upon the university in a dynamic, CEO kind of way. There were none with a matching desire to inform themselves or consult corporate memory, so they never realised that we'd all been here ten times before!
It’s literally a capitalist industry now
What you are saying is very important. I've had to resign from pursuing my PhD as my mental health suffered greatly, and I was ashamed and felt invalid that I didn't manage to achieve my academic goals. I still have regrets about it sometimes.
It feels better when I see that the problem was systemic, and I wasn't alone who had bad experience with academia.
Your message is so important, and I hope that future generations of scientist will have better experience with their scientific careers than I had.
finally I find a person who "weirdly" has largely similar thought as me which consistently questioning this kind of world
At this point, I really wonder if there are any jobs that aren't shit. I'm someone who dropped out of my masters because of medical and financial reasons, and is slowly making progress with my health. But I'm disillusioned. Everything seems to have major drawbacks that can fuck you up.
Farmer in a third world country that never got to the point of farmers getting in debt to buy gmos, pesticides and a fleet of trucks bigger than the ship fleets you see in star wars movies seems decent enough. As long as you don't starve of course.
I agree. Andrew paints a very bleak picture of academia and there is good reason for this. However, I think other jobs are not without their problems. When I was young, people spoke about computers and AI freeing us from all of this work. In the future - and the year 2000 even was the distance future for me at this point (there was a TV programme called Space 1999 that was super futuristic!) - we would also work three-day weeks.
But the future's not what it used to be. It didn't turn out that way. I think if you speak to people in the NHS (UK healthcare) or in sales, for example, you will hear similar stories. When email first came out I remember the excited of the 'ping'. You've got mail!! Fast forward a few decades and my list of more than 10,000 unread and opened emails is a dreadful and depressing (and constant) reminder that I cannot do everything I should be doing. There are just not enough hours in the day.
So, yes, there are a lot of problems with academia. But it is still a job that I enjoy (though I am now a full professor and I appreciate the pressure on PhD students can be extreme). Other jobs have their issues so before people leap out of academic they should be careful that they are not leaping out of the frying pan into the fire.
That's how I feel. Every field I try to get into is horrifically flawed and has the most toxic people running it. It makes you just want to work for yourself.
It’s not just for grant research either. If you don’t meet research thresholds you will be given more teaching to do. This makes it more difficult to perform appropriate research and the pressure and downward spiral gets worse!
News for you, you'll be asked to do more of each.....continually. Until you become an administrator - then.....none.
I would say, play stupid games win stupid prizes, but for some reason value is perceived in being a researcher at a top tier research university, even though it is quite clear what is actually required to achieve success. Some guy who has bought his way into the system or cranked out hundreds of useless papers is considered credible for giving scientific, medical, or consulting advice that sets the priorities of further research. Those that benefit from the system aren't going to reform it.
Exactly! Its the same logic that says a comment from a "student" is not very serious because theyre only a student - they couldnt POSSIBLY have enough background knowledge to ask very sensible and ultimately questions that are challenging the validity of their new claims.... Im 45 years old, in the classroom, (did explain my background) and a lecturer made a statement that I knew categorically to be now factually false, and when i questioned it and said "I thought that (insert xyz new facts here) was actually the understanding" being careful not to add the word "now" at the end... I didnt want to display his ignorance in front of all those soft first year students, just YET anyhow.... I was diplomatic. I heard he did fact check himself later on, but he never brought it up again.
I agree with everything you said. I would just say remember there are a few legit top tier researchers who do deserve to be there. Just not as many as are actually there. The ratio is probably 10/1 fraudsters to legit geniuses. For the record, I am neither, so I toil at a decidedly middle-tier institution, which is probably where I belong.
Kind of like it is the people in the federal government that control investigations that need to investigate themselves - not going to happen.
Another way of putting this: academia has to stop being competitive. Science isn't a competition. The idea that some people are inherently better than others at making scientific finding (assuming they received the necessary professional training) is very spurious. But even if it were true, you don't get the best people by creating a "competitive" bottleneck that only one or two in a hundred can pass through. Instead, you'll get those who game the metrics the best, or, in the slightly less troubling scenario, have some kind of head start (children of academics, for instance). Independent-minded young scholars, in particular, stand no chance in such a "competition". You don't get points for challenging the big shots in the field, do you?
The "Publish or perish" reward is not optimal. The researchers agents will naturally publish as much as possible without quality. The reward should be the total society impact and total new quality knowledge obtained. But measuring the true value of a good paper is hard I guess, to provide a reward based on that. But definitely there should be a better reward than number of papers.
This is a reinforcement learning problem. There are 3 ways that I know of:
-discounted rewards : making sure earlier achievements are more valued than late achievements, to account for the compounding benefits of getting access to results earlier.
-reward shaping : by introducing a proxy reward, we make it easier for the agent to learn while trying to stay true to our original goal. (this is the type of incentive we are trying to redesign there -if not eliminate altogether, I guess. Number of papers and citations is just a proxy that is user as a reward but it's just a veey lazy proxy)
-advantage estimation. This is a more interesting kind of reward where the agent meeds to estimate the advamtage of a situation even in presence of underperforming "rewards", in anticipation for better rewards. I guess it's more aligned with long term tasks.
If you ask me, abnegation should be the human answer but people need incentives in a world that is designed to erase you and make you poorer into oblivion. Even people who start out as decent people end up taking the same shortcuts, hoping that they can step out as soon as they got what they need (most of the time, enough money to live a decent life. Sadly, you simply cant save up money anymore so you just end up doing it until you retire at an old age)
One big problem is there is an underlying assumption that if a paper is valuable, it will be valuable immediately. It drives this sort of academic analogue to corporate next quarter earnings where a paper has a couple of years to take off or it's a bust, when the reality is that the true value of a discovery or a method sometimes doesn't become apparent until one or more other developments happen down the road, sometimes way down the road, and there's not much anyone can do to accelerate that
@@ClickBeetleTV The system must reward researchers to seek the long term gains! It is similar to politicians that get rewarded only for short term gain.
One of the largest problems is that academics often do not perceive themselves as part of the working class. The general administrative logic in academia aims to promote highly individualistic, even narcissistic, actors. In academia, you can find egregious examples of middlemen and administrative stupidity disguised as "innovation," which govern and make decisions that adversely affect us. While academics are smart, they can also be dumb. Their ultra-competitive egos make them easy targets for manipulation in an economic system where solidarity, collaboration, and articulation are not valued. The incentive system is generally flawed, and academics seem to be more interested in resenting their colleagues than in organizing (this can be more noticed in the Humanities). This issue is not just about complaining or who has the right to speak up; it's a matter of collective action. Waiting for some external intervention makes no sense at all. Social solidarity and collective defense of group interests are the only realistic ways to resolve this mess.
Academia is quite elitist, though they try to claim not to be.
Great summary of our condition, but I'm pessimistic. "We can't ignore" except that we are. I think we will ignore these issues until the system collapses. Admin is focused on creating the illusion of academia - look at our first year program, look at our Olympic pool and hotel-style dorms, look at our cool logo. Whatever you do, don't look at our 4-year graduation rates, ROI, and for the love of the flying spaghetti monster don't look at any measure of learning! Until businesses understand that we are producing graduates with little to no skill and plenty of cognitive distortions that hobble them mentally, this will continue because the college premium on income will continue. I do think businesses are starting to wake up, though. It's my only hope.
The similarities with the entirety of our civilization and it's imminent collapse is baffling.
@@al8-.W Some have argued that they are causally connected - the insanity of the academy leaked into the wider society.
It's really weird to me that in Academia hiring, they don't care who that person is, do they have leadership and management skills (they are literally leading a lab?), do they actually know how to train people to be independent researchers.
H-index is a very solid example of Goodhart's law
As you said, any metric can and will be exploited if that somehow secures your livelihood. That being said, I always thought that the number of citations your work gets is a better "long tail" indicator than the published articles per year.
P.S.: Your channel is great. I have followed you for nearly half a year and appreciate your insights.
I am in the last year of my PhD studies. Due to scholarship deadline, I had to write my research article and my thesis in 5 months. I have to say, these have arguably been the worst 5 months of my life. I was traumatized by the immense pressure and the problematic communication with my supervisor. Everything, from the writing itself, the several rounds of corrections (usually just time-consuming minor figure and table adjustments), the paper submission and review, had a very huge impact on my mental health. I believe I will need months, even years to recover from that. It was a grueling process I never want participate in ever again, even though I really enjoyed doing the actual research (experiments and data analysis).
14 years in academia - 12 in clinical burnout (to differing degrees). Every day starts with terror - what grenades are lurking in my inbox?
I’ve been very successful- lots of actual impact and strong industry relationships, 2 research fellowships and I Cannot Wait to resign!!
It’s so toxic and I have it better than most.
I hear you! - I am sure you deserve better.
After I earned my PhD, I have been an associate professor. However, I am considering leaving. The problem is, I am not sure where to take my behavioral science stuff (outside academia).
Advertising & marketing, if you want to sell your soul. I'm assuming it's human behavioral science?
We just need a decentralized university. If no one does it I'll do it when I have the resources. Still need to grind my engineering career a little bit to keep myself off the streets in case my later projects go wrong 😅
I have been a non-academic researcher for years, you just have to take matters in your own hands, the less administrative limits the more you will achieve
@@TheOmegagoldfish Indeed, my recent (and brief) exploration of the possibilities has also led to that as the primary option. I have a genuine interest for research, but, I have little enthusiasm for teaching and overseeing classes, but it's a requirement.
@@CrucialFlowResearch
Interesting. May I ask you two questions? How did you transition out of academia? Do you have any advice for non-academic researchers regarding where to explore opportunities?
Topic relevance that leads to solutions to problems or addresses gaps in knowledge or understanding of a research problem. During my lit review for my dissertation, I came across papers in "journals" that had grammatical and spelling errors. Having also gone through the research article peer-review process, I know the rigors of revision, and appeasing the referees. But the production of papers for sheer volume purposes, and the appearance of being active to keep tenure or funding is pretty silly. I read a quote on the weekend: scholars are the architects of the world. The question is: are they adding a new layer to knowledge, or are they fluffing the pillows of their own beds?
What industry doesn't have some human bullshit at this point.
Yes it's possible to see all these problems in academia as part of a larger process of what the insightful folks at Juice Media call "enshittification."
@@guybeauregardaka capitalism taking over the very foundation of institutions
Oh good God what does "capitalism" have to do with anything here? Do you think this would work better in the glorious communist utopia? You sound like some lazy, know nothing undergrad student trying to sound sophisticated. Capitalism is just a lazy way of saying "human nature sucks" @@AlexIncarnate911
I was a bit depressed last week, because I read the article titled "The natural selection of bad science", 2016.
They first have a review which, this is my interpretation, concludes that academia currently is on the quantity side of the quantity - quality trade-off.
Then they construct a model with the following assumptions:
1 Evaluation by quantity based metrics
2 All actors are ethical
3 All replication attempts are published
The disheartening conclusion (as I understand it) is that a lower research quality/less rigor also applies to replication. Because if replication attempts are of the same standard as the original research, which is a reasonable assumption since it would correspond to a standard in a field, the failed replication attempts can be (validly) dismissed as being not rigorous. This of course ignores that the original research also was not rigorous.
The only way to resolve the issue is a shift away from quantity based metrics to quality (and no metrics). But how to do that, I don't know.
Edit:
That is not to say that replication studies do not matter. They are crucial. It is just that that is not a silver bullet. Other measures also have to be taken.
Everything you mentioned in this video is correct, but you are ultimately wrong when naming the underlying cause of why the change doesn't and won't happen. It's not that we as a local or global community don't have the time to take to figure this shit out, it's that once one has climbed up the ladder and secured tenure, any and all incentive to change the system vanishes. They are now in position that the rest of the system works for them, because the academia is one huge pyramidal scheme and it makes me sick.
A professor with tenure can finally afford to stand up for his or her principles without easily being fired.
This competition amongst researcher really surprises me... Isn't the PHD prestigious eonugh? Doesn't that means you are already amongst the top 0.1% smartest people on earth, in your particular field?
Olympics have a similar issues where the silver / bronze medals often go back home crying, because after all the effort you put in, getting anything less than number 1 means you failed.
Ever since I got serious about generative AI, I got a very disturbing, ever growing fear that deep fakes are going to populate an already saturated papers space. Even casually reading papers can be daunting for an engineer that only needs to quickly replicate results in a field supposedly eaaily replicable such as computer science (easy enoug, one would think, just download the code and the data, eh?). Well, not as much. With credible fake papers were facing a systemic risk here, more than the mere inconvenience we had to deal with in the past. We're moving to a world were scientists won't be able to trust each other and will have to reinvent the wheel every time because they won't be able to discern the new, genuine ideas. We're literally wasting decades of scientific progress while still paying the price for top notch research, multiplied by the number of labs in existence.
Academia in the UK reflects for me the coal mining industry; using other people's money to produce something fewer and fewer people want.
I am going to watch this in a bit, but I will say-in the US at least-the tenure system needs to be extensively reformed, if not completely abolished. That simple 😹 change would go a long way toward remediating some of the more egregious issues.
Actually, tenure is the final thing standing in the way of the complete destruction of universities by Administrative and political churn.
I wonder whether some restructuring happens because we can see what is suboptimal with the current structure, but not with the alternative. So we change it, gaining some efficiency, but losing some as well.
The planned merger of Uni of Adelaide and UniSA - There will be some areas that improve, some that get worse, some that save expense, some that add them. I fail to see how one massive institution would be significantly better than the two.
Spot on. I realized this 20 years ago. I was actively involved in my own research and wanted to write it up. I already knew how, having advised PhD students, but no I had to complete a masters first and then a doctorate. I got 80% through the masters program when I ditched it because the quality of research, of the supervision and even access to the journals was very limited. Since then, I’ve been doing my own research privately and applying it commercially.
I'm an exhigh school teacher who was unreasonably canceled in 2008. I then released the years of pressure cooker strain by getting into trucking while the FIRST Robotics team I founded became a national power house. So, I find hearing your description of administrative systemic failure fascinating. It is point by point parallel to the delusional system I faced and battled for years while it seemed everyone else were cowed into silence. It could take a book full of irony to describe, but I can boil it down into a sentence. This principal violated my speration agreement so that he could of crow over my defeat at the start of school general meeting as warning to terrorize the remaining staff. This was how I accidentally stumbled upon a potential metric for admin disfunction... Investigate the system's decimation rate.
As long as we're trying to survive under capitalism, Academia cannot be any different. Like all the other structures of current society.
Love your vids, the keep me sane! ❤❤❤
I have frequently seen two ways to pad one's publication list. At least one big-name classical historian I know published the same article in three or four different languages - German, French, Italian, English - and listed them separately, though the actual work done and the actual theories presented were the exact same. Then, a few years later, came revisions of the same article that at best take the content forward, but that really should not count as new work. The man in question has a publication list in three figures,and constantly rising. I do not say he would not be an eminent scholar anyway; he certainly knows his field and at least some of his views are original. But he games the system so shamelessly, it's a wonder there hasn't been any kind of exposure or scandal.
As for large disruptive change, a close relative of mine who is a senior academic who should by now be secure in her work, given her very real achievements, her international reputation, and her long term of teaching, has been kept under pressure for years, along with her whole institution, by a hideous sequence of reorganizations and occasional threatened mass sackings that may have lasted about ten years.
As for "we cannot rely on internal investigations," boy, have you hit a painful subject. I don't want to say more than that this kind of behaviour destroyed the career I hoped to have in history.
I completely agree, As an italian PhD, postdoc and now researcher I have experimented the same. But how shall we change this or at least make Europe or our own ministry of the University aware of that? my feeling is that it is impossible to change it locally because the wide system is so well sedimented
I couldn't agree more.
Making a video of a process or sharing the data leads to losing the idea and someone will take the idea from you and ending in publishing results, which happened several times... the system is corrupted unfortuanately!
I would propose an S-index: an index not of citations but attributions of research to the betterment of society. Example: a researcher writes a paper that has some tangible impact on society as evidenced by its use by decision-makers. This would change the imperative from simply getting an ego boost by an H-factor to pointing to the usefulness of research that has an impact on society.
"Use by decision-makers" that would lead to more politically-motivated papers
The video depressed me, but to see a Brazilian researcher cited made my day. Cheers!
This is the BEST chanel about 'today' academia reality. It's now more a 'beam of knowledge' but 'game to produce more papers'.
Great channel. One aspect you might consider covering is the nexus between politicians and academia. Nepotism, bum kissing, political interference for purposeful obfuscation and avoidance of investigation, "honorary" "fellowships", "doctorates", "professorships".....the revolving door. Not typically STEM, but these ex-pollies become VERY senior, extraordinarily highly paid staff members. In the US, often university presidents.
Sad fact is that the ABC in Australia is a problem. James Cook University Reef Research got questioned for systemic method issues and in less than about 2 months was an authoritive article source again...
i found this out as soon as i started my phd
I dont think external control will amount to much. Rather, i'd like the incentives to lie to go away.
For the Love of money is the root of all evil
You rock it! Thanks for sharing your thoughts
I would love to have gone in to research. I had my first paper published with just a bachelors degree. But I decided to leave academia even without a masters because I can see how inefficient and under compensated the system is. I would be surprised if academia doesn’t have a people problem soon younger people are slowly realising the reality of it. In a system with a similar amount of fairness to industry it would have been a dream come true for to become an academic. I know that I will end up in a better place personally now I am an engineer in industry
Hi Andy. Ay Have Just Finished my PhD in neuroscience! I am applying to postdoctoral positions… and I am also applying to the industry in data science and data analytics… the cumulative information I have found in your channel and throughout my experience has convinced me that the best path for me is not in the academia anymore. Thanks a lot. Yes I agree academia is completely broken and you can’t imagine how broken it is in developing countries (I have work and studied in both in the US and in South America, both are bad)
Again, brilliant esp. restructuring. I am an academic at a top UK university and yep, just after suffering the shock of COVID and working to the bone to put our degree programs online, and then revert back again to some sort form of blended horror in 2022 forwards....what happens? A complete restructure and merging of faculties.
I think there's real value in using shared governance models for academia. Academic timetables are very long, and making significant structural changes every couple years does more harm than good. Shared governance should encourage greater consensus-based decision making which should be less susceptible to vacillating between objectives and goals.
There are certainly situations where dramatic changes are necessary but these should be very rare events.
This is the problem with money as an incentive.
I know it's a different setting, the same has happened in schools in the UK - the kids and the communities are so far down the priority list - ofsted, grade numbers, shoving facts down the kids' throats, Trust managers' salaries then the kids ! As soon as the motivation is a figure you lose whatever it was we were aiming for in the first place.
Gosh - a depressing picture Andy. I agree with much of your analysis though.
Certainly academic life has become a lot more stressful. I have become a full professor in a decent UK university. But I often say to people that were I, say, 25 years old now I am not sure I could succeed in the system. The demands of the job have changed so much over the decades.
I did my PhD in the 1980s. The h-index did not exist. I am not sure citations existed. Certainly nobody spoke about them. I was under no pressure to publish during my PhD and I didn't publish - though I did publish a paper from my PhD a few years later. How things have changed!
There are many problems with the h-index; not least that it doesn't work very well in the arts and humanities where journals are often not even the primary place where academics publish. However, the research assessment exercise (the UK government runs this about every 7 years) becomes increasingly reliant on metrics with each cycle. My understanding is that this is, in part, because alternative systems are too expensive/onerous.
But we do need a way to evaluate academics. So what would you propose as an alternative method? Have you produced a video about that? Or, if not, would you consider making one?
Left Academia with a Masters to a 2.000 employees pharma company. In my team, there's a girl with "only" a Bachelors, another guy with a Masters and a guy with a PhD. We all do the same stuff and get paid the same base wage with a factor that increases with job experience. The organigram does not show the titles. That's the future, folks.
There's a guy I work with who's got an engineering masters, literally wrote the book on his discipline. Gets paid less than a project manager with a bachelors in English literature. We work for an engineering firm. In meetings it's clear she knows fuck all about engineering or construction. The simplest things have to be explained to her multiple times. She just delegates everything and arranges meetings. But she is paid far more than all the incredibly competent engineers because she belongs to the project management discipline who are on a different pay structure.
Its all meaningless, just do what makes you happy.
Why not to delegate replications to bachelor and master students as they work towards they degrees? 🤔
Thank you for making these videos. I droped out of my masters due to the poor quality of 'per-reviewed' papers. For exsample. I was doing data analytis and I noticed papers use a complcated looking furmula in stead of computor code. Then claim this furmula is what they did. It Kinda reads like they are setting them selfs up for. 'OO you did'nt the same result as me. Well clerly you did'nt.....'.
This is 💯 correct, I work at a university, the system is not reaching its potential. ❤
The easy steps towards reproducibility are:
- Make all datasets publically available
- Make scripts for the data analysis publically available and well-documented
Unfortunately, even these two simple steps cannot be applied equally to all the fields and types of studies. For example, many papers use huge datasets which should be stored and downloaded by reviewers. Some other papers use restricted data like some satellite products. Similarly, papers using commercial software cannot be checked in a similar manner. I would also understand why some developers might be hesitant to share all their work for a single publication. Yet, we can continue in this direction.
I was told not to have a fancy time on my first day at university. I thought we are going to have a conversation about research, grants, teaching etc. I was totally puzzled. What kind of people work in academia ?
Very interesting standpoints you mentioned Andy. Well done
Its mental to me that unis hire people per project, for every project on contracts that last only as long as that leg of the contract lasts. They dont just have a pool of people (that people come and go from) which gets distributed between projects as projects come and go. Its such a waste of people.
Well, Mr. squeaky wheel, after more than 40 years of this it's time to actually do something about it. I don't suggest making a video. As Buckminster Fuller used to say, "Reform the Environment".
“You never change things by fighting the existing reality.
To change something, build a new model that makes the existing model obsolete.”
― Buckminster Fuller
1988: In Mendel Sach's book, "Einstein vs. Bohr",Joseph Agassi wrote about graduate students having nervous breakdowns before they could adjust to being research scientists rather than scholars.
1991: Robert Root-Bernstein mentions the paper race in his book, "Discovering: Inventing and Solving Problems at the Frontiers of Scientific Knowledge"
In the 1990's there was a lot of talk, talk, talk about peer review and the grant application process. Dr. David Hestenes, PhD wrote an interesting essay on peer review.
Robert O. Becker wrote a book stating that he wanted the public to know about what was actually going on in the medical research field.
Interesting you mention David Hestenes
Great video...
the culture of quantity over quality is this way because universities are run by businessmen. They're basically half-privatized, and it took hard fighting to prevent it from being even more privatized. read dark academia by peter fleming if you havent
Great video! You speak directly out of my heart! Thanks Dr. Stapelton. Keep doing these great videos! I really love your content and you helped me alot during my first phase of my current PostDoc!
Built on exploitation of students, as well. As a Physics major, it became apparent teaching never meant that much. but we were told how important that field of study was so vital. If so, the entire curriculum would have been altered decades ago....
It is all so sad..... We should just create a platform when everyone can publish their paper for free. And to publish you need to review 3 papers. That would take care of the academic who refuse to review. And would make science accessible. If everyone stops publish in these journals then they won't hold any meaning or power. Next, put administration that can actually understand research in university so that they can hire professors based on their competence rather than on the number of publication. Is there anything else ? Let me know.
Thanks for this
The current situation has almost completely turned me away from pursuing a phD but I don’t know what else to do. I love research.
Same 😢
I´m so megahappy I recently left academia.
"We need to sort this shit out!" Absolutely agree. Also, I feel the reason this shit isn't getting resolved is MONEY.
It needs to be changed from the bottom up, not the top down. It will not change until students stop publishing with these journals. They use our tax dollars to fund the research, the students pay tuition, fees, and more, and then literally give up the copyright to their work because the publisher owns the work because they plublished it. Its such a stupid system. Copyright is absolutely broken. Earn prestige for actually accomplishing real work, not because you published with a "reputable publisher" for whatever rationale has been argued for. Its against everyones interest. Its in the interest of science, society, and individuals to have open access which is what it was supposed to be before businesses like elsevier, springer, and others created a mob like moat around it all.
From my own experience as a researcher, my coleagues were to complient with the situation. They allways refused to complain even to the minor things not upset the 'bosses'. They were a bunch of selfish 'yes men' and didn't hesitate to bring their fellow researchers down. If there is to be any change, researchers themselves have to modify their posture and become more combative.
BTW, you yourself are too soft and nice! At the root of academia's problems is capitalism. Someone has to have the courage to admit the obvious and face the elephant in the room.
the non yes-man get filtered out, with so much competition, it is possible to just throw the others out and still have way to many interested in the same position.
Hi Andy, I absolutely love your channel and can relate to at least large parts of it . I am (was?) a teaching lecturer in the social sciences at a Russell group university (one of the best the university had in pure metrics), which no-one ever seemed to care about, and I got made redundant anyway in what was essentially a departmental restructure. I really enjoy and relate to your videos but notice you don't really ever talk about teaching, which is kind of half (at least in theory), of what a university is intended to do. I would be very interested to hear a little bit more of your thoughts on teaching in general in academia.
The Academics... so smug and sure, with over inflated egos, peer reviewing each other with pomp and spite, living in an echo chamber of supreme intellect... where else can such a person thrive but in the campus of greatness known as Academia?
Awesome analysis.
I liked the ideas at 10:00 about reproducibility! It’s so true that we could be showing videos of the processes used. That would take the field in a different, transparent direction!
You mention "impact factor", from the dictionary 'c. 1600, "press closely into something,"' from Latin impactus'. I just happened to look it up previously because I started to notice all our political and media BSers were using the term. Everything in Britain apparently has an impact. Truly lala land.
This reminds me very much of the IT Managed Services industry, where engineers are explicitly made to compete with each other, and performance is judged by number of tickets closed, regardless of whether or not the issue was actually resolved. It's set up in such a way that it actively encourages corner-cutting, discourages competence and discernment, and creates a toxic culture of backstabbing and mutual distrust.
Working with low quantity proteins for blots is so frustrating when we run dozens of tests to try and find a reliable way to detect the protein without 10x the cost just for people to publish fake data and unreproducable garbage is frustrating
Classic STEM problem, lucky I choose a PhD in History. This is the consequence of decades of pushing STEM in schools, and neglecting humanities.
There is dire need for improvement. The trickle down effects of this is more than we could imagine. I just don't see academia succeeding in the future without a complete disconnect of financial incentive.
Clearly, the review process is broken.
Reorgs are a constant at some companies. Particularly regulated monopolies like utility companies. 😂
Papers are hugely important, they are the science; the problem is not with using them as metrics, the problem is using metrics; and individual performance reviews; and artificial scarcity to make academia a battlefield
Andy, honestly, I'm already getting triggered by watching some of your videos... I'm getting discouraged with the research, I've worked hard this year and it feels like I'm a thousand years behind other researchers.
Nah man, dont let academica destroy your mind, be fierce and dont let administrators dictate your value
And it all came tumbling down
We investigated ourselves and found no wrongdoing
Why there is not a system to value the actual quality of papers? Like another index much more valuable than H-index taking into account a lot of variables in science life
How do you value the actual quality of paper without checking if what they claim is reproducible in the first place? I totally agree with you though, such an index would be much more useful than then outdated H-index...
Currently, the peer review process values the quality of the papers, and if some reviewers are serious about their job they will run your code and check if your software works (in my field at least), however in molecular biology for instance this is impossible, they just have to resort looking at gel images who are often faked and so on. What I would like to see (at least in huge journals) is trying to reproduce the experiments (at least once independently) before even allowing a paper to be published. Many of my molecular biologist colleagues are unable to replicate certain results they see in published journals and they always shrug it off, saying probably doesn't work in our hands, I wonder if the reason is something deeper.
@@scienceguys9317 yeah I agree with you since I am becoming a Molecular Biology. However we need to figure out a way to change the system. It can't be so corrupted that scientists try to fake data in order to publish... also the money on the top are a big part of the problem
That paper mill site (which says for $5,000 they’ll put you as first author on a paper published by a major journal) is 100% guaranteed to be a catfish site! They’ll take your $5,000 and… give you nothing in return. What are you going to do, complain? To who?
Well, a good move would be to record all the necessary proofs and make a RUclips video out of it.
If the paper gets published, you get to be a whistle blower who can possibly build a reputation for being a very honest person.
If the paper doesn't get published you still get some credit for exposing a scam.
Either way you have just lost 5k, what were you thinking, silly ? 😑
Where is the link to the News shown in the video?
Being an academic is being a content creator :D :D
Do you have a link to that C&EN article?
How can anyone be expected to do good science under such conditions?
1. Publish as many papers as you can, no matter what quality so long as it is published. Or whether it actually important.
2. Get an R01
3. Go back to step 1.
It stops being about the science.
By definition science has to be independent of the observer/experimentor. How come we got used to publishing reaserch with authors names at all? It seems to me this "personalization" is the root of all evil in academia.
The system is not perfect, but it works. You cannot deny the overall progress. If you have a plan to make it better, lots of people (including the funders) would be very excited to hear about it. Just adding more bureaucracy is not a viable solution. Allowing academics just do what they want at their pace, without evaluation, cannot a solution either. It is a zero-sum game after all. True research is a lot like gold-digging, where hard work is often not enough. You need some luck (and a good project , so choose wisely). You - an expert in your field - are never really sure what you will get; now imagine the funder's predicament...
Last year reviewed a paper for a journal that required the respective author’s contributions be listed. Its level of English was appalling, and there were no conclusions 🤔 Yet, one of the co-authors (who I know professionally) is a very senior Professor and institute head and native English speaker/writer - listed SPECIFICALLY therein as “submission review and proofreading”. The last OBVIOUSLY false.
Indeed, I discovered this person sat on the grant review body that funded this work (in a foreign country). So they (a) steered “independently reviewed” grant money to a group they worked with, then (b) lied about their work on the publication.
Obviously they just asked their name be tacked-on, and as they are well known, & the co-authors went along.
But, it was, and is, clearly a lie & attempted academic fraud.
But every single person reading this knows it happens ALL the time.
Thank f*ck I left academia years ago. And I certainly sympathise with those still dwelling in its swamp. As you indicate, it is seriously warped.