It is probably the most well-known problem in Academia: Publish or Perish, the pressure to write as many articles as possible and getting them printed in top journals. It creates a system in which the quality of research is subordinate to the quality of journals, measured in impact factor, in which the research is published. Your quality and value as a scholar is determined not by what you do, but whether your articles add up to some arbitrary number that is supposed to represent quality. We all know it does not, but numbers like impact factor and h-index, which represents how often your work gets cited, provide a very simple picture and that is what committees want. But as bad as it sometimes seems, and can be, in Western universities, I had not realized how easy we have it compared to some Chinese universities, where if you want a career as an academic, you really cannot have a work-life balance. Work is life and life is work.

Let's start by looking at my own academic career. As a social scientist, I'm obviously not representative of the pressure in the STEM fields (Science, Technology, Engineering, and Math), but it provides a good point for comparison. I got four years to write my dissertation, but with a 0.9fte contract. This obviously did not mean I did not work full-time: I never measured it but I estimate that my average work week exceeded 40hrs. In some extremely busy periods I could spend close to 60 hours on my research. But that seems pretty acceptable; it's no different than in non-academic fields, and I finished my PhD within the alloted time; in fact I submitted the final manuscript four months before the deadline.

But I did my PhD at the University of Groningen, and while it is a top 100 university, the pressure is obviously going to be less than at an elite university. Fortunately, I got a job at Oxford, one of the best academic institutes in the world, particularly in the social sciences and humanities. And indeed, I do spend a lot more time on research now than I used to. But I don't teach anymore, so my work week is actually no different than during my PhD. I barely work more than 40 hours a week, and I don't often have to do research in the weekend. I'm juggling more projects, and as a post-doc I'm more efficient and effective in the way I spend my time, but all in all, I can easily manage a work-life balance.

So how does that compare to a top Chinese instituet like Tsinghua. If you've never heard of it, that's no shame. It has a strong focus on the STEM fields, and let's be honest, most of us will think about Western universities like MIT, Stanford, or Princeton when we think about elite technological institutes. But in fact Tsinghua published more papers in the most cited papers in math and computing in the past few years than all these American institutes. Which is insane if you consider that only a decade ago the university did not rank among the top 50 in the world!

Of course, such a radical change requires a radical investment. Part of the foundation for its success is that Tsinghua has embraced the tenure track system, where you are basically on probation for the first six years, after which you either get a permanent position or let go. The basis for determining whether you get tenure is your publication record, which means to get a job you have to work very hard for six years straight. I've heard stories of how big the pressure can be for tenure at US institutes, but apparently at universities in China like Tsinghua it's not uncommon to work days, nights, weekends, and everything in between, in order to get into the top journals and guarantee tenure. 

An additional benefit is financial compensation. Chinese universities pay bonuses for published articles. This can apparently go up to $165,000 for a single article if it's published in a journal like Nature. While plenty of research has shown that financial stimuli don't necessary lead to quality work, it does provide an environment in which it is attractive for great scholars to stick around. In Western society you can always go into industry, where you can make a lot more money than in academia. But if the pay is good, that's one fewer reason to switch.

While clearly these incentives work well for the reputation of Chinese universities like Tsinghua, there are obvious downsides. The people willing to sacrifice six years of their life are not necessarily the best academics. It also creates an environment where it becomes harder to compete for Western universities, particularly those outside the anglosaxon system in the UK and US. My alma mater got its first Nobel Prize in more than sixty years when Ben Feringa was awarded the prestigious award. Part of the reason the university, and its counterparts in Europe and beyond, gets so few awards is that it cannot compete financially. I don't blame universities like Harvard or Oxford for their financial success, but getting a brand new lab and a salary that's triple what you get is a good incentive to move on: the money also represents opportunity after all.

The result may be comparable to what we see in modern day football (the non-American kind). At the start of a season you can  predict the sixteen clubs that will compete in the round of sixteen of the UEFA Champions Leauge, because they are nearly always the same. The leagues in countries like England, Spain and France have been ruined by the billions that oligarchs have brought with them. Talented players are acquired by clubs before they are out of puberty, and the murderous competition can ruin them. Their countries of origin don't benefit, they don't benefit, and the sport does not benefit.

We should take care that in order to remain competitive, we don't move to a system where young academics have publish papers before they can even start a PhD, and then have to spend all their time researching and writing. Quality research is not the result of stress and pressure; scholars should have some freedom to pursue their own interests at their own pace. Many groundbreaking discoveries were made accidentally. That's not to say there shouldn't be competition, in academia as in industry competition helps create excellence, but the focus should remain on doing good research. And if universities like Groningen cannot compete financially, maybe we should focus on making it an attractive environment to work in, in other respects: like getting rid of that bloody publish or perish drive for academic success.

Most people who live in Western society are not scientists and have little to no scientific background. Yet we are all continuously confronted with science; whether it's about climate change, health, space exploration, social pscyhology, etc. Most news outlets have journalists on the payroll who translate those scientific findings into a language most of us can understand, but that does that actually make it easier to read science? Not only is stuff lost in translation, but scientific journalism relies a lot on some understanding of how science works, an understanding most readers will not have.

The problem with scientific literacy is particularly salient in the area of personal health and diets. Every day there is a new study that suggests that alcohol is either the cure for everything or should be avoided completely. As a lay person, it is basically impossible to get from the news what you should and shouldn't eat. And there is good reason for this, because while studies can suggest trends, every individual has his or her own dietary needs. Just because something works for one, ten, or a hundred people, offers no guarantees that it will work for you.

So if you read about health and food, how should you do that? I thought the following Guardian column can provide a nice illustration. My take-away as a non-dietician is that saturated fat and cholesterol is bad, and that you should ignore people like Nina Teicholz who claim that eating fatty products is actually good for you. A keto diet, meaning a diet high in fat, is clearly the worst thing you could ever do, and people like Jordan Peterson who propose an all meat diet are lunatics who should be put away.

Now, I am not here to say which one is right. Again, I'm not a dietician, I'm a linguist. But that also means I'm a scientist, and I can recognize bias in the debate when I see it.

So what is wrong with the column? Let's start with the obvious: it is a column. That means it is not written to be an objective discussion of a field of research, but an opinion piece. Indeed, the author, Sarah Boseley, has written a book herself on why obesity is a problem. That obviously does not mean she's not right, in fact, it means she is an expert in the field and should have something interesting to tell. But that obviously would apply to the people who's work she argues against. This is not about information, it's about persuasion.

But that's not the real problem. The way in which she frames the debate is as one between a fringe group of science denying idiots and the scientists who've worked for decades in the field and who you should listen to. But this is clearly not a valid argument. She notes that much of the work published about why fat is good, is published in small journals, whereas her camp publishes in the top journals. But all that shows is that her side has control fo the debate, and has for decades.

As much as we scientists may wish to believe we're above bias, we're not. Challenging the status quo is hard, if not impossible, if you haven't already made a name for yourself. There are plenty of examples in the history of science where new and revolutionary ideas were oppressed for decades, before it turns out that the scholars who made the claim were right all along. As long as papers have to be reviewed by the scholars who represent the status quo, and as long as those people decide what is good science, it's going to be hard to get published in major journals.

She also makes another error in this, in that it assumes big journals are better than small journals. Academics are fighting tooth and nail to get right of the ludicrous system of impact factors. When a journal has a high impact factor, when papers in that journal get cited a lot, that must mean it's a good journal. But impact factors don't provide that much information. It's an average. Even in the top journals of science like Nature many papers may receive few to no citations. And papers in small journals may have a far more significant impact. Just because something is published in a big journal, does not mean it's good science—unfortunately.

So to properly understand her column, you need to know how science works. Most people don't, and that's fine, because why would they?! But journalists then shouldn't assume that they do. Now aside from the scientific methodology, there's the issue of the actual science. A large part of her column revolves around cholesterol and whether LDL is bad. For decades science has linked it to heart disease, and saturated fat causes the liver to produce LDL. Sounds simple enough, but cholesterol is more complex than that and should be analyzed on more micro level; dealing with the different molecular structures. But that's way too technical for a general audience. How it all works is thus ignored in service of the larger point. Again, that does not mean it's wrong, but it means you get a limited picture.

As a non-scientists, you don't know these things. Even people who have finished an undergraduate degree have a hard time reading science critically, and they will trip over these problems. And of course, none of what I write here means Ms Boseley and the scientists she is defending is wrong. But the entire point of that 'small group of dissident scientists' as Boseley calls them, is that established science has been going about it wrong. So instead of seeing that as a critique of those other scientists, it's a natural state of a scientific debate, irrespective of who's right and wrong.

This brings us to a bigger problem: who to trust? Should we start ignoring climate change, because a scientist here and there claims that it's all a hoax? Clearly not. There is good reason for established science to be established science: evidence has confirmed it time and again. But as a non-scientist you need to be aware that much is lost in translation between the science and the more lay articles. The only thing you can do is try to be as informed as possible and keep an open mind. Which is basically what scientists (are supposed to) do as well.

If you’ve payed attention to the news, you could currently be fooled into thinking Academia is about to collapse under the weight of its own corruption. A small group of, let’s be kind and call them scholars, submitted a bunch of hoax papers in the past couple of years, some of which even got published. These studies focused particularly on issues of gender and race equality, and were in line with sometimes more extreme progressive positions. To them, as well as many right-leaning media, this was again proof that the so-called grievance studies were ideologically biased and deeply corrupt, and that gender studies, race studies, yes the whole discipline of sociology needs to be defunded and dismantled.
​
Now I haven’t read the papers—there’s a lot of them and I have better things to do with my time. (And I seriously doubt the nay-sayers have.) But quite frankly, one does not to read the articles to understand that the world is not moving towards a dystopic future of children indoctrinated and forced into believing they should identify as a non-binary gender, which would lead to the end of the family as the cornerstone of Western civilization.

Simply put, all these hoax studies show is that peer review cannot control for real or made-up research, but it can still control for quality research within accepted disciplines. Of the many studies that were submitted, most were either rejected, or given a revise and resubmit. The latter may seem like the articles were considered promising, but as any scholar know, R&R does not guarantee acceptance in the second or even third round. It can signify that there is still major work to be done, and articles can easily be rejected even when they’ve been resubmitted.

The fact that reviewers were kind in their critique is also meaningless; we expect others to review in a supportive way, as we would do ourselves. Research is a collaborative, not an individual project. We’re a community, and we should act as part of that community. And I know that there are reviewers that can be overly harsh, but the fact is that I’ve never seen a review that was cruel, hostile, or in any way rude. So it’s no surprise that even hoax papers receive fair reviews.
​
As a researcher who is asked to review a paper, there are a number of questions you have to answer, primarily among them (1) does the research make a useful contribution to the field? and (2) is the research methodologically sound? If both conditions are met, then you have to write either a really bad paper for it to be rejected outright, or you have to submit to a journal that accepts very few papers. But in neither case do we gain any insight into the supposed ideological bias of the field.

​The problem is that these so-called scholars wrote up what they considered ideologically progressive articles and theories. As any undergrad who’s taken a basic methods course knows, that doesn’t prove the field is ideologically progressive, because you’re not actually controlling for that variable. All they’ve shown is that a scary large number of hoax papers (25%) found their way into actual publication. And while that is obviously bad, and we should try to find ways to do better, that happens in every field. In fact, it’s in fields like Medicine where the real problems still lie: that’s where we find the biggest frauds and cheats in Academia, not the Social Sciences.

In a sense, we’re simply dealing with a matter of Confirmation Bias. These authors already “knew” that these fields were biased, and so they went out to find evidence that would support that hypothesis. But again, that’s plainly bad research. A good scholar knows that his or her ideas and theories will cloud his or her analysis of the data. So one should not look for confirmation—also, that brings us to the issue of the white swan, seeing ten does not mean all swans are white—but falsification.

​And the fact is that they might actually have a good point. We do not want any field to be ideologically biased. We know that academia is largely left-leaning, particularly in the social sciences and humanities where there are ten liberals for any conservative. If fields like gender and race studies have a significant impact on society, then we obviously don’t want research in those fields biased towards certain theories. Science makes progress through debate and critique, not propaganda. And so we might indeed discss the value any field of research has for society and whether it should be funded by a government or public institute.

People cry foul when any industry, whether it’s food, tobacco, or finance, is given the right to evaluate and sanction itself. We need to hold academia to the same standard. But we need to do that in a constructive manner, and so we cannot rely on the opinion of ideologically driven scholars alone, whether they be liberal or conservative. If a field is worth funding, the people in it should be able to explain that to the funders, the public. And if they disagree, then maybe we have to rethink what type of research we are willing to fund through tax-payer money.

In many scientific fields number are the golden standard: if you cannot quantify what you're investigating, you're not doing (scientific) research. While I won't dispute the value of quantification, this has always struck we and my qualitative colleagues as a strange obsession. Numbers, after all, have no meaning in and of themselves. How do you know what you should quantify and what adequate means of quantification are. And once you've done quantification, how should you interpret those findings?

​In the hard sciences these problems have been largely addressed for centuries. Physics, genetics, astronomy: none of them have problems interpreting the meanings of numbers and combining qualitative and quantitative findings, because their goal is to produce unifying theories. In the social sciences and humanities on the other hand, most scholars are busy investigating causal mechanisms: if I put a pen in my mouth, will I be happier; if I put a graphic warning label on sigaret packages, will that cause fear; if there is dissonance between word and picture, will that affect speech production, etc. (Dr Trafimow provides a comprehensible explanation of these two different methods.)

The results of these studies may be very interesting—or they be entirely useless—but they do not contribute to a deeper understanding of the human condition. At least, not by themselves. Without a proper underlying theory, findings are nothing more than mere curiosities. One could build a digital museum and maybe do something interactive for children with them, but that's about the extent of their use.

There are of course disciplines in the Humanities and Social Sciences that are supported by proper theories. Generative syntax is a prime example. You can disagree with the field based on its findings or its maxims, but you cannot deny that one reason that it's been so productive for the past sixty years, is that there is an underlying theory that is continuously being refined or revised, and that is to account for human language. A proper theory makes the field inherently superior to many of its competitors that do nothing more than ad hoc analysis. (Although, to be fair, there is plenty of ad hoc analysis going on in generative syntax as well. But even the hard sciences are guilty of that sometimes.)

he problem with how to interpret numbers came to the forefront again this week. After the US Open final, in which Serena Williams claimed the umpire treated her unfairly because she is woman, the question was whether women are indeed treated more unfairly than men on the court. For some prominent feminists this was not a matter of discussion: Billy Jean King claimed that women who speak out on the court are 'hysterical' whereas men are just 'outspoken'. She, and others like even the WTA, saw the incident as a larger problem of sexism in tennis. And while I do not dispute that women are treated differently than men, I would like at least some form of evidence that this was not a one off.

The days following the incident were livened up by a bunch of anecdotal "evidence". It did not take long for men to point out that, contrary to claims by Williams' coach, Ramos has been just as strict to top men like Rafael Nadal. He has been one of few umpires who has repeatedly sanctioned Nadal for taking too long in between points. An infraction that is barely more sever than a coaching violation, for which Williams was punished. But of course, there were also cases where Ramos did not sanction men for behavior similar to that of Williams. Both sides claimed their points vindicated, showing just how useless anecdotal evidence is.

The New York Times attempted to resolve the issue definitively by investigating the past twenty years. They counted all the times men were penalized and compared that to all the times women were penalized. After correcting for the fact that men play more tennis—best of 5 instead of best of 3—and there are more men in the qualifications at all slams except the US Open—128 men vs 96 women—they claimed that men were still penalized more. Interestingly enough, the only exception was coaching, for which women were more often sanctioned.

​Should we conclude from these numbers that Serena, Billy Jean King, and the WTA were wrong? That women are not sanctioned more severely than men, and thus that Ramos' penalty was not a symptom of an underlying problem of sexism? The answer will not surprise you: it's a resounding no. While it's great that the NYT has gathered all these data, they are meaningless. And no statistical test in the world could give them meaning. The problem is that there may be a lot of underlying factors. Men may be more outspoken than woman are on the court.  There may be a few men, like Nick Kyrgios and Benoit Paire, both of whom have faced severe financial penalties and even suspensions for their behavior, who completely mess up the numbers. We simply don't know.

These kind of articles reinforce the methodological point that you cannot do quantitative analysis without also doing qualitative analysis (and vice versa). In this case, the numbers provided by the NYT would be a nice starting point to figure out if something is really going on—are men treated more unfairly?—and if so, what. Every case has to be appreciated for its individual relevance: no two sanctions are alike. That sounds like a massive amount of work, and it is, but without such an investigation, all we have are numbers. Just as without those numbers, all we have is a series of anecdotes. Real evidence comes in the form of theories that are both qualitatively and quantitatively supported.

Conversation Analysis is, to paraphrase Levinson, an odd duck in the Humanities and Social Sciences. Linguists often consider it either irrelevant because it doesn’t deal with “language proper” or because it’s not rigorous enough (read: we usually don’t do stats), whereas sociologists and psychologists tend to take the position that social interaction is not a particularly interesting subject and its findings are subjective anyway (read: we usually don’t do stats). Nonetheless, CA is a burgeoning field of research as was once again attested during the International Conference for Conversation Analysis, which was held at Loughborough this year. Five days – or seven if we include the pre-conference workshops – in which over 500 people from all over the world came together to discuss their work from sunup to sundown. And while quantity is of course not an argument for quality, the breadth and depth of the work presented was truly inspiring.

The fun started before the actual conference got well under way. For three days there were workshops to enjoy on a variety of subjects, ranging from fundamental CA concepts such as Preference and Sequence Organization to practical discussions about quantification and how to write good research proposals. All these workshops were taught by the foremost experts in the field, and they provided a great opportunity to expand your understanding of the subject, or to get a grasp on concepts with which you previously had limited familiarity.

I joined the action on the second day with a great workshop by Merran Toerien on decision making in medical interaction. The workshop addressed the two major themes that any scholar nowadays will have to deal with: on the one hand there’s the actual analysis that we as CA researchers are interested in, and on the other there’s the matter of how to communicate these findings to practitioners who have no knowledge of the field and may be highly skeptical of any conclusion that isn’t supported by numbers or “hard data”.

While one day is obviously never enough to get a good grip on any of these discussions, Merran – who’s enthusiasm makes any workshop all the more inspiring – made great use of the time, starting with a data session, then giving presentations of her own work, and allowing us to get some hands-on experience with actual data, and finally even putting us in the position of practitioners to show how she has in the past gone about trying to convey these findings to physicians themselves. I think we all left with a far better understanding of how to go about analyzing our data, and a renewed vigor to help improve actual medical practice.

My second workshop as on presenting non-English data, taught by Giovanni Rossi. Anybody who’s ever published or presented findings using non-English data knows the struggle: how do you present your recordings and your transcripts in a way that they can easily be understood by scholars who don’t speak the language, without it all taking up so much room that, in case of articles, there is not enough room for the actual analysis. Giovanni focused on best practices from his own experience with Italian.

As a PhD graduate from the MPI for Psycholinguistics, he dealt extensively with the linguistic side of things such as the Leipzig Glossing Rules, but we also got to talk about problems such as how to make sure people focus on the original language, not the English translation. While it’s too early yet for definitive answers, it’s clear that we are moving towards at least some standardization that should make this all a lot easier for all of us.

After the final workshops, we only had a few hours before the conference was officially opened and Jeffrey Robinson got us started with the first plenary lecture. And we couldn’t have gotten off to a better start. Jeff not only tackled one of the bigger questions in CA – Is there a preference for agreement with positively formulated interrogative polar questions? – but he also showed a truly rigorous way of doing analysis. The goal of CA should not just be to compare apples with apples, but Granny Smiths with Granny Smiths. This means that when we build a collection, we need to make sure that there is no unnecessary variation in the data that could influence the findings. One may think that he is taking lessons from experimental methods, and he may well have, but one still has to analyze each case independently, as a single case. He still focused on the normative organization of social interaction.

Jeff’s conclusion: there is no preference for agreement, but a preference for what he called non-conditioned answers, answers that go along with the agendas set in the question. The future will see whether it holds, but one cannot be anything but impressed by the analytic rigor of the methodology and Jeff in doing so demonstrated that there is no reason why CA cannot be considered just has hard a science as any other ground in experimental methods.

The four subsequent days were progressively more exhausting, the unnatural heat not being particularly helpful in rooms without AC and hundreds of hard working academics, but also great fun. The plenary speakers addressed engaging and inspiring topics such as eye rolls in everyday and public interaction (Rebecca Clift), deal making by physicians (Tanya Stivers), and even literary translation of particles using Lewis Carroll’s Alice in Wonderland (Marja-Leena Sorjonen). Particularly the latter may have gotten some skeptical looks at the beginning – at least from me – but Marja-Leena demonstrated that even conversation in literature, and indeed conversation wherever it can be found, can be a great source of data. You just have to ask the right questions.

There was so much to see at the actual conference that there is no way to provide a short discussion of all the interesting talks. But then again, with all the live tweeting that went on, I don’t think there is a reason to. I’ll just say this. It was great to see the diversity in the way people go about presenting their research. There was obviously the classic approach: people starting with a clear outline and going from an introduction, to their method/data, an analysis, and to wrap up with the conclusions. But that was certainly not the only way. Danielle Pillet-Shore must have taken her audience through twenty examples, if not more, taking the data as the heart and soul of her talk about what she called registerings.

I was also surprised by how many people presented completely written oud talks. It’s a practice I have rarely seen at earlier conferences, but at least half the talks, particularly by non-European scholars, were done in this manner. I often found these harder to follow. Presenting has to be engaging, it’s a form of entertainment, and that part often gets lost when a scholar just stands there, reading their presentation aloud. But it guarantees that presenters get to say what they want, exactly how they want to. And of course, presenting is a skill that takes time and practice to master, whether you read aloud, do it entirely from memory, or merely prepare a rough structure.

With the pre-conference workshops, long days of talks, and of course the heat, it was no surprise that by the final day everybody was pretty exhausted – the silent disco and karaoke the night before also weren’t conducive to a sharp mind. Nonetheless, plenty of people stuck around until the bitter end. (I skipped the closing to watch the World Cup final – Allez Les Blues!). And whether it was intentional or not, but by planning a number of great speakers, such as Candy Goodwin and Merran Toerien, for the final sessions it was easy to engage with the presentations despite the fatigue. It may not be ideal for the presenters, but I consider it a smart plan.

But exhausting it was nonetheless. Fortunately, none of this will be a problem at ICCA 2022; not only will it take place in winter – or what counts for winter Down Under – but we were promised there will be koalas(!), and they will undoubtedly wash any fatigue away. And they need to in order for the conference to top this one; it was superbly organized by a great committee and supported by a group of outstanding volunteers. And as always, it’s the people who make the conferences: the talks, discussions, lunches, dinners, and conversations with friends and colleagues are what make ICCA into what it is; a holiday for conversation analysts.