Scientists know it is happening, even if they don’t do it themselves. Some of their peers are using chatbots, like ChatGPT, to write all or part of their papers.

In a paper published Wednesday in the journal Science Advances, Dmitry Kobak of the University of Tübingen and his colleagues report that they found a way to track how often researchers are using artificial intelligence chatbots to write the abstracts of their papers. The A.I. tools, they say, tend to use certain words — like “delves,” “crucial,” “potential,” “significant” and “important” — far more often than human authors do.

The group analyzed word use in the abstracts of more than 15 million biomedical abstracts published between 2010 and 2024, enabling them to spot the rising frequency of certain words in abstracts.

The findings tap into a debate in the sciences over when it is and is not appropriate to use A.I. helpers for writing papers.

When ChatGPT was introduced in November 2022, a collection of words started showing up with unusual frequency. Those words, the investigators report, were not used so often before the release of ChatGPT. They infer that the change in word usage is a telltale sign of A.I.

In 2024, there were a total of 454 words used excessively by chatbots, the researchers report. Based on the frequency of the A.I.-favored words, Dr. Kobak and his team calculate that at least 13.5 percent of all biomedical abstracts appeared to have been written with the help of chatbots. And as many as 40 percent of abstracts by authors from some countries writing in a few less selective journals were A.I.-generated.

Those numbers, said Dr. Adam Rodman, director of A.I. programs at Beth Israel Deaconess Medical Center in Boston, “are almost certainly a lower bound,” because they don’t account for human editing of what the chatbot wrote or the chatbot editing of what humans wrote. Dr. Rodman was not involved in the study.

In an interview, Dr. Kobak said he was “somewhat surprised” to see so much use of A.I. in abstracts, summaries of paper’s results and conclusions that often are the only things people read.

“I would think for something as important as writing an abstract of your paper, you would not do that,” he said. (Dr. Kobak and colleagues said no A.I.s were used in the writing of their paper.)

In the academic sciences, some researchers have grown wary of even a whiff of A.I. assistance in their publications.

Computer scientists are aware that A.I. favors certain words, although it’s not clear why, said Subbarao Kambhampati, a professor of computer science at Arizona State University and the past president of the Association for the Advancement of Artificial Intelligence. Some scientists, he said, have been deliberately refraining from using words like “delve” for fear of being suspected of using A.I. as a writing tool.

Other scientists seem blasé about the risk of being caught using chatbots.

Dr. Kambhampati gives some examples, like a case report in a radiology journal that includes: “I’m very sorry but I don’t have access to real time-information or patient-specific data as I am an AI language model.”

The journal Nature recently surveyed more than 5,000 researchers and asked when, if ever, is it OK to use A.I. to write a paper.

There was no consensus.

Opinions varied, depending on whether A.I. was used to write an abstract, or the entire paper, and whether it was used to edit or summarize.

For the situation analyzed in the new paper — writing an abstract — just 23 percent of the Nature respondents said it was OK to use A.I. without acknowledging the assistance. Some 45 percent said it was acceptable only if the researcher reported using A.I., and 33 percent said it was never acceptable.

“It’s all very ambiguous right now,” said Dr. Jonathan H. Chen, director for medical education in artificial intelligence at Stanford. “We’re in this gray zone. It’s the wild west.”

Sometimes it is difficult to see the hand of A.I., an issue that raises questions of whether an A.I.-generated submission to a journal should be rejected simply because there is no human author.

Keith Humphreys, professor of psychiatry and behavioral science at Stanford, says he was once tricked by a letter to the editor of the journal “Addiction.”

Dr. Humphreys, who is deputy editor of the journal, said he thought the letter about a recently published paper made reasonable points. As is the journal’s custom, he sent the letter to the authors of the paper to give them a chance to reply.

They told him that they had never heard of the authors of the letter, who were purportedly from China, and that “our field isn’t that big and no one has ever heard of these people,” he said.

The authors of the original paper also did a search to see what the letter’s authors had published. They found lots of publications — all letters and comments in a variety of journals in a variety of specialties, including cardiology, emergency medicine, endocrinology, gastroenterology, hepatology, immunology, intensive care medicine, psychiatry, surgery and urology.

“And this was all within the span of six months,” Dr. Humphreys said. “They had mastered every single field.”

It seemed quite likely that the group had run journal articles through a chatbot and had asked it to generate letters to the journals’ editors. Dr. Humphreys speculates that the authors might have academic positions in which their pay depended on a list of publications.

So he wrote to the letter’s authors telling them that, if they had used A.I., they had to say so.

The authors did not reply.

“We didn’t publish it,” Dr. Humphreys said.

“But the problem is, it wasn’t a bad letter. And it’s not fraud in the traditional sense of hurting science,” he added. “Should we deny everyone the chance to see it?”

One colleague said that since everyone has chatbots these days, why publish anything written by them? Anyone who wants to see such an analysis can generate one for themselves.

What about having a chatbot write an editorial making suggestions for what is needed for further progress in a field?

An A.I.-generated editorial might be prescient, Dr. Humphreys said. But, he added, a colleague had made a point that resonated with him.

“The reason I care about an editorial is the name on it. A leading researcher is willing to take the career risk,” he said, and say things that may be harsh or unpopular.

“A.I. has no meaning, no reputation. I have no trust in it,” Dr. Humphreys said. “It has no moral weight.”