Giving Credit Where It’s Long Overdue

Digital scholarship helps researcher develop novel approach to measuring the impact of scholarly publications

Implicit bias — the attitudes and stereotypes that can affect our actions in an unconscious manner — is increasingly a hot topic in academic hiring and admissions. But what about its impact on how researchers cite each other’s work — or not — in the scholarly literature?

“Across many fields in academia there is evidence that women accrue fewer citations than men do — not because they’ve done less pivotal research, but because men are more likely to cite men. Now we can correct for that,” says Catherine Brinkley, an assistant professor of human ecology at UC Davis who uses social network analysis in the fields of urban studies and environmental science, in explaining her current project in partnership with the digital scholarship program at the UC Davis Library.

Brinkley had observed how underrepresented the contributions of Rachel Carson, the author of Silent Spring, and Jane Jacobs, an urban scholar who toppled conventional urban theory in the 1960s, are in the academic literature. Despite the fact that their works ultimately altered the course of environmental and urban studies, Brinkley noticed that their contributions are often side-stepped, with authors instead citing men who later built on their work, rather than crediting the women for their original insights.

She suspected that they were but two examples of a much more pervasive problem, and as a woman in academia herself, she was naturally concerned. After all, citations are a key factor in academic assessments of influence and impact. Yet without the tools to efficiently analyze the vast literature of their fields, Brinkley would have no way to test her hypothesis. She needed someone to write the code.

So, with Carson and Jacobs as the initial case studies, Brinkley enlisted the help of Carl Stahmer, who leads the library’s digital scholarship program, to investigate the issue.

Leveraging his experience with text mining and social network analysis, Stahmer worked with Brinkley to develop a plan. Using data from Google Scholar — in this case, a much more complete record of the scholarly literature in both the sciences and social sciences than sources like PubMed or Web of Science — they gathered over 100,000 citations of Carson and Jacobs’ works, and then began developing a process for analyzing these citations as a means of computationally following the trace of the ideas presented by the two authors.

The methodology Stahmer and Brinkley are developing to analyze this initial citation network will ultimately be applied to a much larger dataset, enabling them to trace the influence of Carson and Jacobs’ work even when they are not formally cited by name. According to Stahmer, “As we are able to gather more full-text data, we’ll be able to use text mining methods to see how the ideas of Jacobs and Carson are copied, built upon and evolve as they move from author to author over time.”

Their findings will have implications for both ecology and gender studies. But it’s the larger potential application of the quantitative method they’re developing that has Brinkley really excited.

“This approach could be applied to all citations,” she says. “It could be the new h-index — one that corrects not just for self-citation, but for all different types of biases.” In addition to gender bias, she notes, underrepresented racial and ethnic groups often also face marginalization and lack of recognition of their work in academia.

Without the library’s help, she adds, “it would have been very, very difficult to have done this project.”