Pitfalls of Peer Rerview

November 14th, 2011 by pcahn

In 2006, Nature tried an experiment. The journal receives about 10,000 manuscripts a year and sends 40% of them out for traditional peer review. In the trial, the editors asked authors if they would also submit their paper for open peer review where any scientists could leave signed comments. 71 authors agreed.

The journal promoted the experiment heavily on their website, through e-mail blasts, and with targeted invitations to scholars in the field. After four months, they reviewed the results. nature05535-i2Despite sizable web traffic to the site, 33 papers received no comments, and the most heavily commented on paper received only 10 replies.

Nor did the editors find the comments influential in their decisions whether to publish. They found that although many scientists approved of the idea of open review, very few would perform it.

Their experiment demonstrates both the promise and the pitfalls of social media. It opens up the possibility for dialogue, but it depends on self-motivated users to enrich the content.

LGBT Health

November 8th, 2011 by pcahn

In a survey of medical school curricula, researchers found that the average medical student receives only five hours of instruction on the health of gay, lesbian, and transgendered patients. Nine schools devoted no time to the topic during the preclinical years.

Doug Hughes, Assistant Dean of Diversity and Multicultural Affairs and Professor of Psychiatry at BU, served as a panelist at the AAMC meeting in Denver. He reported that BU students receive 10 hours of LGBT health instruction, twice the national average.

A reporter from U.S. News interviewed Doug, who reported on the strides that BU and other medical schools have made in welcoming LGBT students. The visible presence of supportive faculty is particularly important, he said.

A Crisis of Confidence

October 26th, 2011 by pcahn

Two theories have emerged to explain the low proportion of women in engineering:

  1. Family plans: women's desires to care for their family lead them to choose more flexible careers.
  2. Self-assessment: women's perceived deficiency in core mathematics skills lead them to opt for careers outside of science.

A group of researchers tested these theories with a longitudinal cohort of engineering students and found no evidence for them. Instead, they posit a third theory:

3.  Professional role confidence: women and men develop different levels of confidence in their ability to become  successful  members of their profession. F2.large

The study, published in the American Sociological Review, finds that family plans and self-assessment scores does not reduce female students' likelihood to stay in engineering. By contrast, confidence in professional expertise predicts career persistence.

The authors speculate that these results may explain why women in medicine cluster in more typically female-friendly specialties. Fortunately, programs can help address the gap in professional role confidence by exposing students to work internships so that they can build familiarity with the profession.

Minority Scholars

October 17th, 2011 by pcahn

A new NSF study finds that African-American recipients of Ph.Ds in science, engineering, and health fields are less likely than other minorities to work in research-intensive institutions. nsf11320

One possible explanation is that African-American scholars are more likely than other faculty to work at historically black colleges and universities, which are classified as master's level institutions.

Another contributing factor may be that African-American doctorates earned their degrees from universities outside the top tier of research intensity.

The data show, however, that overall minority doctoral recipients are increasing, which could lead to changes in faculty composition in the future.

PowerlessPoint

October 7th, 2011 by pcahn

SlideWe've heard the rules for using PowerPoint effectively:

  • 7 lines per slide
  • 7 words per line
  • No flashing animation

Not bad guidelines, but they fail to affect the key problem behind PowerPoint presentations. Slides should not provide a kind of closed captions for your talk or a repository for reams of data.

As this presentation demonstrates, PowerPoint works best as a backdrop to a speech. Use it for images, simple figures, and emphasis. Then leave it in the background and talk to the audience.

Executive Compensation

September 28th, 2011 by pcahn

The Center for Social Philanthropy has released a report on the most highly paid employees at Massachusetts's private universities. As the figure shows, issue-brief-exec-comp-201109the average of the 8 top earners at Harvard far outpaced all other universities.

The Harvard numbers, however, include salary for the investors who manage the endowment. Removing those outliers, Harvard comes closer to BU as the institution with the highest average top earners.

The report points out that the outsized earnings came at the same time as the height of the financial crisis. At BU, the highest paid employees earn more than 17 times what the average faculty member earns.

Working Mothers

September 19th, 2011 by pcahn

The magazine Working Mother released its list of the 100 best employers. Although for-profit companies dominated the top rankings, three academic medical centers rated highly. University of Wisconsin Hospital and Clinics, Virginia Commonwealth University Health System, and Yale-New Haven Hospital have made family-friendly policies a priority.

Magazine editors looked for several programs that support working mothers:

  • Flex coupons that allow workers to take paid time off in increments
  • The ability to shift start and stop times
  • Compressed workweeks
  • Job sharing
  • On- and off-ramp programs
  • Options for employees to speed or slow their advancement without penalty

Some of the hospitals' innovations include:

  • subsidized care for elderly relatives
  • on-site daycare
  • benefits for adopting children
  • college counseling for children of employees
  • legal and financial seminars

It's telling that financial services companies figure highly on the list. Their commitment to flexible work policies should be a model for academic institutions keen on retaining faculty.

Peer Review for Trainees

September 13th, 2011 by pcahn

In one week in 2010, educators nominated the best articles about rethinking higher education. Organizers as the Roy Rosenzweig Center for History and New Media at George Mason University compiled the submissions into an e-book called Hacking the Academy.

One of the provocative ideas there for shaking up traditional academia is from Cathy Davidson, former Vice Provost for Interdisciplinary Studies at Duke. She describes an experiment in "crowdsourcing" student grades. Rather than the faculty member alone evaluate student performance, she had fellow class members determine if a student's work was satisfactory.

This approach eliminated some of the usual student jockeying to perform for the teacher and gave them a wider audience. The method could easily be applied to clinical teaching settings where peers observe each other's performance.

Action on Retraction

September 9th, 2011 by pcahn

RETRACTED SCIENCE AND THE RETRACTION INDEX -- Fang and Casadevall, 10.1128IAIAfter the editors of Infection and Immunity retracted six articles in one year, they got to thinking about the frequency of retractions.

They took a sampling of 17 journals with a range of impact factors and then created an index for each one that measured the number of retracted articles from 2001-2010 as a proportion of total articles published.

Their findings show that the higher the impact factor of the journal, the greater the frequency of retractions. They speculate that the rewards of publishing in prestigious venues may motivate researchers to engage in scientific misconduct.

Others who have looked at the data remind us that the number of retractions in all journals is vanishingly small. While it's important to enforce ethical research practices, we may be overstating the impact of retracted papers.

Punking Peer Review

September 7th, 2011 by pcahn

The Open Information Science Journal is an open-access, peer-reviewed journal published by Bentham and indexed in Open J-Gate and Genamics JournalSeek.

Phil Davis, a postdoc at Cornell, was interested to see how rigorous the review process at the journal was. So, he used software to generate a realistic-looking but gibberish article called "Deconstructing Access Points." Access Points

As the figure on the right shows, the article looked scientific but in reality made no sense. Still, four months after submission, Dr. Davis received word from the editor that his article had passed peer review and was accepted for publication. All he had to do was send $800.

He declined to pay, but wrote about the experiment for a scholarly publishing blog. His trick recalls the Sokal hoax where a physicist submitted a nonsense paper to a humanities journal, got it published, and revealed it later. But where Sokal was poking fun at the meaninglessness of postmodernism, Davis is pointing to the lax regulation of open access journals.

Not all online journals are this craven, but it shows that peer review is no guarantee of quality.