Rigor and reproducibility: What you need to know

Starting in 2012, Brian Nosek, Ph.D., of the University of Virginia, began a project that would attempt to reproduce the results of 100 psychology studies that had been published in three scientific journals. Enlisting the help of several hundred colleagues across the country, the scientists simply re-ran the studies described in the journals to see how many of them would match up to the results of the original publications.

Only 39 of them did.

Nosek’s findings, published in the journal Nature in 2015, are among the reasons that the National Institutes of Health are placing an added emphasis on rigor and reproducibility.

The former begets the latter – or so the hope is – and that’s why new requirements are popping up on grant applications to ensure that researchers are using rigorous, robust methodology that has been carefully planned out.

Among the changes to NIH grants:

• The addition of a “scientific premise” section that discusses the strengths and weaknesses of previous research. Researchers must show how their study is going to fill in some areas of weakness or expand on areas of strength.
• Added emphasis on the research approach, including biostatistical methods, inclusion and exclusion criteria, and diversity in study subjects.
• Requirements that researchers authenticate key biological or chemical resources, to protect against false findings caused by cross contamination.

“The NIH really wants to make sure that the methods you’re using have been planned out correctly,” said Carrie Dykes, Ph.D., research engagement specialist at the CTSI.

Fortunately, the Research Help Desk is available to answer questions and provide expertise about the grant changes. View a presentation detailing the new requirements or email ResearchHelp@urmc.rochester.edu for more information.

Nature also recently posted an article on the subject here.

Meet the winner of the 3rd Annual “America’s Got Regulatory Science Talent” competition

Joan Adamo, Angela Ryck, Martin Zand, and Scott Steele.

by Samreen Jatana
Special to CTSI Stories

Angela Ryck is the winner of the University of Rochester’s 3^rd Annual America’s Got Regulatory Science Talent Student Competition Winner. She spoke with CTSI Stories about her project.

Could you tell me a little bit about your background and what got you interested in the competition?

I went to the University of Notre Dame for my undergraduate studies and I majored in Science Business, which was one of the pre-med tracks. In 2013, I went to Washington D.C. to do my Masters in Public Health at George Washington University (GW). Last July, I came to the University of Rochester to do another Masters degree in Biomedical Engineering through the CMTI program (Center for Medical Technology and Innovation). One of the requirements of the program is that we take the Pathways to Medical Innovation course, which is taught by Joan Adamo, Ph.D., who organizes this competition along with Scott Steele, Ph.D. It is through the Pathways class that I became familiar with the competition. We had to actively think about a Regulatory Science topic throughout the semester and come up with an innovative idea to address a particular problem.

What was the format of the competition? Could you discuss the structure of it?

They did not give us any strict guidelines to follow, which was great. But the basis was that the FDA set up eight priority areas in 2011 specifying the issues that they really want to address. For the competition we had to choose one priority area and devise a solution to address it. We were not required to write a proposal or perform any sort of study for this purpose; it was quite open-ended in terms of how we could choose to present our ideas. The competition was open to teams or individuals, and I chose to work independently. We had just five minutes to present our ideas to the three judges.

Which priority area did you focus on? Could you talk about the ideas you presented in your talk?

I addressed FDA’s 7^th priority area, which is focused on enhancing medical counter measures, referred to as MCMs. Broadly speaking, these are the tools that we use to address situations including chemical threats or food that is contaminated or not safe. Within that area I focused on enhancing emergency communication, which I geared specifically towards food safety. To enhance emergency communications, the FDA wants to first assess past communications to see specifically where improvements are needed.  My idea was a sort of two-fold system that improves on the present while also assessing past communications; I called it the “ESCAPE” System. This is an acronym for “Emergency Situation Communication and Preparedness Evaluation”. To enhance current communications, it involves a universal card (the ESCAPE Card) that could be used at a wide range of stores or restaurants when a person buys food. That card will be linked to their contact information such as phone number or email. That way, in the event of a recall, those who purchased the recalled item can receive direct notification such as a text or phone call.
Stores like Wegmans and Costco are already doing exactly this, but it is not yet widespread or used by restaurants. Often people receive news of a recall via TV news or Internet, but this could make it more personalized and thus more effective. This will not only allow faster communication but will relieve people from anxiety if they were not involved in a certain recall case.  Aside from this, people will still need an incentive to participate.  Discounts and rewards systems tied to participating stores and restaurants could serve that purpose; and in return those stores and restaurants could retain some customer loyalty and perhaps some useful marketing information.

For the second aspect of assessing past communications, I designed a community-level evaluation to be used across the country. It would be a short series of questions answered online and part of the activation process when users first receive their ESCAPE card. The data collected could help the FDA to get a sense of how people in different geographical areas have received and/or understood past communications, and could also help to identify trends in health literacy. I was interested in this because of a previous experience of mine. During my masters in public health at GW, I interned for the Center for Disease Control & Prevention (CDC) in the State of Maine. I traveled throughout the State of Maine to conduct Rural Active Living Assessments, which look at individual communities and their capacity to promote physical activity through their facilities and programs. Getting down to the community level is important because it shows how the strengths and needs of communities can differ greatly. I applied this experience to the area of emergency communication.  Furthermore, Diana Monaco, a public affairs specialist at the FDA is a regional representative at the Buffalo office, and speaking with her was very helpful as well.

What next?

In the spring of 2016, I will travel to Maryland with Joan Adamo and Scott Steele and I will get an opportunity to present my project to the FDA. Before that I will continue to research and keep up with the news on this topic to ensure that all aspects of my proposal are in line with the goals of the FDA. Specifically, I’d like to further develop the evaluation portion of ESCAPE.

How does this competition fit into your future career goals? Do you see yourself working for the FDA at some point?

I definitely plan to work within the medical field. This competition and proposal are both in line with that. I really enjoyed this experience because it is so pertinent to what I learned during my public health degree and also very relevant to biomedical engineering. For example, you cannot design a medical device without familiarity with the FDA and the regulatory procedures. I learned a lot about the FDA during this project even though food safety is a little bit different than medical devices. Still, the ESCAPE Card concept could be applicable to medical devices as well. The idea can definitely be expanded to improve upon communication in a wide range of areas.

New clinical trial data sharing requirements for publications: comment period opens

The International Committee of Medical Journal Editors (ICMJE) is proposing changes in clinical trial data sharing requirements for publishing. The new rules would require investigators to include data sharing plans, and share de-identified data. The ICMJE is currently accepting feedback through their website (www.icmje.org). Changes will go into effect one year after formal adoption. The ICMJE defines a clinical trial as “any research project that prospectively assigns people or a group of people to an intervention, with or without concurrent comparison or control groups, to study the cause-and-effect relationship between a health-related intervention and a health outcome.”

Data sharing plans: When studies are registered in a database like clinicaltrials.gov, a data sharing plan will need to be included. Clinicaltrials.gov allows posting of such information. This requirement would need to be completed prior to the first participant being enrolled, a time when you may not necessarily know in which journal you will publish your work.

Data sharing: When a manuscript is accepted for publication for a clinical trial, the raw de-identified individual participant data will need to be shared within six months. This includes any data that was used to generate tables, figures, and appendices or supplementary material. In addition, when the manuscript is submitted for review, authors will be required to explain how the de-identified participant data will be shared.

The ICMJE also proposes certain safeguards to protect investigators: “First, ICMJE editors will not consider the deposition of data in a registry to constitute prior publication. Second, authors of secondary analyses using these shared data must attest that their use was in accordance with the terms (if any) agreed to upon their receipt. Third, they must reference the source of the data using a unique identifier of a clinical trial’s data set to provide appropriate credit to those who generated it and allow searching for the studies it has supported. Fourth, authors of secondary analyses must explain completely how theirs differ from previous analyses.”

Journals that belong to ICMJE include the Annals of Internal Medicine, the Lancet, the New England Journal of Medicine, JAMA and PLOS Medicine, just to name a few. Many other journals follow ICMJE publishing requirements.

UNYTE Scientific Session Explores Innovative Methods for Translational Population Health Research

UNYTE, the CTSI-organized group of 18 regional biomedical research institutions, presented its latest “Scientific Session” on December 3^rd. Entitled “Transforming Population Health Research: Advances, New Methods, and Community Partnerships”, the session brought together investigators from institutions across Upstate New York to share innovative methods, develop new collaborations and promote population-health research programs.  The topic ties together UNYTE’s mission of promoting translational research through cross-institutional research collaboration, and the CTSI’s increased emphasis on improving population health.

Dr. Kevin Fiscella, recent appointee to a Dean’s Professorship, was the keynote speaker.  His talk, “Population Health Matters:  It’s About Lives, It’s About Time, It’s About Action” presented a comprehensive vision for population health. He challenged researchers to work across the translational research spectrum and through community partnerships – taking advantage of progress toward a learning healthcare system, advances in the availability of real-time data, and patient input – to improve clinical interventions, preventive care, behavioral health, socioeconomic factors and social cohesion.  Dr. Harriet Kitzman reviewed her highly successful Nurse Family Partnership program, a nationally and internationally successful model for moving from research to implementation of highly effective population-health programs and policies. Wade Norwood, a leader in the Rochester community and the Chief Strategy Officer at the Finger Lakes Health Systems Agency, demonstrated that community-identified needs and community input in all phases of the research life cycle can be the engine that drives health improvement. Kimberly O’Brien, Professor from The Division of Nutritional Sciences (DNS) at Cornell University, reviewed her department’s innovative programs and institutional infrastructure for building the bridges between population health and basic science.

The UNYTE Scientific Session distinguishes itself from a scientific symposium by providing dedicated time on the day’s agenda for informal “meetings of the minds” and cross-institutional discussions. Small group break-outs included targeted topics on community and patient engagement, novel data collection methods for research, patient-centered care and the learning healthcare system, promoting population health across the urban-rural continuum, and an extended conversation on the necessary meeting points between basic science and population health. Awards for the poster session were given:  Dr. Deirdre Schlehofer, Assistant Professor at the National Technical Institute for the Deaf at the Rochester Institute of Technology and her research team won the Faculty Poster Award for their poster “Health Perceptions of Deaf and Hard-of-Hearing Female College Students;” Sheila Bruce, BS,  Post-BA Fellow/ Research Coordinator at the Deaf Health Communication and Quality of Life Center at The Rochester Institute of Technology won the Student/Trainee Poster Award for her poster “Population Health Research with Deaf/HH Adults in USA.”

Details for the next UNYTE Scientific Session, including the topic focus, will be announced soon. If you would like to learn more about UNYTE, please visit their website, http://www.urmc.rochester.edu/ctsi/unyte/.

Translational Biomedical Science alum lands postdoc at NIH

Lesley Chapman, PhD

Lesley Chapman, PhD graduated with a degree in Translational Biomedical Science (TBS) in August 2015 and is currently a NIH/NIAID post doctoral fellow in Computational Biology. Chapman has continued in research and is working with the Research Technologies Branch (RTB) to help develop software for electron microscopy imaging analysis. The multidisciplinary nature of the TBS program gave her exposure to both clinical and basic research and allowed her to select a mentor from various departments through the medical center. This has prepared her for a career in independent research that integrates bioinformatics and health sciences.

Learn more about the PhD in Translational Biomedical Science.

 

Spring CTSI Seminar Series Previews “The New CTSI”

The CTSI Seminar series resumes for the spring 2016 semester on Tuesday, January 19^th with a look at the “new CTSI.” The theme for the series is “The New CTSI: Connect, Learn, Get What You Need” and will feature presentations by the writing teams who collaborated on the recent Clinical and Translational Science Award (CTSA) application, which is intended to provide major funding for the CTSI’s next five-year cycle beginning July 2016.

CTSI Co-Directors (left to right): Nancy Bennett, MD, Director of the Center for Community Health, Professor of Medicine; Martin Zand, MD, PhD, Professor, of Medicine; and Karl Kieburtz, MD, MPH, Senior Associate Dean, Clinical Research, Robert J. Joynt Professor in Neurology

The objective of the seminar series is two-fold:

• To describe future plans for the CTSI as described in the 2015 renewal application
• To inform the University community about resources offered by the CTSI

The mission of the University of Rochester Clinical and Translational Science Institute (UR-CTSI) is to develop, demonstrate and disseminate methods and approaches to advance translational research, by: 1) providing education and training, 2) fostering transdisciplinary teams, 3) improving quality and efficiency, and 4) engaging community and national stakeholders and partners. Our vision for the next five years is to become a replicable model environment for research, across the translational spectrum from molecules to populations, responsive to community priorities, conducted by transdisciplinary, patient- and community-engaged teams, that improves population health.

Co-Director Karl Kieburtz, MD, MPH, said “We look forward to sharing the evolution of the CTSA program on the national level and how we plan to respond to those changes, as well as the evolving needs of the Rochester research community, with our plans for the next 5 years”

The CTSI Seminar series is held on Tuesdays from 12:00-1:00 pm. The first session (January 19^th) of the seminar series will be held in the Ryan Case Method Room (1-9576) in the Medical Center. The remaining sessions will be held in the Helen Wood Hall Auditorium (1w-304). All sessions will be streamed live. Lunch is provided; attendees should bring their own beverage. CME credit is also available for most sessions. The full schedule of seminar sessions is listed below and is also available here on the CTSI website. Also see the website for links to live streaming and archived recordings.

Date	Speaker (s)	Title
1/19/2016	Karl Kieburtz, MD, MPH; Martin Zand, MD, PhD; and Nancy Bennett, MD	Overview of the CTSA Program
1/20/2016	E. Ray Dorsey, MD, MBA and Karen Rabinowitz, JD	Telemedicine and Research (Part of the Good Advice: CTSI Skill-Building Workshop Series)
1/26/2016	Office for Human Subjects Protection (OHSP)	TBA
2/2/2016	Martin Zand, MD, PhD; Tom Fogg, MS; Ann Dozier, PhD; Sharyl Zaccaglino	Administrative Core: Organization, Governance, Collaboration, Communication; Evaluation and Continuous Improvement; Quality and Efficiency
2/9/2016	Timothy Dye, PhD	Informatics
2/16/2016	Nancy Bennett, MD; Gail Newton; and Katia Noyes, PhD, MPH	Community and Collaboration: Community Engagement; Collaboration and Multi-Disciplinary Science
2/23/2016	Martin Zand, MD, PhD; James Dolan, MD; Edward Schwarz, PhD	Translational Endeavors: Translational Workforce Development; Pilot and Translational Clinical Studies
3/1/2016	Karl Kieburtz, MD, MPH; Edwin van Wijngaarden, PhD; Rob Strawderman, ScD; Eric Rubinstein, JD, MPH	Research Methods: Biostatistics, Epidemiology, and Research Design; Regulatory Knowledge and Support
3/8/2016	Office for Human Subjects Protection (OHSP)	TBA
3/15/2016	Karl Kieburtz, MD, MPH; Karen Rabinowitz, JD; Carrie Dykes, PhD	Network Capacity: Liaison to Trial Innovation Centers; Liaison to Recruitment Innovation Centers
3/22/2016	Nancy Bennett, MD; Steven Barnett, MD; Giovanni Schifitto, MD, MS	Hub Research Capacity: Integrating Special Populations; Participant and Clinical Interactions
3/29/2016	Martin Zand, MD, PhD and Scott Steele, PhD	Network Science / Regulatory Science
4/5/2016	Office for Human Subjects Protection (OHSP)	TBA
4/12/2016	Timothy Dye, PhD and Robert Holloway, MD, MPH	KL2 Career Development Program and TL1 Training Program
4/19/2016	CTSI Trainees	CTSI Trainee Presentations
4/27/2016	Charles Duffy, MD, PhD and John Fahner-Vihtelic	Cognivue: Assessing the aging brain – Patenting, Licensing, and Commercializing a Breakthrough Technology (Part of the Good Advice: CTSI Skill-Building Workshop Series)