Archives for July 26, 2023

by Sara Elhawash

In a significant collaboration aimed at advancing data science research and training, the Data Sciences Institute (DSI) at the University of Toronto is partnering with the United Nations Children’s Fund (UNICEF)’s Frontier Data and Tech team to leverage data for addressing complex challenges concerning children. This collaborative effort aligns with DSI’s strategic goal of fostering knowledge mobilization to promote the greater public good. This partnership represents a significant milestone in the journey towards leveraging the immense potential of data for effecting positive social change.  

DSI will work with UNICEF to strengthen UNICEF’s knowledge and capacities to use data science and methodologies to innovate learning, through joint research and training. This collaboration will involve joint research and training initiatives.  DSI collaborates with organizations committed to supporting world-class researchers, educators, and trainees who are at the forefront of advancing data sciences.  

“This partnership is a significant milestone for our Frontier Data Network, a global community of practice that leverages data science to positively impact the lives of children worldwide. Together, we are poised to unlock new insights, drive evidence-based decision-making, and pave the way to a brighter future for children everywhere,” says Yves Jaques, Chief of the Frontier Data and Technology Unit, UNICEF. 

As a first collaboration, Dr. Manuel Garcia-Herranz, Data Principal Researcher and Karen Avanesyan, Statistics and Monitoring education specialist at UNICEF’s Division of Data, Analytics, Planning and Monitoring (DAPM) at UNICEF, are collaborating with Professor Zahra Shakeri, Dalla Lana School of Public Health on a 2023 Summer Undergraduate Data Science (SUDS) Opportunities Program. The SUDS project, Understanding Predictive Models that can be Used to Prevent School Dropouts aims to revolutionize early warning systems in education through the application of cutting-edge AI technology. The SUDS opportunity allows a SUDS Scholar, Ziqi Shu, to gain practical experience by working on fictional yet reality-based case studies focusing on social problems affecting children. By identifying at-risk students and schools with high dropout rates, UNICEF aims to support countries with a strong Education Management Information System (EMIS) and household survey data.  

The SUDS Scholar project aims to use and generate new sources of real-time information to better inform decision makers in the development and humanitarian ecosystem. UNICEF’s Frontier Data Tech Network is a global initiative to explore and use frontier data technologies to address the most complex challenges for children in an ethical way.  

“Our aim is to develop a pilot tool that provides a comprehensive representation of the machine learning-based school dropout prediction landscape, bridging the knowledge gap in this area. This tool will utilize innovative data analysis and visualization techniques, benefiting researchers, practitioners, and other stakeholders in exploring the factors influencing school dropout among children. The long-term goal of this project is to harness the power of data science and create an adaptable, publicly accessible system that could support countries in addressing the critical issue of school dropouts. By leveraging AI technology and early warning systems, our aim is to identify and support at-risk students and schools, ultimately safeguarding every child’s right to education,” says Zahra Shakeri, Director of HIVE Lab, Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health. 

The UNICEF-DSI partnership paves the way for further research and training collaborations. There will be opportunities to connect with the DSI community during its DSI Research Day on September 27, 2023. Dr. Manuel Garcia-Herranz will deliver the keynote address and Yves Jaques will participate in a panel discussion on Developing an Effective Data Science Workforce. The discussion will focus on equipping graduates with essential Data Science skills required in today’s diverse fields and industries. The DSI Research Day showcases the work of the DSI community, fostering connections and engagement among academia, industry, and government stakeholders. 

“By combining our community’s expertise in data science with UNICEF’s commitment to driving results for children globally, we have the opportunity to make a profound impact. Through our joint efforts, we aim to strengthen UNICEF’s knowledge and capacities in utilizing data science methodologies, fostering innovation in learning and ultimately creating a brighter future for children worldwide,” says Lisa Strug, Director, Data Sciences Institute.  

by Sara Elhawash

Can reproducibility pave the way for groundbreaking advancements in the field of precision oncology and transform cancer treatment decisions? A resounding answer emerges from an exciting reproducibility project born out of the DSI Student-Led Reproducibility Challenge. This project, led by DSI members and Professor Benjamin Haibe-Kains (University Health Network and Medical Biophysics, Temerty Faculty of Medicine, University of Toronto) and Bo Wang (Department of Laboratory Medicine & Pathobiology, University of Toronto) and a team of U of T student researchers including Emily So and Grace Fengqing Yu, is currently making significant strides in advancing research within the field. 

Reproducibility and Reusability in Action 

In a recent Reusability report Evaluating reproducibility and reusability of a fine-tuned model to predict drug response in cancer patient samples published in Nature Machine Intelligence, the team successfully reproduced and applied a new Artificial Intelligence (AI) method, called Transfer of Cellular Response Prediction (TCRP), originally published by the Ideker group at the University of California San Diego, in Nature Cancer in 2021, to clinical trial data. 

The project originated from the DSI Thematic Program in Reproducibility, which aims to raise awareness of reproducibility, including a Student-Led Reproducibility Challenge in 2022. Given the increasing utilization of large-scale and intricate datasets and computational methods across various disciplines, the challenge of reproducibility has come to the forefront. Establishing reproducibility standards for research has emerged as a foundational aspect of data science. Therefore, it becomes essential to clearly articulate and widely integrate standards for open, reproducible research with big data. This integration is crucial not only within the University of Toronto but also on an international scale. 

Emily So, a master’s student and co-researcher, reflects on the importance of reproducibility and open science principles in the context of groundbreaking methods like AI and machine learning, “In agreement with FAIR (Findability, Accessibility, Interoperatibility and Reusability) principles well established in research, usually new articles will come with data and computer code available for the scientific community. To fully understand the impact of new innovations and uncover their applications to new scientific problems, it is imperative that available resources are fully reproducible and can produce expected results easily.“ 

The DSI Student-Led Reproducibility Challenge attracted researchers and trainees dedicated to exploring reproducibility. “DSI support was instrumental in organizing the Challenge where students showcased their efforts in reproducing key papers in the field of engineering, social and health sciences. Emily So and Grace Yu were part of one of these teams. Their results were so exciting that we decided to push the analysis further and publish it as a Reusability Reports in Nature Machine Intelligence,“ says Benjamin Haibe-Kains. 

“We were able to demonstrate the gaps that exist in open science for computational biology. This outreach made available by the DSI has allowed our group to project our experience to the scientific community as well as provide further rationale for our subsequent documentation about our project,“ says Emily So. 

The team’s work aims to address two key objectives: confirming the performance of the TCRP model in its published context and expanding its application to a larger compendium of preclinical pharmacogenomic and clinical trial data.  

Through extensive evaluation, the researchers found that the TCRP method surpassed established statistical and machine learning approaches in predicting drug response in novel clinical contexts. This remarkable finding highlights the superiority of TCRP in both preclinical and clinical settings. 

“Our results highlight the immense potential of the TCRP method and its ability to outperform existing approaches. This opens new avenues for optimizing clinical trial design and improving patient outcomes,“ says Haibe-Kains. 

In the field of precision oncology, ensuring the reliability and generalizability of new techniques in clinical settings is crucial. Reproducibility studies play a vital role in verifying claims made by predictive models, while reusability studies assess their applicability in diverse contexts. The publication of the Reusability Report in Nature Machine Intelligence signifies a significant step forward in promoting reproducibility and reusability in the field. 

“Our work emphasizes the importance of reproducibility and reusability, which are essential for advancing precision oncology. By documenting new data contexts and exploring the model’s reusability, we can drive further progress in tailored cancer treatments,“ says Haibe-Kains 

“Reproducing the results of this method was no easy task, but it provided a glimpse into the power and impact it could have. It was an exciting endeavor to explore the possibilities of this machine learning approach,“ shares Emily So, masters student and co-researcher. 

Collaboration, Transparency, and Future Applications 

The impact of this work extends beyond the research community. The study’s reliance on open science principles, where authors share their code and data, highlights the importance of collaboration and transparency. By making their materials publicly available, the researchers contribute to education, enabling the training of future health data scientists, bioinformaticians and computational biologists. 

Emily So emphasizes the potential future applications of their models, “This evaluation is timely because there is a potential future application of these models in assisting clinicians in the treatment decision process. Setting a reproducibility standard is crucial to properly evaluate machine learning approaches suitable for preclinical and clinical settings, ultimately optimizing the course of action for patients.“ 

With the successful reproduction of the TCRP model and its outperformance of existing approaches, the potential for optimized clinical trial design and improved patient outcomes becomes a tangible reality.