Bringing personalized and actionable decision making to patient care

We work end-to-end through development, validation, and prospective evaluation to bring computational decision support to the clinic.

Integrating clinical knowledge and data science

We integrate knowledge of the underlying details of clinical data with machine learning expertise to guide algorithmic development.

Uncovering insights from real world data

We apply computational methods for real world data-driven clinical knowledge discovery and hypothesis generation.

Crossing multiple data streams

We extract and combine information from multi-modal data sources to generate insights and predictions.

The Hong Lab is part of the Department of Radiation Oncology, Bakar Computational Health Sciences Institute, and the Center for Real World Evidence at the University of California, San Francisco (UCSF) and the UCSF-UC Berkeley Joint Program in Computational Precision Health.

News

We are hiring! Check out our opportunities below.

Congrats to Lisa Ni, Christina Phuong, and Gabe Vidal on their chapters in Artificial Intelligence in Radiation Oncology!
We are excited to have our latest SHIELD-RT-related study published in BMJ Health & Care Informatics, describing healthcare provider reactions to the implementation of our machine learning-directed care strategy.
Our publication describing challenges involved in implementing machine learning in the clinic based on our experience on SHIELD-RT was published in BMC Bioinformatics.
We are back from ASTRO and proud of our team!
- Izzy Friesner gave an oral presentation about our machine learning work to predict hospitalization based on wearable activity monitoring in collaboration with Montefiore Einstein Cancer Center. Media coverage in the ASTRO Press Release, Medscape, Aunt Minnie, Inside Precision Medicine, and Health IT Analytics.
- Jane Chen gave an oral presentation about our natural language processing work to understand patterns of documented symptoms following metastatic cancer diagnoses and prior to palliative radiotherapy.
- Daphna Spiegel gave an oral presentation on our natural language procesing work to characterize conversations on social media around breast cancer treatments.
- Will Chen gave an oral presentation on our work with collaborators investigating deep learning applications in predicting cardiac events based on radiotherapy treatment planning scans.
Congratulations to Travis Zack for the publication of our study A Clinical Reasoning-Encoded Case Library Developed through Natural Language Processing in the Journal of General Internal Medicine! Check out the website!
Congratulations to Travis Zack for receiving the UCSF Artificial Intelligence/Machine Learning Demonstration Award for his project “Oncologic treatment history identification and visualization through Apex and Information Commons!”
Congratulations to Yash Huilgol and the publication of his work with our awesome collaborators Julia Adler-Milstein and Susan Ivey describing the opportunities for electronic health record audit loges to improve cancer care in Cancer Medicine!
SHIELD-RT was recognized with the 2021 Radiation Oncology Institute Publication Award. It was previously recognized by Cambridge Healthtech Institute and Bio-IT World with an Innovative Practice Award at the Bio-IT World Conference and Expo as well as the New England Journal of Medicine Travel Award at the Symposium for Artificial Intelligence in Learning Health Systems (SAIL; video available online). It is one of the early randomized controlled interventional studies of healthcare machine learning and was published in the Journal of Clinical Oncology and covered by the ASCO Post.
Great to be back at the AMIA Annual Symposium this year! Congratulations to Divya Natesan on the Distinguished Poster award and to Saira Somnay for presenting her work.
Congratulations to Will Chen on the publication of our study investigating the association of mental health diagnoses with race and mortality in patients with cancer in Cancer. Coverage in CURE and Healio.
Julian Hong's commentary describing best practices in causal inference and real world data was recently published in JAMA Network Open.
Kevin Miao is presenting our work on applying machine learning to predicting potentially preventable acute care based on OP-35 at the ASCO Quality Care Symposium.
Our collaboration with the Durham Veteran's Affairs Cooperative Studies Program Epidemiology Center (CSPEC) applying network analysis to identify patterns of comorbid medical conditions was recently published in Scientific Reports.
Congratulations to Divya Natesan for receiving the Conquer Cancer ASCO Merit Award!
Congratulations to Sumi Sinha, Meera Garriga, Nishali Naik, and Brian McSteen on the publication of our study investigating longitudinal disparities in patient portal enrollment among cancer patients in JAMA Oncology! This study was covered by the ASCO Post and Cancer Therapy Advisor.
Congratulations to Meera Garriga, Sumi Sinha, Nishali Naik, Sasha Yousefi, Brian McSteen, Clark Wang, and Jon Wang for their accepted presentations for the AMIA Informatics Summit!
Our natural language processing pipeline to extract patient symptoms during cancer radiotherapy from clinical notes was published in JAMIA Open.
Julian Hong participated as a panelist at the National Cancer Institute Webinar, Leveraging Clinical Narrative Text at Scale: Applying Natural Language Processing to Understand the Patient Care Experience (recording available online).
Our recent commentary with collaborators regarding the translation of machine learning to clinical cancer care was published in JAMA Oncology.

News archive

Our research

Bringing machine learning to the oncology clinic

The primary goal of our lab is to develop computational analyses and tools that reach patient care to improve outcomes. The convergence of machine learning and expansive routine clinical data enable the development of automated predictive models. However, many barriers remain, impacting the real-world accuracy, deployability, and clinical benefit of artificial intelligence and predictive tools.

Among tools developed by our group, we developed an electronic health record-based machine learning algorithm to predict acute care visits during outpatient cancer therapy (Hong, JCO Clin Cancer Inform 2018). This model was recently evaluated in one of the first prospective randomized studies of clinical machine learning, demonstrating that its use directed clincial management to reduce rates of acute care during cancer therapy (Hong, J Clin Oncol 2020).

We have developed a range of other predictive models to improve cancer care, including risk stratification to improve care for patients with short-term mortality (Wu, JAMA Netw Open), identifying favorable characteristics in patients with oligometastatic cancers (Hong, PLOS ONE 2018) and predicting testosterone recovery following androgen deprivation (Spiegel, Int J Radiat Oncol Biol Phys 2019).

Learning from real world data

Real world data offers insights that can complement randomized controlled clinical trials. This can provide information regarding practice patterns and generate knowledge to inform future trials. Our team has expertise analyzing data from electronic health records, national databases, institutional experiences, claims data, and payor data to both learn from prior experiences and characterize data limitations (Hong, JAMA Netw Open 2021).

Among prior studies investigating real world data, we have demonstrated the disparities in patient enrollment to online patient portals (Sinha, JAMA Oncology 2021). We have also evaluated practice patterns in a range of cancers including lung (Hong, J Natl Compr Canc Netw 2019), prostate, and gynecologic malignancies. Among these studies, we have used Medicare provider data to identify disparities in access to specialized therapies for prostate and gynecologic malignancies (Hong, Brachytherapy 2018).

Unlocking unstructured clinical data

Large amounts of data are captured in free-text clinical notes, which can pose challenges to clinical study. Extracting this unstructured information can both inform clinical research and facilitate deployable computational tools.

We have adapted clinical natural language processing pipelines to extract patient outcomes during cancer therapy from routine clinical documentation. Our studies indicate high variability even among human experts, which may motivate the use of computational approaches (Fairchild, Int J Radiat Oncol Biol Phys 2020). Our natural language processing pipeline facilitates systematic extraction of patient symptoms during cancer therapy (Hong, JAMIA Open 2020).

Our team

Julian Hong, MD, MS

Principal Investigator

Amir Ashraf-Ganjouei, MD

Postdoctoral Fellow

UCSF Surgery, Radiation Oncology,

Bakar Computational Health Sciences Institute

Ji Hyun Chang, MD, PhD

Assistant Professor

Seoul National University

Jane Chen, MD

Resident Physician

UCSF Radiation Oncology

William Chen, MD

Resident Physician

UCSF Radiation Oncology

Izzy Friesner, BA

Clinical Data Scientist

UCSF Radiation Oncology,

Bakar Computational Health Sciences Institute

Yash Huilgol, MS

Medical and Graduate Student

UC Berkeley-UCSF Joint Medical Program

Havish Kantheti, MS

Medical Student

Texas A&M College of Medicine

Somya Mohindra, BS

Undergraduate

UC Berkeley Computer Science

Lisa Ni, MD

Resident Physician

UCSF Radiation Oncology

Christina Phuong, MD

Resident Physician

UCSF Radiation Oncology

Sumi Sinha, MD

Instructor

UCSF Radiation Oncology

Saira Somnay

Undergraduate

UC Berkeley Molecular and Cellular Biology

Michael Waters, MD, PhD

Resident Physician

Washington University in St. Louis

Radiation Oncology

Travis Zack, MD, PhD

Postdoctoral Fellow

UCSF Internal Medicine

Close collaborators

Atul Butte, MD, PhD (Bakar Computational Health Sciences Institute, UCSF)

Steve Braunstein, MD, PhD (Radiation Oncology, UCSF)

Felix Feng, MD (Radiation Oncology, UCSF)

Jean Feng, PhD (Epidemiology and Biostatistics; Bakar Computational Health Sciences Institute, UCSF)

Amy Lin, MD (Hematology & Oncology, UCSF)

Osama Mohamad, MD, PhD (Radiation Oncology, UCSF)

Olivier Morin, PhD (Radiation Oncology, UCSF)

Nitin Ohri, MD (Radiation Oncology, Montefiore Medical Center/Albert Einstein College of Medicine)

Catherine Park, MD (Radiation Oncology, UCSF)

Manisha Palta, MD (Radiation Oncology, Duke University)

Eric Small, MD (Hematology & Oncology, UCSF)

Daphna Spiegel, MD, MS (Radiation Oncology, Beth Israel Deaconess Medical Center; Harvard Medical School)

Alumni

Andrew Fairchild, MD, MA - Radiation Oncologist, Piedmont Radiation Oncology

Meera Garriga, BA - Medical Student, UCSF School of Medicine

Morgan Lafferty, BA- Graduate Student, UCSF

Anthony Lin, MD - Fellow, Duke Cardiology

Brian McSteen, MD - Resident Physician, Weill-Cornell Internal Medicine

Kevin Miao, BA - Graduate Student, UC Berkeley

Gavril Moniaga, BS - Facebook

Nishali Naik, BA - Aon

Divya Natesan, MD - Assistant Professor, University of North Carolina at Chapel Hill

Shawn Patel - Undergraduate, UC Berkeley

Isis Trenchard, PhD - Benchling

Eric Xu - Undergraduate, Stanford

Clark Wang - Undergraduate, UC Berkeley

Jon Wang, MS - Shimmer; Medical Student, UCSF School of Medicine

Sasha Yousefi, BA - Graduate Student, Stanford University

William Zhang - Undergraduate student, UC Berkeley

Positions

Join us!

Our group is always interested in finding new lab members and collaborators to contribute with clinical domain knowledge or computational expertise. Join our multidisciplinary team of clinicians and scientists in the Department of Radiation Oncology and the Bakar Computational Health Sciences Institute at UCSF and across UCSF and UC Berkeley in the Joint Program in Computational Precision Health. Lab members are also be encouraged to take advantage of other training and learning opportunities within the UCSF Helen Diller Family Comprehensive Cancer Center, the Bakar Computational Health Sciences Institute, and other related entities at UCSF and UC Berkeley.

If you are interested in joining the lab or collaborating on research in a role not listed below, please contact us.

Currently funded priorities are focused on machine learning with electronic health record data and wearable devices and natural language processing in the areas of cancer care delivery and prostate cancer care.

Post-doctoral opportunities

We are recruiting highly motivated investigators to develop algorithms and analyses with multi-modal clinical data to guide personalized oncology care. Projects will be tailored to the individual's prior experience and interests but will center on the utilization of computational techniques to drive clinical prediction or discover new clinical insights using routine clinical data (including electronic health records, imaging, digital pathology, and genomic data).

Ideal candidates will have a PhD and/or MD with a strong background in informatics or a related computational science and demonstrated publication record. Strong problem-solving skills, creative thinking, and programming ability are required. Applicants must possess good communication skills and be fluent in both spoken and written English. Prior experience in machine learning, natural language processing, network analysis, and computer vision are a plus.

Natural language processing and machine learning (wearables and electronic health record data) are the current priority areas of the lab.

The post-doctoral fellow will join a multidisciplinary team of clinicians and scientists in the Department of Radiation Oncology and the Bakar Computational Health Sciences Institute.

Further details are available on the UCSF Office of Career and Professional Development website.

Undergraduate opportunities

Our lab welcomes undergraduates and recent graduates with an interest in medicine and informatics to join us. We hope that this experience will provide preparation for future opportunities in medicine and informatics. Clinical shadowing opportunities are available to those interested.

UC Berkeley students can apply through the Undergraduate Research Apprentice Program.

Data Science Research Specialist

We are recruiting for data science researchers for the lab. Interested candidates across the range of experience are encouraged to apply. Post-baccalaureate candidates are encouraged! We hope that this opportunity will help prepare candidates for future careers in medicine or informatics and data science. Further details and application available on UCSF Recruit.

Clinical trainee opportunities

Our lab welcomes medical students, residents, and fellows from all backgrounds to join us. Projects can be tailored based on the trainee's areas of interest and computational experience (not required).

Additionally, we have specific projects focused on implementation and clinical trials that can benefit from medical knowledge, such as those focused on real world data or protocol development.

UCSF medical students and trainees can apply at UCSF LabSpot.

Graduate training opportunities

Our lab is part of the UCSF-UC Berkeley Joint Program in Computational Precision Health. For more details on how to apply to our PhD program, check out the program website. Please feel free to reach out to us if you are interested.

Selected publications

2022

Healthcare provider evaluation of machine learning-directed care: reactions to deployment on a randomised controlled study. Hong JC, Patel P, Eclov NCW, Stephens SJ, Mowery YM, Tenenbaum JD, Palta M. BMJ Health Care Inform. 2023 Feb;30(1):e100674. doi: 10.1136/bmjhci-2022-100674. PMID: 36764680.

A Clinical Reasoning-Encoded Case Library Developed through Natural Language Processing. Zack T, Dhaliwal G, Geha R, Margaretten M, Murray S, Hong JC. J Gen Intern Med. 2023 Jan;38(1):5-11. doi: 10.1007/s11606-022-07758-0. Epub 2022 Sep 7. PMID: 36071325.

Opportunities to use electronic health record audit logs to improve cancer care. Huilgol YS, Adler-Milstein J, Ivey SL, Hong JC. Cancer Med. 2022 Mar 29. doi: 10.1002/cam4.4690. PMID: 35348298.

Implementation of machine learning in the clinic: challenges and lessons in prospective deployment from the System for High Intensity EvaLuation During Radiation Therapy (SHIELD-RT) randomized controlled study. Hong JC, Eclov NCW, Stephens SJ, Mowery YM, Palta M. BMC Bioinformatics. 2022 Sep 30;23(Suppl 12):408. doi: 10.1186/s12859-022-04940-3. PMID: 36180836.

Analysis of Serious Adverse Event Reporting for Patients Enrolled in Cancer Clinical Trials During the COVID-19 Pandemic. Ragavan MV, Legaspi N, LaLanne A, Hong JC, Small EJ, Borno HT. JAMA Oncol. 2022 Dec 1;8(12):1849-1851. doi: 10.1001/jamaoncol.2022.4919. PMID: 36301577.

COVID-19 Outcomes Among Patients With Cancer: Observations From the University of California Cancer Consortium COVID-19 Project Outcomes Registry. Borno HT, Kim MO, Hong JC, Yousefi S, Lin A, Tolstykh I, Zhang S, McKay RR, Harismendy O, Cinar P, Rugo H, Koshkin VS, Rabow M, Wang C, Bailey A, Small EJ. Oncologist. 2022 Mar 28:oyac038. doi: 10.1093/oncolo/oyac038. PMID: 35348771.

COVID-19 outcomes in patients with cancer: Findings from the University of California health system database. Kwon DH, Cadena J, Nguyen S, Chan KHR, Soper B, Gryshuk AL, Hong JC, Ray P, Huang FW. Cancer Med. 2022 Mar 9. doi: 10.1002/cam4.4604. PMID: 35261195.

2021

Disparities in Electronic Health Record Patient Portal Enrollment Among Oncology Patients. Sinha S, Garriga M, Naik N, McSteen BW, Odisho AY, Lin A, Hong JC. JAMA Oncol. 2021 Apr 8. doi: 10.1001/jamaoncol.2021.0540. PMID: 33830178.

Association of mental health diagnosis with race and all-cause mortality after a cancer diagnosis: Large-scale analysis of electronic health record data. Chen WC, Boreta L, Braunstein SE, Rabow MW, Kaplan LE, Tenenbaum JD, Morin O, Park CC, Hong JC. Cancer. 2021 Sep 22. doi: 10.1002/cncr.33903. PMID: 34550601.

Strategies to Turn Real-world Data Into Real-world Knowledge. Hong JC. JAMA Netw Open. 2021 Oct 1;4(10):e2128045. doi: 10.1001/jamanetworkopen.2021.28045. PMID: 34618043.

Assessing Clinical Outcomes in a Data-Rich World-A Reality Check on Real-World Data. Hong JC, Butte AJ. JAMA Netw Open. 2021 Jul 1;4(7):e2117826. doi: 10.1001/jamanetworkopen.2021.17826. PMID: 34309673.

An artificial intelligence framework integrating longitudinal electronic health records with real-world data enables continuous pan-cancer prognostication. Morin O, Vallieres M, Braunstein S, Barrios Ginart J, Upadhaya T, Woodruff HC, Zwanenburg A, Chatterjee A, Villanueva-Meyer JE, Valdez G, Chen W, Hong JC, Yom SS, Solberg TD, Lock S, Seuntjens J, Park C, Lambin P. Nat Cancer 2, 709–722 (2021). https://doi.org/10.1038/s43018-021-00236-2.

Risk Stratification for Imminent Risk of Death at the Time of Palliative Radiotherapy Consultation. Wu SY, Yee E, Vasudevan HN, Fogh SE, Boreta L, Braunstein SE, Hong JC. JAMA Netw Open. 2021 Jul 1;4(7):e2115641. doi: 10.1001/jamanetworkopen.2021.15641. PMID: 34196716.

Characterizing chronological accumulation of comorbidities in healthy veterans: a computational approach. Hong JC, Hauser ER, Redding TS 4th, Sims KJ, Gellad ZF, O'Leary MC, Hyslop T, Madison AN, Qin X, Weiss D, Bullard AJ, Williams CD, Sullivan BA, Lieberman D, Provenzale D. Sci Rep. 2021 Apr 14;11(1):8104. doi: 10.1038/s41598-021-85546-2. PMID: 33854078.

2020

System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning–Directed Clinical Evaluations During Radiation and Chemoradiation. Hong JC, Eclov NCW, Dalal NH, Thomas SM, Stephens SJ, Malicki M, Shields S, Cobb A, Mowery YM, Niedzwiecki D, Tenenbaum JD, Palta M. J Clin Oncol. 2020 Sep 4:JCO2001688. doi: 10.1200/JCO.20.01688. PMID: 32886536.

Closing the Gap Between Machine Learning and Clinical Cancer Care-First Steps Into a Larger World. Kang J, Morin O, Hong JC. JAMA Oncol. 2020 Sep 24.doi: 10.1001/jamaoncol.2020.4314. PMID: 32970129.

Natural language processing for abstraction of cancer treatment toxicities: accuracy versus human experts. Hong JC, Fairchild AT, Tanksley JP, Palta M, Tenenbaum JD. JAMIA Open. 2020 Dec 5. doi: 10.1093/jamiaopen/ooaa064.

Inter-rater reliability in toxicity identification: Limitations of current standards. Fairchild AT, Tanksley JP, Tenenbaum JD, Palta M, Hong JC. Int J Radiat Oncol Biol Phys. 2020 May 1. pii: S0360-3016(20)31084-1. doi: 10.1016/j.ijrobp.2020.04.040. PMID: 32371073.

Electronic health record (EHR) data mining for AI healthcare. Lin AL, Chen WC, Hong JC. Artificial Intelligence in Medicine: Technical Basis and Clinical Applications. Xing L, Giger ML, Min J, editors. Amsterdam: Elsevier; 2020.

2019

A Nomogram for Testosterone Recovery After Combined Androgen Deprivation and Radiation Therapy for Prostate Cancer. Spiegel DY, Hong JC, Oyekunle T, Waters L, Lee WR, Salama JK, Koontz BF. Int J Radiat Oncol Biol Phys. 2019 Mar 15;103(4):834-842. doi: 10.1016/j.ijrobp.2018.11.007. Epub 2018 Nov 10. PMID: 30419308.

Increasing PET Use in Small Cell Lung Cancer: Survival Improvement and Stage Migration in the VA Central Cancer Registry. Hong JC, Boyer MJ, Spiegel DY, Williams CD, Tong BC, Shofer SL, Moravan MJ, Kelley MJ, Salama JK. J Natl Compr Canc Netw. 2019 Feb;17(2):127-139. doi: 10.6004/jnccn.2018.7090. PMID: 30787126.

Radiotherapy Treatment Planning in the Age of AI: Are We Ready Yet? Zheng D, Hong JC, Wang C, Zhu X. Technol Cancer Res Treat. 2019 Jan-Dec;18:1533033819894577. doi: 10.1177/1533033819894577. PMID: 31858890.

Artificial Intelligence in Radiotherapy Treatment Planning: Present and Future. Wang C, Zhu X, Hong JC, Zheng D. Technol Cancer Res Treat. 2019 Jan 1;18:1533033819873922. doi: 10.1177/1533033819873922. PMID: 31495281.

2018

Predicting Emergency Visits and Hospital Admissions During Radiation and Chemoradiation: An Internally Validated Pretreatment Machine Learning Algorithm. Hong JC, Niedzwiecki D, Palta M, Tenenbaum JD. JCO Clin Cancer Inform. 2018 Dec;2:1-11. doi: 10.1200/CCI.18.00037. PMID: 30652595.

High-volume providers and brachytherapy practice: A Medicare provider utilization and payment analysis. Hong JC, Spiegel DY, Havrilesky LJ, Chino JP. Brachytherapy. 2018 Nov-Dec;17(6):906-911. doi: 10.1016/j.brachy.2018.07.007. Epub 2018 Jul 26. PMID: 30057292.

Intratreatment Response Assessment With 18F-FDG PET: Correlation of Semiquantitative PET Features With Pathologic Response of Esophageal Cancer to Neoadjuvant Chemoradiotherapy. Tandberg DJ, Cui Y, Rushing CN, Hong JC, Ackerson BG, Marin D, Zhang X, Czito BG, Willett CW, Palta M. Int J Radiat Oncol Biol Phys. 2018 Nov 15;102(4):1002-1007. doi: 10.1016/j.ijrobp.2018.07.187. Epub 2018 Jul 25. PMID: 30055238.

Association of Interim FDG-PET Imaging During Chemoradiation for Squamous Anal Canal Carcinoma With Recurrence. Hong JC, Cui Y, Patel BN, Rushing CN, Faught AM, Eng JS, Higgins K, Yin FF, Das S, Czito BG, Willett CG, Palta M. Int J Radiat Oncol Biol Phys. 2018 Nov 15;102(4):1046-1051. doi: 10.1016/j.ijrobp.2018.04.062. Epub 2018 May 4. PMID: 29891206.

Classification for long-term survival in oligometastatic patients treated with ablative radiotherapy: A multi-institutional pooled analysis. Hong JC, Ayala-Peacock DN, Lee J, Blackstock AW, Okunieff P, Sung MW, Weichselbaum RR, Kao J, Urbanic JJ, Milano MT, Chmura SJ, Salama JK. PLoS One. 2018 Apr 12;13(4):e0195149. doi: 10.1371/journal.pone.0195149. PMID: 29649281.