Bringing personalized and actionable decision making to patient care

We work end-to-end through development, validation, and prospective evaluation to bring computational decision support to the clinic.

Integrating clinical knowledge and data science

We integrate knowledge of the underlying details of clinical data with machine learning expertise to guide algorithmic development.

Uncovering insights from real world data

We apply computational methods for real world data-driven clinical knowledge discovery and hypothesis generation.

Crossing multiple data streams

We extract and combine information from multi-modal data sources to generate insights and predictions.

The Hong Lab is part of the Department of Radiation Oncology and Bakar Computational Health Sciences Institute at the University of California, San Francisco (UCSF).

News

We are hiring! Check out our opportunities below.
SHIELD-RT was recognized with the 2021 Radiation Oncology Institute Publication Award. It was previously recognized by Cambridge Healthtech Institute and Bio-IT World with an Innovative Practice Award at the Bio-IT World Conference and Expo as well as the New England Journal of Medicine Travel Award at the Symposium for Artificial Intelligence in Learning Health Systems (SAIL; video available online). It is one of the early randomized controlled interventional studies of healthcare machine learning and was published in the Journal of Clinical Oncology and covered by the ASCO Post.
Great to be back at the AMIA Annual Symposium this year! Congratulations to Divya Natesan on the Distinguished Poster award and to Saira Somnay for presenting her work.

Congratulations to Will Chen on the publication of our study investigating the association of mental health diagnoses with race and mortality in patients with cancer in Cancer. Coverage in CURE and Healio.
Julian Hong's commentary describing best practices in causal inference and real world data was recently published in JAMA Network Open.
Kevin Miao is presenting our work on applying machine learning to predicting potentially preventable acute care based on OP-35 at the ASCO Quality Care Symposium.
Our collaboration with the Durham Veteran's Affairs Cooperative Studies Program Epidemiology Center (CSPEC) applying network analysis to identify patterns of comorbid medical conditions was recently published in Scientific Reports.
Congratulations to Divya Natesan for receiving the Conquer Cancer ASCO Merit Award!
Congratulations to Sumi Sinha, Meera Garriga, Nishali Naik, and Brian McSteen on the publication of our study investigating longitudinal disparities in patient portal enrollment among cancer patients in JAMA Oncology! This study was covered by the ASCO Post and Cancer Therapy Advisor.
Congratulations to Meera Garriga, Sumi Sinha, Nishali Naik, Sasha Yousefi, Brian McSteen, Clark Wang, and Jon Wang for their accepted presentations for the AMIA Informatics Summit!
Our natural language processing pipeline to extract patient symptoms during cancer radiotherapy from clinical notes was published in JAMIA Open.
Julian Hong participated as a panelist at the National Cancer Institute Webinar, Leveraging Clinical Narrative Text at Scale: Applying Natural Language Processing to Understand the Patient Care Experience (recording available online).
Our recent commentary with collaborators regarding the translation of machine learning to clinical cancer care was published in JAMA Oncology.
Congratulations to Susan Wu for her Best of ASTRO 2020 award!
Congratulations to Anthony Lin and Will Chen for their recently published book chapter in Artificial Intelligence in Medicine!
Congratulations to Jon Wang for his accepted presentation for the ASCO Quality Care Symposium!
Congratulations to Will Chen, Susan Wu, and Neville Eclov for their accepted presentations for ASTRO 2020!
Our new study on the variability of experts in identifying patient symptoms published in the International Journal of Radiation Oncology, Biology, Physics.

Our research

Bringing machine learning to the oncology clinic

The primary goal of our lab is to develop computational analyses and tools that reach patient care to improve outcomes. The convergence of machine learning and expansive routine clinical data enable the development of automated predictive models. However, many barriers remain, impacting the real-world accuracy, deployability, and clinical benefit of artificial intelligence and predictive tools.

Among tools developed by our group, we developed an electronic health record-based machine learning algorithm to predict acute care visits during outpatient cancer therapy (Hong, JCO Clin Cancer Inform 2018). This model was recently evaluated in one of the first prospective randomized studies of clinical machine learning, demonstrating that its use directed clincial management to reduce rates of acute care during cancer therapy (Hong, J Clin Oncol 2020).

We have developed a range of other predictive models to improve cancer care, including risk stratification to improve care for patients with short-term mortality (Wu, JAMA Netw Open), identifying favorable characteristics in patients with oligometastatic cancers (Hong, PLOS ONE 2018) and predicting testosterone recovery following androgen deprivation (Spiegel, Int J Radiat Oncol Biol Phys 2019).

Learning from real world data

Real world data offers insights that can complement randomized controlled clinical trials. This can provide information regarding practice patterns and generate knowledge to inform future trials. Our team has expertise analyzing data from electronic health records, national databases, institutional experiences, claims data, and payor data to both learn from prior experiences and characterize data limitations (Hong, JAMA Netw Open 2021).

Among prior studies investigating real world data, we have demonstrated the disparities in patient enrollment to online patient portals (Sinha, JAMA Oncology 2021). We have also evaluated practice patterns in a range of cancers including lung (Hong, J Natl Compr Canc Netw 2019), prostate, and gynecologic malignancies. Among these studies, we have used Medicare provider data to identify disparities in access to specialized therapies for prostate and gynecologic malignancies (Hong, Brachytherapy 2018).

Unlocking unstructured clinical data

Large amounts of data are captured in free-text clinical notes, which can pose challenges to clinical study. Extracting this unstructured information can both inform clinical research and facilitate deployable computational tools.

We have adapted clinical natural language processing pipelines to extract patient outcomes during cancer therapy from routine clinical documentation. Our studies indicate high variability even among human experts, which may motivate the use of computational approaches (Fairchild, Int J Radiat Oncol Biol Phys 2020). Our natural language processing pipeline facilitates systematic extraction of patient symptoms during cancer therapy (Hong, JAMIA Open 2020).

Our team

Julian Hong, MD, MS

Principal Investigator

William Chen, MD

Resident Physician

UCSF Radiation Oncology

Meera Garriga, BA

Medical Student

UCSF School of Medicine

Yash Huilgol, AB

Medical and Graduate Student

UC Berkeley-UCSF Joint Medical Program

Izzy Friesner, BA

Clinical Data Scientist

UCSF Radiation Oncology,

Bakar Computational Health Sciences Institute

Havish Kantheti, MS

Medical Student

Texas A&M College of Medicine

Morgan Lafferty, BA

Research Assistant

UC Berkeley Computer Science,

Applied Mathematics

Divya Natesan, MD

Resident Physician

Duke Radiation Oncology

Lisa Ni, MD

Resident Physician

UCSF Radiation Oncology

Sumi Sinha, MD

Resident Physician

UCSF Radiation Oncology

Saira Somnay

Undergraduate

UC Berkeley Molecular and Cellular Biology

Travis Zack, MD, PhD

Fellow

UCSF Internal Medicine

William Zhang

Undergraduate

UC Berkeley Computer Science

Collaborators

Atul Butte, MD, PhD (Bakar Computational Health Sciences Institute, UCSF)

Lauren Boreta, MD (Radiation Oncology, UCSF)

Steve Braunstein, MD, PhD (Radiation Oncology, UCSF)

Felix Feng, MD (Radiation Oncology, UCSF)

Osama Mohamad, MD, PhD (Radiation Oncology, UCSF)

Olivier Morin, PhD (Radiation Oncology, UCSF)

Catherine Park, MD (Radiation Oncology, UCSF)

Manisha Palta, MD (Radiation Oncology, Duke University)

Jessie Tenenbaum, PhD (North Carolina Department of Health and Human Services; Biostatistics and Bioinformatics, Duke University)

Daphna Spiegel, MD, MS (Radiation Oncology, Beth Israel Deaconess Medical Center; Harvard Medical School)

Alumni

Andrew Fairchild, MD, MA - Radiation Oncologist, Piedmont Radiation Oncology

Anthony Lin, MD - Resident Physician, UCSF Internal Medicine

Kevin Miao, BA - Graduate Student, UC Berkeley

Gavril Moniaga, BS - Facebook

Nishali Naik, BA - Aon

Shawn Patel - Undergraduate, UC Berkeley

Isis Trenchard, PhD - Benchling

Eric Xu - Undergraduate, Stanford

Clark Wang - Undergraduate, UC Berkeley

Jon Wang, MS - Shimmer; Medical Student, UCSF School of Medicine

Sasha Yousefi, BA - Graduate Student, Stanford University

Positions

Join us!

Our group is always interested in finding new lab members and collaborators to contribute with clinical domain knowledge or computational expertise. Join our multidisciplinary team of clinicians and scientists in the Department of Radiation Oncology and the Bakar Computational Health Sciences Institute. Lab members are also be encouraged to take advantage of other training and learning opportunities within the UCSF Helen Diller Family Comprehensive Cancer Center, the Bakar Computational Health Sciences Institute, and other related entities at UCSF.

If you are interested in joining the lab or collaborating on research in a role not listed below, please contact us.

Post-doctoral opportunities

We are recruiting highly motivated investigators to develop algorithms and analyses with multi-modal clinical data to guide personalized oncology care. Projects will be tailored to the individual's prior experience and interests but will center on the utilization of computational techniques to drive clinical prediction or discover new clinical insights using routine clinical data (including electronic health records, imaging, digital pathology, and genomic data).

Ideal candidates will have an MD and/or PhD with a strong background in bioinformatics or related computational science and demonstrated publication record. Strong problem-solving skills, creative thinking, and programming ability are required. Applicants must possess good communication skills and be fluent in both spoken and written English. Prior experience in machine learning, natural language processing, network analysis, and computer vision are a plus. Natural language processing is a current priority area of the lab.

The post-doctoral fellow will join a multidisciplinary team of clinicians and scientists in the Department of Radiation Oncology and the Bakar Computational Health Sciences Institute.

Further details are available on the UCSF Office of Career and Professional Development website.

Undergraduate and post-baccalaureate opportunities

Our lab welcomes undergraduates and recent graduates with an interest in medicine and informatics to join us. We hope that this experience will provide preparation for future opportunities in medicine and informatics. Clinical shadowing opportunities are available to those interested.

UC Berkeley students can apply through the Undergraduate Research Apprentice Program.

We are recruiting for a Data Science Research Specialist, application available on UCSF Recruit.

Clinical trainee opportunities

Our lab welcomes medical students, residents, and fellows from all backgrounds to join us. Projects can be tailored based on the trainee's areas of interest and computational experience (not required).

UCSF medical students and trainees can apply at UCSF LabSpot.

Selected publications

2021

Disparities in Electronic Health Record Patient Portal Enrollment Among Oncology Patients. Sinha S, Garriga M, Naik N, McSteen BW, Odisho AY, Lin A, Hong JC. JAMA Oncol. 2021 Apr 8. doi: 10.1001/jamaoncol.2021.0540. PMID: 33830178.

Association of mental health diagnosis with race and all-cause mortality after a cancer diagnosis: Large-scale analysis of electronic health record data. Chen WC, Boreta L, Braunstein SE, Rabow MW, Kaplan LE, Tenenbaum JD, Morin O, Park CC, Hong JC. Cancer. 2021 Sep 22. doi: 10.1002/cncr.33903. PMID: 34550601.

Strategies to Turn Real-world Data Into Real-world Knowledge. Hong JC. JAMA Netw Open. 2021 Oct 1;4(10):e2128045. doi: 10.1001/jamanetworkopen.2021.28045. PMID: 34618043.

Assessing Clinical Outcomes in a Data-Rich World-A Reality Check on Real-World Data. Hong JC, Butte AJ. JAMA Netw Open. 2021 Jul 1;4(7):e2117826. doi: 10.1001/jamanetworkopen.2021.17826. PMID: 34309673.

An artificial intelligence framework integrating longitudinal electronic health records with real-world data enables continuous pan-cancer prognostication. Morin O, Vallieres M, Braunstein S, Barrios Ginart J, Upadhaya T, Woodruff HC, Zwanenburg A, Chatterjee A, Villanueva-Meyer JE, Valdez G, Chen W, Hong JC, Yom SS, Solberg TD, Lock S, Seuntjens J, Park C, Lambin P. Nat Cancer 2, 709–722 (2021). https://doi.org/10.1038/s43018-021-00236-2.

Risk Stratification for Imminent Risk of Death at the Time of Palliative Radiotherapy Consultation. Wu SY, Yee E, Vasudevan HN, Fogh SE, Boreta L, Braunstein SE, Hong JC. JAMA Netw Open. 2021 Jul 1;4(7):e2115641. doi: 10.1001/jamanetworkopen.2021.15641. PMID: 34196716.

Characterizing chronological accumulation of comorbidities in healthy veterans: a computational approach. Hong JC, Hauser ER, Redding TS 4th, Sims KJ, Gellad ZF, O'Leary MC, Hyslop T, Madison AN, Qin X, Weiss D, Bullard AJ, Williams CD, Sullivan BA, Lieberman D, Provenzale D. Sci Rep. 2021 Apr 14;11(1):8104. doi: 10.1038/s41598-021-85546-2. PMID: 33854078.

2020

System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning–Directed Clinical Evaluations During Radiation and Chemoradiation. Hong JC, Eclov NCW, Dalal NH, Thomas SM, Stephens SJ, Malicki M, Shields S, Cobb A, Mowery YM, Niedzwiecki D, Tenenbaum JD, Palta M. J Clin Oncol. 2020 Sep 4:JCO2001688. doi: 10.1200/JCO.20.01688. PMID: 32886536.

Closing the Gap Between Machine Learning and Clinical Cancer Care-First Steps Into a Larger World. Kang J, Morin O, Hong JC. JAMA Oncol. 2020 Sep 24.doi: 10.1001/jamaoncol.2020.4314. PMID: 32970129.

Natural language processing for abstraction of cancer treatment toxicities: accuracy versus human experts. Hong JC, Fairchild AT, Tanksley JP, Palta M, Tenenbaum JD. JAMIA Open. 2020 Dec 5. doi: 10.1093/jamiaopen/ooaa064.

Inter-rater reliability in toxicity identification: Limitations of current standards. Fairchild AT, Tanksley JP, Tenenbaum JD, Palta M, Hong JC. Int J Radiat Oncol Biol Phys. 2020 May 1. pii: S0360-3016(20)31084-1. doi: 10.1016/j.ijrobp.2020.04.040. PMID: 32371073.

Electronic health record (EHR) data mining for AI healthcare. Lin AL, Chen WC, Hong JC. Artificial Intelligence in Medicine: Technical Basis and Clinical Applications. Xing L, Giger ML, Min J, editors. Amsterdam: Elsevier; 2020.

2019

An Exploratory Radiomics Approach to Quantifying Pulmonary Function in CT Images. Lafata KJ, Zhou Z, Liu JG, Hong J, Kelsey CR, Yin FF. Sci Rep. 2019 Aug 8;9(1):11509. doi: 10.1038/s41598-019-48023-5. PMID: 31395937.

A Nomogram for Testosterone Recovery After Combined Androgen Deprivation and Radiation Therapy for Prostate Cancer. Spiegel DY, Hong JC, Oyekunle T, Waters L, Lee WR, Salama JK, Koontz BF. Int J Radiat Oncol Biol Phys. 2019 Mar 15;103(4):834-842. doi: 10.1016/j.ijrobp.2018.11.007. Epub 2018 Nov 10. PMID: 30419308.

Increasing PET Use in Small Cell Lung Cancer: Survival Improvement and Stage Migration in the VA Central Cancer Registry. Hong JC, Boyer MJ, Spiegel DY, Williams CD, Tong BC, Shofer SL, Moravan MJ, Kelley MJ, Salama JK. J Natl Compr Canc Netw. 2019 Feb;17(2):127-139. doi: 10.6004/jnccn.2018.7090. PMID: 30787126.

Radiotherapy Treatment Planning in the Age of AI: Are We Ready Yet? Zheng D, Hong JC, Wang C, Zhu X. Technol Cancer Res Treat. 2019 Jan-Dec;18:1533033819894577. doi: 10.1177/1533033819894577. PMID: 31858890.

Artificial Intelligence in Radiotherapy Treatment Planning: Present and Future. Wang C, Zhu X, Hong JC, Zheng D. Technol Cancer Res Treat. 2019 Jan 1;18:1533033819873922. doi: 10.1177/1533033819873922. PMID: 31495281.

2018

Predicting Emergency Visits and Hospital Admissions During Radiation and Chemoradiation: An Internally Validated Pretreatment Machine Learning Algorithm. Hong JC, Niedzwiecki D, Palta M, Tenenbaum JD. JCO Clin Cancer Inform. 2018 Dec;2:1-11. doi: 10.1200/CCI.18.00037. PMID: 30652595.

High-volume providers and brachytherapy practice: A Medicare provider utilization and payment analysis. Hong JC, Spiegel DY, Havrilesky LJ, Chino JP. Brachytherapy. 2018 Nov-Dec;17(6):906-911. doi: 10.1016/j.brachy.2018.07.007. Epub 2018 Jul 26. PMID: 30057292.

Intratreatment Response Assessment With 18F-FDG PET: Correlation of Semiquantitative PET Features With Pathologic Response of Esophageal Cancer to Neoadjuvant Chemoradiotherapy. Tandberg DJ, Cui Y, Rushing CN, Hong JC, Ackerson BG, Marin D, Zhang X, Czito BG, Willett CW, Palta M. Int J Radiat Oncol Biol Phys. 2018 Nov 15;102(4):1002-1007. doi: 10.1016/j.ijrobp.2018.07.187. Epub 2018 Jul 25. PMID: 30055238.

Association of Interim FDG-PET Imaging During Chemoradiation for Squamous Anal Canal Carcinoma With Recurrence. Hong JC, Cui Y, Patel BN, Rushing CN, Faught AM, Eng JS, Higgins K, Yin FF, Das S, Czito BG, Willett CG, Palta M. Int J Radiat Oncol Biol Phys. 2018 Nov 15;102(4):1046-1051. doi: 10.1016/j.ijrobp.2018.04.062. Epub 2018 May 4. PMID: 29891206.

Classification for long-term survival in oligometastatic patients treated with ablative radiotherapy: A multi-institutional pooled analysis. Hong JC, Ayala-Peacock DN, Lee J, Blackstock AW, Okunieff P, Sung MW, Weichselbaum RR, Kao J, Urbanic JJ, Milano MT, Chmura SJ, Salama JK. PLoS One. 2018 Apr 12;13(4):e0195149. doi: 10.1371/journal.pone.0195149. PMID: 29649281.