# trivedi2018NLPReViz

# NLPReViz: an interactive tool for natural language processing on clinical text


Trivedi et al. introduce NLPReViz, a visual analytic and visualization tool that supports interactive Natural Language Processing (NLP) and machine learning in real time [1].

Users are able to train, review and revise NLP models by rectifying results from the previous execution. Re-trained models will be used for next execution and provide a more accurate result.

A user study including 9 participants with MD degrees and knowledge of colonoscopy was conducted. The average System Usability Scale [2] score of 70.56 was given, indicating the system is considered highly usable.

Figure 1. An overview of NLPReViz [1:1].


Figure 1 shows an overview of NLPReViz visualizing 453 colonoscopy reports. Section A shows a grid view of extracted boolean variables in columns and the corresponding documents in rows. Statistics on the selected variable are shown in Section B) the distribution of classification results, and Section C) the list of top indicators for that variable across all documents, and Section D) the list of indicators from the corresponding document.

Section E shows the content of the selected document, with indicators being highlighted. Section F allows the user to revise the result and feedback it into the re-train process. Section G visualizes a Word Tree which allows the user to select an indicator and drill down on the indicator. A search function is also available. Section H shows an aggregated list of user feedback, and a button to trigger the re-train process.

Related Work

NLPReVis is developed upon the work of Harkema et al. [3], which provides a collection of NLP-ed EHR documents with gold standard labels. Support Vector Machine is used as the machine learning classifier for predicting the outcomes based on the extracted information from the dataset.

Interactive information visualization techniques are inspired by Jigsaw [4].

Word Tree is built upon the work of Wattenberg and Viegas [5].

Data Characteristics

  • Data source: The Medical ARchival System (MARS) of the University of Pittsburgh Medical Center
  • Size: 453 patients
  • Spatial dimensionality: 2D
  • Temporal dimensionality: static
  • Type: multivariate
  • Libraries used: front end written in angular (javascript), server written in Java

Visualization Techniques

  • Word Tree
  • Grid view

Papers Cited


Years Spanned


Application Domain

  • Electronic Health Record Visualization
  • Natural Language Processing
  • Medical focus: Colonoscopy


  1. Trivedi, G., Pham, P., Chapman, W. W., Hwa, R., Wiebe, J., & Hochheiser, H. (2018). NLPReViz: An interactive tool for natural language processing on clinical text. Journal of the American Medical Informatics Association, 25(1), 81–87. https://doi.org/10.1093/jamia/ocx070 ↩︎ ↩︎

  2. Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry. ↩︎

  3. Harkema, H., Chapman, W. W., Saul, M., Dellon, E. S., Schoen, R. E., & Mehrotra, A. (2011). Developing a natural language processing application for measuring the quality of colonoscopy procedures. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2011-000431 ↩︎

  4. Stasko, J., Görg, C., & Liu, Z. (2008). Jigsaw: Supporting investigative analysis through interactive visualization. Information Visualization. https://doi.org/10.1057/palgrave.ivs.9500180 ↩︎

  5. Wattenberg, M., & Viégas, F. B. (2008). The word tree, an interactive visual concordance. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2008.172 ↩︎

🔄 Last Updated: 7/8/2020, 9:48:18 PM