Total: 1
This paper describes a simple yet robust approach to performing reference-free estimation of the quality of automatically-generated clinical notes derived from doctor-patient conversations. In the absence of human-written reference notes, this approach works by generating a diverse collection of "pseudo-reference notes" and comparing the generated note against those pseudo-references. This method has been applied to estimate the quality of clinical note sections generated by three different note generation models, using a collection of evaluation metrics that are based on natural language inference and clinical concept extraction. Our experiments show the proposed approach is robust to the choice of note generation models, and consistently produces higher correlations with reference-based counterparts when compared against a strong baseline method.