flâneur

Ahmed Durrani

0 followers · 4 following · 253 views

on the atlas — 1

highlights — 15

  • We leverage PyTorch, using the parameter-efficient fine-tuning [52] and the generative pre-trained transformers quantization [28] libraries to implement QLoRA. We fine-tune models with QLoRA for five epochs using the Adam optimizer with weight decay fix [51]. An initial learning rate of 1 e − 3 was decayed linearly to 1 e − 4 after a 100-step warm-up; we determined this configuration after experimenting with different learning rates and schedulers. To achieve an effective batch size of 24 on each experiment, we adjust (1) individual batch size and (2) number of gradient accumulation steps to f…
    2309.07430.pdf
  • Temperature determines conditional probability distributions during sampling, hence impacting how often the model will output less likely tokens. Higher temperatures lead to more randomness and “creativity,” while lower temperatures produce more deterministic outputs
    2309.07430.pdf
  • or example, we achieve better performance by nudging the model to have expertise in medicine than an expertise in wizardry
    2309.07430.pdf
  • The goal of this task is to summarize a doctor-patient conversation into an “assessment and plan” paragraph. We employ the ACI-Bench dataset [1, 88, 89], which contains (1) 207 doctor-patient conversations (2) corresponding patient visit notes, which were first generated by a seq2seq model and subsequently corrected and validated by expert medical scribes and physicians
    2309.07430.pdf
  • ProbSum contains (1) progress notes averaging > 1 , 000 tokens and substantial presence of unlabeled numerical data, e.g. dates and test results (2) corresponding problem lists created by attending medical experts in the ICU.
    2309.07430.pdf
  • notes The goal of this task is to generate a “problem list,” or condensed list of diagnoses and medical problems using the provider’s progress notes during hospitalization.
    2309.07430.pdf
  • Question summarization consists of generating a condensed question expressing the minimum information required to find correct answers to the original question [5]
    2309.07430.pdf
  • Low-rank adaptation (LoRA) [35] has emerged as an effective, lightweight approach for fine-tuning LLMs by altering a small subset of model weights—often < 0 . 1% [81].
    2309.07430.pdf
  • ICL is a lightweight adaptation method that requires no altering of model weights; instead, one includes a handful of in-context examples within the model prompt [43
    2309.07430.pdf
  • This T5 model family has proven effective for various clinical NLP tasks [44, 81].
    2309.07430.pdf
  • We include prominent seq2seq models due to their strong summarization performance
    2309.07430.pdf
  • To enable domain adaptation with limited computational resources, prefix tuning [45] and low-rank adaptation (LoRA) [35] have emerged as effective methods that require tuning less than 1% of total parameters over a small training set. LoRA has been shown to work well for medical question-answering [76] and summarizing radiology reports [81]. Another adaptation method, requiring no parameter tuning, is in-context learning: supplying the LLM with task-specific examples in the prompt [43]. Because in-context learning does not alter model weights, it can be performed with black-box model access us…
    2309.07430.pdf
  • Our results demonstrate that LLMs often outperform human experts for clinical text summarization across the diverse range of documents we evaluate. This implies that LLMs could be leveraged to reduce documentation load and thus support clinicians—not supplant them.
    2309.07430.pdf
  • he subsequent evaluation via NLP metrics is the most comprehensive assessment of contemporary LLMs for clinical text summarization
    2309.07430.pdf
  • Crucially, machine-generated summaries must be non-inferior to that of seasoned clinicians—especially when used to support sensitive clinical decision-making
    2309.07430.pdf