Sentence-Level Rationale Extraction Methods Large For Closed-domain Question Answering Explainability

This poster addresses the problem of Rationale Extraction
(RE) from Natural Language Processing: we have a context
(C), a related question (Q) and its answer (A) and the goal
is to find the set of sentences (R) in the context that is suffi-
cient to produce the answer. We have constructed a database
where each entry is composed of the four terms (C, Q, A, R)
to explore different methods in the particular case where the
answer is a full sentence. The methods studied are N-grams,
embedding similarity, classifier-based and attention-focused.
Each one has been scored based on its ability to find the best
rationale using a sentence overlap metric akin to the Intersec-
tion over Union (IoU). Results show that the best scores were
achieved by the classifier-based approach. Additionally, we
showed the increasing difficulty of the task with the number
of sentences in the context. Finally, we underlined a correla-
tion in the case of the attention-focused method between the
attention performance and the ability of the underlying Large
Language model to provide given C and Q an answer similar
to A.