|
View: |
Part 1: Document Description
|
|
Citation |
|
|---|---|
|
Title: |
SalChartQA: Question-driven Saliency on Information Visualisations (Dataset and Reproduction Data) |
|
Identification Number: |
doi:10.18419/darus-3884 |
|
Distributor: |
DaRUS |
|
Date of Distribution: |
2024-01-26 |
|
Version: |
1 |
|
Bibliographic Citation: |
Wang, Yao, 2024, "SalChartQA: Question-driven Saliency on Information Visualisations (Dataset and Reproduction Data)", https://doi.org/10.18419/darus-3884, DaRUS, V1 |
|
Citation |
|
|
Title: |
SalChartQA: Question-driven Saliency on Information Visualisations (Dataset and Reproduction Data) |
|
Alternative Title: |
SalChartQA |
|
Identification Number: |
doi:10.18419/darus-3884 |
|
Authoring Entity: |
Wang, Yao (Universität Stuttgart) |
|
Other identifications and acknowledgements: |
Abdelhafez, Abdullah |
|
Other identifications and acknowledgements: |
Wang, Yao |
|
Other identifications and acknowledgements: |
Bulling, Andreas |
|
Grant Number: |
251654672 |
|
Distributor: |
DaRUS |
|
Access Authority: |
Bulling, Andreas |
|
Depositor: |
Wang, Yao |
|
Date of Deposit: |
2024-01-19 |
|
Holdings Information: |
https://doi.org/10.18419/darus-3884 |
|
Study Scope |
|
|
Keywords: |
Computer and Information Science, Information Visualization, Data Visualization, Deep Learning |
|
Abstract: |
Understanding the link between visual attention and user’s needs when visually exploring information visualisations is under-explored due to a lack of large and diverse datasets to facilitate these analyses. To fill this gap, we introduce SalChartQA - a novel crowd-sourced dataset that uses the BubbleView interface as a proxy for human gaze and a question-answering (QA) paradigm to induce different information needs in users. SalChartQA contains 74,340 answers to 6,000 questions on 3,000 visualisations. Informed by our analyses demonstrating the tight correlation between the question and visual saliency, we propose the first computational method to predict question-driven saliency on information visualisations. Our method outperforms state-of-the-art saliency models, improving several metrics, such as the correlation coefficient and the Kullback-Leibler divergence. These results show the importance of information needs for shaping attention behaviour and paving the way for new applications, such as task-driven optimisation of visualisations or explainable AI in chart question-answering. The files of this dataset are documented in <a href="https://darus.uni-stuttgart.de/file.xhtml?fileId=276393">README.md</a>. |
|
Date of Collection: |
2023-07-20-2023-08-15 |
|
Kind of Data: |
information visualisation |
|
Kind of Data: |
deep learning code |
|
Kind of Data: |
visual saliency maps |
|
Methodology and Processing |
|
|
Sources Statement |
|
|
Data Access |
|
|
Other Study Description Materials |
|
|
Related Studies |
|
|
the visualisations in SalChartQA are from the ChartQA dataset: <a href="https://github.com/vis-nlp/ChartQA">https://github.com/vis-nlp/ChartQA</a> |
|
|
Related Publications |
|
|
Citation |
|
|
Title: |
Y. Wang, W. Wang, A. Abdelhafez, M. Elfares, Z. Hu, M. Bâce, A. Bulling. "SalChartQA: Question-driven Saliency on Information Visualisations", in Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. |
|
Identification Number: |
10.1145/3613904.3642942 |
|
Bibliographic Citation: |
Y. Wang, W. Wang, A. Abdelhafez, M. Elfares, Z. Hu, M. Bâce, A. Bulling. "SalChartQA: Question-driven Saliency on Information Visualisations", in Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. |
|
Label: |
README.md |
|
Text: |
Readme document |
|
Notes: |
text/markdown |
|
Label: |
SalChartQA.zip |
|
Text: |
The SalChartQA dataset, containing fixationByVis, image_questions.json, raw_img, saliency_all, saliency_ans and unified_approved.csv |
|
Notes: |
application/zip |
|
Label: |
dataset_new.py |
|
Text: |
dataloader for SalChartQA |
|
Notes: |
text/x-python-script |
|
Label: |
env.py |
|
Text: |
python envorinment $TORCH_HOME and $TRANSFORMERS_CACHE |
|
Notes: |
text/x-python-script |
|
Label: |
evaluation.py |
|
Text: |
evaluation script to load VisSalFormer weights and make predictions |
|
Notes: |
text/x-python-script |
|
Label: |
evaluation.sh |
|
Text: |
bash script to run evaluation.py |
|
Notes: |
text/x-sh |
|
Label: |
model_swin.py |
|
Text: |
definition of the VisSalFormer model |
|
Notes: |
text/x-python-script |
|
Label: |
tokenizer_bert.py |
|
Text: |
tokenizer of Bert |
|
Notes: |
text/x-python-script |
|
Label: |
VisSalFormer_weights.tar |
|
Text: |
weights of VisSalFormer |
|
Notes: |
application/x-tar |