A Novel model for Chart-to-Text Generation by utilizing NN models

Abstract

Abstract : This research introduces a novel neural network-based model designed to enhance the automatic generation of textual descriptions for scientific charts, building on the foundational work of "SCICAP: Generating Captions for Scientific Figures." Our proposed model combines advanced Convolutional Neural Network (CNN) architectures for effective feature extraction from scientific charts with multiple Recurrent Neural Network (RNN) layers to generate descriptive, accurate captions. This study is distinguished due to its comparative analysis of different RNN architectures, such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), to identify the most effective model for textual caption generation. Central to our methodology is the utilization of CNNs to parse visual input, transforming complex chart images into a rich set of feature vectors. These vectors serve as inputs for the sequential RNN layers, which are tasked with constructing coherent and contextually relevant textual descriptions. The performance of various CNN-RNN combinations is rigorously evaluated using the BLEU-4 (Bilingual Evaluation Understudy) metric, a standard in the natural language processing field for assessing the linguistic quality of machine-generated text against human-written references. Our Research showed that Bi- LSTM performed the best with a BLEU score of 0.3, future researches will be required for more reliable captioning.