Advances in Knowledge Discovery and Data Mining

While some methods are confined to linear embeddings and others exhibit limited robustness, high-dimensional time series factorization techniques employ scalable matrix factorization for forecasting in latent space. This paper introduces a novel factorization method that employs a non-contrastive approach, guiding an autoencoder-like architecture to extract robust latent series while minimizing redundant information within the embeddings. The resulting learned representations are utilized by a temporal forecasting model, generating forecasts within the latent space, which are subsequently decoded back to the original space through the decoder. Extensive experiments demonstrate that our model achieves state-of-te-art performance on numerous commonly used datasets.

Many engineering applications rely on simulations based on partial differential equations. Different numerical schemes to approximate solutions exist. These schemes typically require setting parameters to appropriately model the problem at hand. We study the problem of parameter selection for applications that rely on simulations, where standard methods like grid search are computationally prohibitive. Our solution supports engineers in setting parameters based on knowledge gained through analyzing metadata acquired while partially executing specific simulations. Selecting these so-called farming runs of simulations is guided by an optimization algorithm that leverages the acquired knowledge. Experiments demonstrate that our solution outperforms state-of-the-art approaches and generalizes to a wide range of application settings.

We propose a variational autoencoder (VAE)-based model for building forward and inverse structure-property linkages, a problem of paramount importance in computational materials science. Our model systematically combines VAE with regression, linking the two models through a two-level prior conditioned on the regression variables. The regression loss is optimized jointly with the reconstruction loss of the variational autoencoder, learning microstructure features relevant for property prediction and reconstruction. The resultant model can be used for both forward and inverse prediction i.e., for predicting the properties of a given microstructure as well as for predicting the microstructure required to obtain given properties. Since the inverse problem is ill-posed (one-to-many), we derive the objective function using a multi-modal Gaussian mixture prior enabling the model to infer multiple microstructures for a target set of properties. We show that for forward prediction, our model is as accurate as state-of-the-art forward-only models. Additionally, our method enables direct inverse inference. We show that the microstructures inferred using our model achieve desired properties reasonably accurately, avoiding the need for expensive optimization loops.

In this paper, we propose a Weighted Cross-modal Aggregation network (WCAN) for rumor detection in order to combine highly correlated features in different modalities and obtain a unified representation in the same space. WCAN exploits an adversarial training method to add perturbations to text features to enhance model robustness. Specifically, we devise a weighted cross-modal aggregation (WCA) module that measures the distance between text, image and social graph modality distributions using KL divergence, which leverages correlations between modalities. By using MSE loss, the fusion features are progressively closer to the original features of the image and social graph while taking into account all of the information from each modality. In addition, WCAN includes a feature fusion module that uses dual-modal co-attention blocks to dynamically adjust features from three modalities. Experiments are conducted on two datasets, WEIBO and PHEME, and the experimental results demonstrate the superior performance of the proposed method.

Social media echo chambers are known to be common sources of misinformation and harmful ideologies that have detrimental impacts on society. Therefore, techniques to detect echo chambers are of great significance. Reinforcement of supporting opinions and rejection of dissenting opinions are two significant echo chamber properties that help detecting them in social networks. However, existing echo chamber detection methods do not capture the opinion rejection behaviour, which leads to poor echo chamber detection accuracy. Measures used by them do not facilitate quantifying both properties simultaneously while preserving the connectivity between echo chamber members. To address this problem, we propose a new measure, Signed Echo (SEcho) that quantifies opinion reinforcement and rejection properties of echo chambers and an echo chamber detection algorithm, Signed Echo Detection Algorithm (SEDA) based on this measure, which preserves the connectivity among echo chamber members. The experimental results for real-world data show that SEDA outperforms the state-of-the-art echo chamber detection methods in detecting the communities with echo chamber properties, such as reinforcement of supporting opinions, rejection of dissenting opinions, connectivity between community members, spread of mis/disinformation and emotional contagion.

Despite the success of prompt learning-based models in text generation tasks, they still suffer from the introduction of external commonsense knowledge, especially from biased knowledge introduction. In this work, we propose KiProL, a knowledge-injected prompt learning framework to improve language generation and training efficiency. KiProL tackles ineffective learning and utilization of knowledge, reduces the biased knowledge introduction, as well as high training expenses. Then, we inject the recommended knowledge into the prompt learning encoder to optimize guiding prefixes without modifying the pre-trained model’s parameters, resulting in reduced computational expenses and shorter training duration. Our experiments on two publicly available datasets (i.e., Explanation Generation and Story Ending Generation) show that KiProL outperforms baseline models. It improves fluency by an average of 2%, while diversity increases by 3.4% when compared with advanced prompt learning-based methods. Additionally, KiProL is 45% faster than the state-of-the-art knowledgeable, prompt learning method in training efficiency.

The WSDM 2023 Toloka VQA challenge introduces a new Grounding-based Visual Question Answering (GVQA) dataset, elevating multimodal task complexity. This challenge diverges from traditional VQA by requiring models to identify a bounding box in response to an image-question pair, aligning with Visual Grounding tasks. Existing VG approaches, when applied to GVQA, often necessitate external data or larger models for satisfactory results, leading to high computational demands. We approach this as a language modeling problem, utilizing prompt tuning with multiple state-of-the-art VQA models. Our method, operating solely on an NVIDIA RTX3090 GPU without external data, secured third place in the challenge, achieving an Intersection over Union (IoU) of 75.658. Our model notably provides explainability between textual and visual data through its attention mechanism, offering insights into its decision-making process. This research demonstrates that high performance in GVQA can be achieved with minimal resources, enhancing understanding of model dynamics and paving the way for improved interpretability and efficiency. Our code is available here: https://​github.​com/​IKMLab/​GViG.​git

The detection of misinformation as “fake news” is vital for a well-informed and highly functioning society. Most of the recent works on the identification of fake news make use of deep learning and large language models to achieve high levels of performance. However, traditional fake news detection methods may lack a nuanced “understanding” of content, including ignoring important information in the form of potential aspects in documents or relying on external knowledge sources to identify such aspects. This paper focuses on aspect-based fake news detection, which aims to uncover deceptive narratives through fine-grained analysis of news articles. We propose a novel aspect-based fake news detection method based on a lower, paragraph-level attention mechanism that identifies different aspects within a news-related document. The proposed approach utilizes aspects to provide concise yet meaningful representations of long news articles without reliance on any external reference knowledge. We investigate the impact of learning aspects from documents on the effectiveness of fake news detection. Our experiments on four benchmark datasets show statistically significant improvements over the results of several baseline models.

Recent Query Auto-completion (QAC) systems leverage natural language generation or pre-trained language models (PLMs) to demonstrate remarkable performance. However, these systems also suffer from biased and toxic completions. Efforts have been made to address language detoxification within PLMs using controllable text generation (CTG) techniques, involving training with non-toxic data and employing decoding time approaches. As the completions for QAC systems are usually short, these existing CTG methods based on decoding and training are not directly transferable. Towards these concerns, we propose the first public QAC detoxification model, Detoxifying Query Auto-Completion (or DQAC), which utilizes adapters in a CTG framework. DQAC operates on latent representations with no additional overhead. It leverages two adapters for toxic and non-toxic cases. During inference, we fuse these representations in a controlled manner that guides the generation of query completions towards non-toxicity. We evaluate toxicity levels in the generated completions across two real-world datasets using two classifiers: a publicly available (Detoxify) and a search query-specific classifier which we develop (QDetoxify). DQAC consistently outperforms all existing baselines and emerges as a state-of-the-art model providing high quality and low toxicity. We make the code publicly available\(^{1}\).(\(^{1}\) https://​shorturl.​at/​zJ024)

This study addresses the challenge of detecting semantic column types in relational tables, a key task in many real-world applications. While language models like BERT have improved prediction accuracy, their token input constraints limit the simultaneous processing of intra-table and inter-table information. We propose a novel approach using Graph Neural Networks (GNNs) to model intra-table dependencies, allowing language models to focus on inter-table information. Our proposed method not only outperforms existing state-of-the-art algorithms but also offers novel insights into the utility and functionality of various GNN types for semantic type detection. The code is available at https://​github.​com/​hoseinzadeehsan/​GAIT

In the realm of fake news detection, conventional Graph Neural Network (GNN) methods are often hamstrung by their dependency on non-textual auxiliary data for graph construction, such as user interactions and content spread patterns, which are not always accessible. Furthermore, these methods typically fall short in capturing the granular, intricate correlations within text, thus weakening their effectiveness. In this work, we propose Text-Clustering Graph Neural Network (TCGNN), a novel approach that circumvents these limitations by solely utilizing text to construct its detection framework. TCGNN innovatively employs text clustering to extract representative words and harnesses multiple clustering dimensions to encapsulate a multi-faceted representation of textual semantics. This multi-layered approach not only delves into the fine-grained correlations within text but also bridges them to a broader context, significantly enriching the model’s interpretative fidelity. Our rigorous experiments on a suite of benchmark datasets have underscored TCGNN’s proficiency, outperforming extant GNN-based models. This validates our premise that an adept synthesis of text clustering within a GNN architecture can profoundly enhance the detection of fake news, steering the course towards a more reliable and textually-aware future in information verification.

Aspect-Based Sentiment Analysis (ABSA) is a fine-grained linguistics problem that entails the extraction of multifaceted aspects, opinions, and sentiments from the given text. Both standalone and compound ABSA tasks have been extensively used in the literature to examine the nuanced information present in online reviews and social media posts. Current ABSA methods often rely on static hyperparameters for attention-masking mechanisms, which can struggle with context adaptation and may overlook the unique relevance of words in varied situations. This leads to challenges in accurately analyzing complex sentences containing multiple aspects with differing sentiments. In this work, we present adaptive masking methods that remove irrelevant tokens based on context to assist in Aspect Term Extraction and Aspect Sentiment Classification subtasks of ABSA. We show with our experiments that the proposed methods outperform the baseline methods in terms of accuracy and F1 scores on four benchmark online review datasets. Further, we show that the proposed methods can be extended with multiple adaptations and demonstrate a qualitative analysis of the proposed approach using sample text for aspect term extraction.

Given one or more time sequences, how can we extract their governing equations? Single and co-evolving time sequences appear in numerous settings, including medicine (neuroscience - EEG signals, cardiology - EKG), epidemiology (covid/flu spreading over time), physics (astrophysics, material science), marketing (sales and competition modeling; market penetration), and numerous more. Linear differential equations will fail, since the underlying equations are often non-linear (SIR model for virus/product spread; Lotka-Volterra for product/species competition, Van der Pol for heartbeat modeling).

We propose DiffFind and we use genetic algorithms to find suitable, parsimonious, differential equations. Thanks to our careful design decisions, DiffFind has the following properties - it is: (a) Effective, discovering the correct model when applied on real and synthetic nonlinear dynamical systems, (b) Explainable, gives succinct differential equations, and (c) Hands-off, requiring no manual hyperparameter specification.

DiffFind outperforms traditional methods (like auto-regression), includes as special case and thus outperforms a recent baseline (‘SINDy’), and wins first or second place for all 5 real and synthetic datasets we tried, often achieving excellent, zero or near-zero RMSE of 0.005.

Current work on Active Learning (AL) tends to assume that the relationship between input and target variables does not change, i.e., the oracle is static. However, oracles can be stream-like and exhibit concept drift, which requires updating the learned relationship. Standard drift detection and adaption methods rely on constantly observing the target variables, which is too costly in AL. Current work on AL for regression has not addressed the challenge of frequently drifting oracles. We propose a new AL method that estimates its error due to drift by learning statistics about how often and how severe drift occurs, based on a Gaussian Process model with a time-variant kernel. Whenever the estimated error reaches a user-required threshold, our model measures the target variables and recalibrates the learned relationship as well as the drift statistics. Our drift-aware model requires up to 20 times fewer measurements than widely used methods.

Graph Recurrent Neural Networks (GRNN) excel in time-series prediction by modeling complicated non-linear relationships among time-series. However, most GRNN models target small datasets that only have tens of time-series or hundreds of time-series. Therefore, they fail to handle large-scale datasets that have tens of thousands of time-series, which exist in many real-world scenarios. To address this scalability issue, we propose Evolving Super Graph Neural Networks (ESGNN), which target large-scale datasets and significantly boost model training. Our ESGNN models multivariate time-series based on super graphs, where each super node is associated with a set of time-series that are highly correlated with each other. To further precisely model dynamic relationships between time-series, ESGNN quickly updates super graphs on the fly by using the LSH algorithm to construct the super edges. The embeddings of super nodes are learned through end-to-end learning and are then used with each target time-series for forecasting. Experimental result shows that ESGNN outperforms previous state-of-the-art methods with a significant runtime speedup (\(3{\times }\)–\(40{\times }\) faster) and space-saving (\(5{\times }\)–\(4600{\times }\) less), while only sacrificing little or negligible prediction accuracy. An ablation study is also conducted to investigate the effectiveness of the number of super nodes and the graph update interval.

Multivariate time series representation learning employs unsupervised tasks to extract meaningful representations from time series data, enabling their application in diverse downstream tasks. However, despite the promising advancements in contrastive learning-based representation learning, the study of task-related feature learning is still in its early stages. This gap exists because current unified representation learning frameworks lack the ability to effectively disentangle task-related features. To address this limitation, we propose DisT, a novel contrastive learning-based method for efficient task-related feature learning in time series representation. DisT disentangles task-related features by incorporating feature network structure learning and contrastive sample pair selection. Specifically, DisT incorporates a feature decoupling module, which prioritizes global features for time series classification tasks, while emphasizing periodic and seasonal features for forecasting tasks. Additionally, DisT leverages contrastive loss and task-related feature loss to adaptively select data augmentation methods, preserving task-relevant shared information between positive samples across different data and tasks. Experimental results on various multivariate time-series datasets including classification and forecasting tasks show that DisT achieves state-of-the-art performance.

Time series data plays a significant role in many research fields since it can record and disclose the dynamic trends of a phenomenon with a sequence of ordered data points. Time series data is dynamic, of variable length, and often contains complex patterns, which makes its analysis challenging especially when the amount of data is limited. In this paper, we propose a multi-view feature construction approach that can generate multiple feature sets of different resolutions from a single dataset and produce a fixed-length representation of variable-length time series data. Furthermore, we propose a multi-encoder-decoder Transformer (MEDT) architecture to effectively analyze these multi-view representations. Through extensive experiments using multiple benchmarks and a real-world dataset, our method shows significant improvement over the state-of-the-art methods.

Correlations between variables in complex ecosystems such as weather and financial markets lead to a great amount of dynamic and co-evolving time series data, posing a significant challenge to the current forecast methods. Discovering dynamic patterns (aka regimes) is crucial to an accurate forecast, especially for the interpretability of the outcome. In this paper, we develop a kernel-based method to learn effective representations for capturing dynamically changing regimes. Each such representation accounts for the non-linear interactions among multiple time series, thereby facilitating more effective regime discovery. On the basis of regime information, we build a regression model to forecast all the variables simultaneously for the next multiple time points. The results on six real-life datasets demonstrate that our method can yield the most accurate forecast (with the lowest root mean square error) in comparison with seven predictive models.

Time series forecasting attempts to predict future events by analyzing past trends and patterns. Although well researched, certain critical aspects pertaining to the use of deep learning in time series forecasting remain ambiguous. Our research primarily focuses on examining the impact of specific hyperparameters related to time series, such as context length and validation strategy, on the performance of the state-of-the-art MLP model in time series forecasting. We have conducted a comprehensive series of experiments involving 4800 configurations per dataset across 20 time series forecasting datasets, and our findings demonstrate the importance of tuning these parameters. Furthermore, in this work, we introduce the largest metadataset for time series forecasting to date, named TSBench, comprising 97200 evaluations, which is a twentyfold increase compared to previous works in the field. Finally, we demonstrate the utility of the created metadataset on multi-fidelity hyperparameter optimization tasks.

Semi-supervised time series classification has become an increasingly popular task due to the limited availability of labeled data in practice. Recently, Similarity-aware Time Series Classification (SimTSC) has been proposed to address the label scarcity problem by using a graph neural network on the graph generated from pairwise Dynamic Time Warping (DTW) distance of batch data. While demonstrating superior accuracy compared to the state-of-the-art deep learning models, SimTSC relies on pairwise DTW distance computation and thus has limited usability in practice due to the quadratic complexity of DTW. To address this challenge, we propose a novel efficient semi-supervised time series classification technique with a new graph construction module. Instead of computing the full DTW distance matrix, we propose to approximate the dissimilarity between instances in linear time using a lower bound, while retaining the relative proximity relationships one would have obtained via DTW. The experiments conducted on the ten largest datasets from the UCR archive demonstrate that our model can be up to 104x faster than SimTSC when constructing the graph on large datasets without significantly decreasing classification accuracy.

Reliable forecasting of traffic flow requires efficient modeling of traffic data. Indeed, different correlations and influences arise in a dynamic traffic network, making modeling a complicated task. Existing literature has proposed many different methods to capture traffic networks’ complex underlying spatial-temporal relations. However, given the heterogeneity of traffic data, consistently capturing both spatial and temporal dependencies presents a significant challenge. Also, as more and more sophisticated methods are being proposed, models are increasingly becoming memory-heavy and, thus, unsuitable for low-powered devices. To this end, we propose Spatio-Temporal Lightweight Graph GRU, namely STLGRU, a novel traffic forecasting model for predicting traffic flow accurately. Specifically, our proposed STLGRU can effectively capture dynamic local and global spatial-temporal relations of traffic networks using memory-augmented attention and gating mechanisms in a continuously synchronized manner. Moreover, instead of employing separate temporal and spatial components, we show that our memory module and gated unit can successfully learn the spatial-temporal dependencies with reduced memory usage and fewer parameters. Extensive experimental results on three real-world public traffic datasets demonstrate that our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency. Our code is available at https://​github.​com/​Kishor-Bhaumik/​STLGRU.