Instruction: What field is the article from?

Input:
Title: A New Approach to Intuitionistic Fuzzy Decision Making Based on Projection Technology and Cosine Similarity Measure
Abstract: For a multi-attribute decision making (MADM) problem, the information of
alternatives under different attributes is given in the form of intuitionistic
fuzzy number(IFN). Intuitionistic fuzzy set (IFS) plays an important role in
dealing with un-certain and incomplete information. The similarity measure of
intuitionistic fuzzy sets (IFSs) has always been a research hotspot. A new
similarity measure of IFSs based on the projection technology and cosine
similarity measure, which con-siders the direction and length of IFSs at the
same time, is first proposed in this paper. The objective of the presented
pa-per is to develop a MADM method and medical diagnosis method under IFS using
the projection technology and cosine similarity measure. Some examples are used
to illustrate the comparison results of the proposed algorithm and some
exist-ing methods. The comparison result shows that the proposed algorithm is
effective and can identify the optimal scheme accurately. In medical diagnosis
area, it can be used to quickly diagnose disease. The proposed method enriches
the exist-ing similarity measure methods and it can be applied to not only
IFSs, but also other interval-valued intuitionistic fuzzy sets(IVIFSs) as well.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Deep Intrinsic Decomposition with Adversarial Learning for Hyperspectral Image Classification
Abstract: Convolutional neural networks (CNNs) have been demonstrated their powerful
ability to extract discriminative features for hyperspectral image
classification. However, general deep learning methods for CNNs ignore the
influence of complex environmental factor which enlarges the intra-class
variance and decreases the inter-class variance. This multiplies the difficulty
to extract discriminative features. To overcome this problem, this work
develops a novel deep intrinsic decomposition with adversarial learning, namely
AdverDecom, for hyperspectral image classification to mitigate the negative
impact of environmental factors on classification performance. First, we
develop a generative network for hyperspectral image (HyperNet) to extract the
environmental-related feature and category-related feature from the image.
Then, a discriminative network is constructed to distinguish different
environmental categories. Finally, a environmental and category joint learning
loss is developed for adversarial learning to make the deep model learn
discriminative features. Experiments are conducted over three commonly used
real-world datasets and the comparison results show the superiority of the
proposed method. The implementation of the proposed method and other compared
methods could be accessed at https://github.com/shendu-sw/Adversarial Learning
Intrinsic Decomposition for the sake of reproducibility.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Few-Shot Classification & Segmentation Using Large Language Models Agent
Abstract: The task of few-shot image classification and segmentation (FS-CS) requires
the classification and segmentation of target objects in a query image, given
only a few examples of the target classes. We introduce a method that utilises
large language models (LLM) as an agent to address the FS-CS problem in a
training-free manner. By making the LLM the task planner and off-the-shelf
vision models the tools, the proposed method is capable of classifying and
segmenting target objects using only image-level labels. Specifically,
chain-of-thought prompting and in-context learning guide the LLM to observe
support images like human; vision models such as Segment Anything Model (SAM)
and GPT-4Vision assist LLM understand spatial and semantic information at the
same time. Ultimately, the LLM uses its summarizing and reasoning capabilities
to classify and segment the query image. The proposed method's modular
framework makes it easily extendable. Our approach achieves state-of-the-art
performance on the Pascal-5i dataset.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Deep Metric Learning for Computer Vision: A Brief Overview
Abstract: Objective functions that optimize deep neural networks play a vital role in
creating an enhanced feature representation of the input data. Although
cross-entropy-based loss formulations have been extensively used in a variety
of supervised deep-learning applications, these methods tend to be less
adequate when there is large intra-class variance and low inter-class variance
in input data distribution. Deep Metric Learning seeks to develop methods that
aim to measure the similarity between data samples by learning a representation
function that maps these data samples into a representative embedding space. It
leverages carefully designed sampling strategies and loss functions that aid in
optimizing the generation of a discriminative embedding space even for
distributions having low inter-class and high intra-class variances. In this
chapter, we will provide an overview of recent progress in this area and
discuss state-of-the-art Deep Metric Learning approaches.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Understanding the Inner Workings of Language Models Through Representation Dissimilarity
Abstract: As language models are applied to an increasing number of real-world
applications, understanding their inner workings has become an important issue
in model trust, interpretability, and transparency. In this work we show that
representation dissimilarity measures, which are functions that measure the
extent to which two model's internal representations differ, can be a valuable
tool for gaining insight into the mechanics of language models. Among our
insights are: (i) an apparent asymmetry in the internal representations of
model using SoLU and GeLU activation functions, (ii) evidence that
dissimilarity measures can identify and locate generalization properties of
models that are invisible via in-distribution test set performance, and (iii)
new evaluations of how language model features vary as width and depth are
increased. Our results suggest that dissimilarity measures are a promising set
of tools for shedding light on the inner workings of language models.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Length is a Curse and a Blessing for Document-level Semantics
Abstract: In recent years, contrastive learning (CL) has been extensively utilized to
recover sentence and document-level encoding capability from pre-trained
language models. In this work, we question the length generalizability of
CL-based models, i.e., their vulnerability towards length-induced semantic
shift. We verify not only that length vulnerability is a significant yet
overlooked research gap, but we can devise unsupervised CL methods solely
depending on the semantic signal provided by document length. We first derive
the theoretical foundations underlying length attacks, showing that elongating
a document would intensify the high intra-document similarity that is already
brought by CL. Moreover, we found that isotropy promised by CL is highly
dependent on the length range of text exposed in training. Inspired by these
findings, we introduce a simple yet universal document representation learning
framework, LA(SER)$^{3}$: length-agnostic self-reference for semantically
robust sentence representation learning, achieving state-of-the-art
unsupervised performance on the standard information retrieval benchmark.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Cross-Axis Transformer with 2D Rotary Embeddings
Abstract: Despite lagging behind their modal cousins in many respects, Vision
Transformers have provided an interesting opportunity to bridge the gap between
sequence modeling and image modeling. Up until now however, vision transformers
have largely been held back, due to both computational inefficiency, and lack
of proper handling of spatial dimensions. In this paper, we introduce the
Cross-Axis Transformer. CAT is a model inspired by both Axial Transformers, and
Microsoft's recent Retentive Network, that drastically reduces the required
number of floating point operations required to process an image, while
simultaneously converging faster and more accurately than the Vision
Transformers it replaces.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Enhancing LLM Intelligence with ARM-RAG: Auxiliary Rationale Memory for Retrieval Augmented Generation
Abstract: Large Language Models (LLMs) are smart but forgetful. Recent studies, (e.g.,
(Bubeck et al., 2023)) on modern LLMs have shown that they are capable of
performing amazing tasks typically necessitating human-level intelligence.
However, unlike humans, frozen LLMs do not improve over time; they neither
acquire new knowledge nor learn from their successes or failures. Some
approaches to improving the intelligence of LLMs include fine-tuning models
based on problem-solving performance (Zelikman et al., 2022), and building
bigger and more sophisticated models (Bubeck et al., 2023). However, these
methods have the drawback of requiring substantial data and computational
resources to retrain existing models. In this paper, we explore the use of
Retrieval Augmented Generation, also known as RAG (Lewis et al., 2021) to
improve problem-solving performance. We propose ARM-RAG (Auxiliary Rationale
Memory for Retrieval Augmented Generation), a system that learns from its
successes without incurring high training costs. We demonstrate that the
storage and subsequent retrieval of reasoning chains have a positive influence
on performance in grade-school math problems.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: A Systematic Review for Transformer-based Long-term Series Forecasting
Abstract: The emergence of deep learning has yielded noteworthy advancements in time
series forecasting (TSF). Transformer architectures, in particular, have
witnessed broad utilization and adoption in TSF tasks. Transformers have proven
to be the most successful solution to extract the semantic correlations among
the elements within a long sequence. Various variants have enabled transformer
architecture to effectively handle long-term time series forecasting (LTSF)
tasks. In this article, we first present a comprehensive overview of
transformer architectures and their subsequent enhancements developed to
address various LTSF tasks. Then, we summarize the publicly available LTSF
datasets and relevant evaluation metrics. Furthermore, we provide valuable
insights into the best practices and techniques for effectively training
transformers in the context of time-series analysis. Lastly, we propose
potential research directions in this rapidly evolving field.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Triplet Edge Attention for Algorithmic Reasoning
Abstract: This work investigates neural algorithmic reasoning to develop neural
networks capable of learning from classical algorithms. The main challenge is
to develop graph neural networks that are expressive enough to predict the
given algorithm outputs while generalizing well to out-of-distribution data. In
this work, we introduce a new graph neural network layer called Triplet Edge
Attention (TEA), an edge-aware graph attention layer. Our algorithm works by
precisely computing edge latent, aggregating multiple triplet messages using
edge-based attention. We empirically validate our TEA layer in the CLRS
benchmark and demonstrate a $5%$ improvement on average. In particular, we
achieve a $30%$ improvement for the string algorithms compared to the
state-of-the-art model.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Graph Agent: Explicit Reasoning Agent for Graphs
Abstract: Graph embedding methods such as Graph Neural Networks (GNNs) and Graph
Transformers have contributed to the development of graph reasoning algorithms
for various tasks on knowledge graphs. However, the lack of interpretability
and explainability of graph embedding methods has limited their applicability
in scenarios requiring explicit reasoning. In this paper, we introduce the
Graph Agent (GA), an intelligent agent methodology of leveraging large language
models (LLMs), inductive-deductive reasoning modules, and long-term memory for
knowledge graph reasoning tasks. GA integrates aspects of symbolic reasoning
and existing graph embedding methods to provide an innovative approach for
complex graph reasoning tasks. By converting graph structures into textual
data, GA enables LLMs to process, reason, and provide predictions alongside
human-interpretable explanations. The effectiveness of the GA was evaluated on
node classification and link prediction tasks. Results showed that GA reached
state-of-the-art performance, demonstrating accuracy of 90.65%, 95.48%, and
89.32% on Cora, PubMed, and PrimeKG datasets, respectively. Compared to
existing GNN and transformer models, GA offered advantages of explicit
reasoning ability, free-of-training, easy adaption to various graph reasoning
tasks

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Stable Modular Control via Contraction Theory for Reinforcement Learning
Abstract: We propose a novel way to integrate control techniques with reinforcement
learning (RL) for stability, robustness, and generalization: leveraging
contraction theory to realize modularity in neural control, which ensures that
combining stable subsystems can automatically preserve the stability. We
realize such modularity via signal composition and dynamic decomposition.
Signal composition creates the latent space, within which RL applies to
maximizing rewards. Dynamic decomposition is realized by coordinate
transformation that creates an auxiliary space, within which the latent signals
are coupled in the way that their combination can preserve stability provided
each signal, that is, each subsystem, has stable self-feedbacks. Leveraging
modularity, the nonlinear stability problem is deconstructed into algebraically
solvable ones, the stability of the subsystems in the auxiliary space, yielding
linear constraints on the input gradients of control networks that can be as
simple as switching the signs of network weights. This minimally invasive
method for stability allows arguably easy integration into the modular neural
architectures in machine learning, like hierarchical RL, and improves their
performance. We demonstrate in simulation the necessity and the effectiveness
of our method: the necessity for robustness and generalization, and the
effectiveness in improving hierarchical RL for manipulation learning.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union
Abstract: Semantic segmentation datasets often exhibit two types of imbalance:
\textit{class imbalance}, where some classes appear more frequently than others
and \textit{size imbalance}, where some objects occupy more pixels than others.
This causes traditional evaluation metrics to be biased towards
\textit{majority classes} (e.g. overall pixel-wise accuracy) and \textit{large
objects} (e.g. mean pixel-wise accuracy and per-dataset mean intersection over
union). To address these shortcomings, we propose the use of fine-grained mIoUs
along with corresponding worst-case metrics, thereby offering a more holistic
evaluation of segmentation techniques. These fine-grained metrics offer less
bias towards large objects, richer statistical information, and valuable
insights into model and dataset auditing. Furthermore, we undertake an
extensive benchmark study, where we train and evaluate 15 modern neural
networks with the proposed metrics on 12 diverse natural and aerial
segmentation datasets. Our benchmark study highlights the necessity of not
basing evaluations on a single metric and confirms that fine-grained mIoUs
reduce the bias towards large objects. Moreover, we identify the crucial role
played by architecture designs and loss functions, which lead to best practices
in optimizing fine-grained metrics. The code is available at
\href{https://github.com/zifuwanggg/JDTLosses}{https://github.com/zifuwanggg/JDTLosses}.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification
Abstract: The conventional few-shot classification aims at learning a model on a large
labeled base dataset and rapidly adapting to a target dataset that is from the
same distribution as the base dataset. However, in practice, the base and the
target datasets of few-shot classification are usually from different domains,
which is the problem of cross-domain few-shot classification. We tackle this
problem by making a small proportion of unlabeled images in the target domain
accessible in the training stage. In this setup, even though the base data are
sufficient and labeled, the large domain shift still makes transferring the
knowledge from the base dataset difficult. We meticulously design a cross-level
knowledge distillation method, which can strengthen the ability of the model to
extract more discriminative features in the target dataset by guiding the
network's shallow layers to learn higher-level information. Furthermore, in
order to alleviate the overfitting in the evaluation stage, we propose a
feature denoising operation which can reduce the feature redundancy and
mitigate overfitting. Our approach can surpass the previous state-of-the-art
method, Dynamic-Distillation, by 5.44% on 1-shot and 1.37% on 5-shot
classification tasks on average in the BSCD-FSL benchmark. The implementation
code will be available at https://github.com/jarucezh/cldfd.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System
Abstract: Training AI models has always been challenging, especially when there is a
need for custom models to provide personalized services. Algorithm engineers
often face a lengthy process to iteratively develop models tailored to specific
business requirements, making it even more difficult for non-experts. The quest
for high-quality and efficient model development, along with the emergence of
Large Language Model (LLM) Agents, has become a key focus in the industry.
Leveraging the powerful analytical, planning, and decision-making capabilities
of LLM, we propose a TrainerAgent system comprising a multi-agent framework
including Task, Data, Model and Server agents. These agents analyze
user-defined tasks, input data, and requirements (e.g., accuracy, speed),
optimizing them comprehensively from both data and model perspectives to obtain
satisfactory models, and finally deploy these models as online service.
Experimental evaluations on classical discriminative and generative tasks in
computer vision and natural language processing domains demonstrate that our
system consistently produces models that meet the desired criteria.
Furthermore, the system exhibits the ability to critically identify and reject
unattainable tasks, such as fantastical scenarios or unethical requests,
ensuring robustness and safety. This research presents a significant
advancement in achieving desired models with increased efficiency and quality
as compared to traditional model development, facilitated by the integration of
LLM-powered analysis, decision-making, and execution capabilities, as well as
the collaboration among four agents. We anticipate that our work will
contribute to the advancement of research on TrainerAgent in both academic and
industry communities, potentially establishing it as a new paradigm for model
development in the field of AI.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models
Abstract: We present Learning to Explain (LTX), a model-agnostic framework designed for
providing post-hoc explanations for vision models. The LTX framework introduces
an "explainer" model that generates explanation maps, highlighting the crucial
regions that justify the predictions made by the model being explained. To
train the explainer, we employ a two-stage process consisting of initial
pretraining followed by per-instance finetuning. During both stages of
training, we utilize a unique configuration where we compare the explained
model's prediction for a masked input with its original prediction for the
unmasked input. This approach enables the use of a novel counterfactual
objective, which aims to anticipate the model's output using masked versions of
the input image. Importantly, the LTX framework is not restricted to a specific
model architecture and can provide explanations for both Transformer-based and
convolutional models. Through our evaluations, we demonstrate that LTX
significantly outperforms the current state-of-the-art in explainability across
various metrics.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Dyport: Dynamic Importance-based Hypothesis Generation Benchmarking Technique
Abstract: This paper presents a novel benchmarking framework Dyport for evaluating
biomedical hypothesis generation systems. Utilizing curated datasets, our
approach tests these systems under realistic conditions, enhancing the
relevance of our evaluations. We integrate knowledge from the curated databases
into a dynamic graph, accompanied by a method to quantify discovery importance.
This not only assesses hypothesis accuracy but also their potential impact in
biomedical research which significantly extends traditional link prediction
benchmarks. Applicability of our benchmarking process is demonstrated on
several link prediction systems applied on biomedical semantic knowledge
graphs. Being flexible, our benchmarking system is designed for broad
application in hypothesis generation quality verification, aiming to expand the
scope of scientific discovery within the biomedical research community.
Availability and implementation: Dyport framework is fully open-source. All
code and datasets are available at: https://github.com/IlyaTyagin/Dyport

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: MVSA-Net: Multi-View State-Action Recognition for Robust and Deployable Trajectory Generation
Abstract: The learn-from-observation (LfO) paradigm is a human-inspired mode for a
robot to learn to perform a task simply by watching it being performed. LfO can
facilitate robot integration on factory floors by minimizing disruption and
reducing tedious programming. A key component of the LfO pipeline is a
transformation of the depth camera frames to the corresponding task state and
action pairs, which are then relayed to learning techniques such as imitation
or inverse reinforcement learning for understanding the task parameters. While
several existing computer vision models analyze videos for activity
recognition, SA-Net specifically targets robotic LfO from RGB-D data. However,
SA-Net and many other models analyze frame data captured from a single
viewpoint. Their analysis is therefore highly sensitive to occlusions of the
observed task, which are frequent in deployments. An obvious way of reducing
occlusions is to simultaneously observe the task from multiple viewpoints and
synchronously fuse the multiple streams in the model. Toward this, we present
multi-view SA-Net, which generalizes the SA-Net model to allow the perception
of multiple viewpoints of the task activity, integrate them, and better
recognize the state and action in each frame. Performance evaluations on two
distinct domains establish that MVSA-Net recognizes the state-action pairs
under occlusion more accurately compared to single-view MVSA-Net and other
baselines. Our ablation studies further evaluate its performance under
different ambient conditions and establish the contribution of the architecture
components. As such, MVSA-Net offers a significantly more robust and deployable
state-action trajectory generation compared to previous methods.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Modeling Boundedly Rational Agents with Latent Inference Budgets
Abstract: We study the problem of modeling a population of agents pursuing unknown
goals subject to unknown computational constraints. In standard models of
bounded rationality, sub-optimal decision-making is simulated by adding
homoscedastic noise to optimal decisions rather than explicitly simulating
constrained inference. In this work, we introduce a latent inference budget
model (L-IBM) that models agents' computational constraints explicitly, via a
latent variable (inferred jointly with a model of agents' goals) that controls
the runtime of an iterative inference algorithm. L-IBMs make it possible to
learn agent models using data from diverse populations of suboptimal actors. In
three modeling tasks -- inferring navigation goals from routes, inferring
communicative intents from human utterances, and predicting next moves in human
chess games -- we show that L-IBMs match or outperform Boltzmann models of
decision-making under uncertainty. Inferred inference budgets are themselves
meaningful, efficient to compute, and correlated with measures of player skill,
partner skill and task difficulty.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: SurvTimeSurvival: Survival Analysis On The Patient With Multiple Visits/Records
Abstract: The accurate prediction of survival times for patients with severe diseases
remains a critical challenge despite recent advances in artificial
intelligence. This study introduces "SurvTimeSurvival: Survival Analysis On
Patients With Multiple Visits/Records", utilizing the Transformer model to not
only handle the complexities of time-varying covariates but also covariates
data. We also tackle the data sparsity issue common to survival analysis
datasets by integrating synthetic data generation into the learning process of
our model. We show that our method outperforms state-of-the-art deep learning
approaches on both covariates and time-varying covariates datasets. Our
approach aims not only to enhance the understanding of individual patient
survival trajectories across various medical conditions, thereby improving
prediction accuracy, but also to play a pivotal role in designing clinical
trials and creating new treatments.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Unpacking the Ethical Value Alignment in Big Models
Abstract: Big models have greatly advanced AI's ability to understand, generate, and
manipulate information and content, enabling numerous applications. However, as
these models become increasingly integrated into everyday life, their inherent
ethical values and potential biases pose unforeseen risks to society. This
paper provides an overview of the risks and challenges associated with big
models, surveys existing AI ethics guidelines, and examines the ethical
implications arising from the limitations of these models. Taking a normative
ethics perspective, we propose a reassessment of recent normative guidelines,
highlighting the importance of collaborative efforts in academia to establish a
unified and universal AI ethics framework. Furthermore, we investigate the
moral inclinations of current mainstream LLMs using the Moral Foundation
theory, analyze existing alignment algorithms, and outline the unique
challenges encountered in aligning ethical values within them. To address these
challenges, we introduce a novel conceptual paradigm for aligning the ethical
values of big models and discuss promising research directions for alignment
criteria, evaluation, and method, representing an initial step towards the
interdisciplinary construction of the ethically aligned AI
  This paper is a modified English version of our Chinese paper
https://crad.ict.ac.cn/cn/article/doi/10.7544/issn1000-1239.202330553, intended
to help non-Chinese native speakers better understand our work.

Output: Computers and Society


Instruction: What field is the article from?

Input:
Title: Non-Euclidean Spatial Graph Neural Network
Abstract: Spatial networks are networks whose graph topology is constrained by their
embedded spatial space. Understanding the coupled spatial-graph properties is
crucial for extracting powerful representations from spatial networks.
Therefore, merely combining individual spatial and network representations
cannot reveal the underlying interaction mechanism of spatial networks.
Besides, existing spatial network representation learning methods can only
consider networks embedded in Euclidean space, and can not well exploit the
rich geometric information carried by irregular and non-uniform non-Euclidean
space. In order to address this issue, in this paper we propose a novel generic
framework to learn the representation of spatial networks that are embedded in
non-Euclidean manifold space. Specifically, a novel message-passing-based
neural network is proposed to combine graph topology and spatial geometry,
where spatial geometry is extracted as messages on the edges. We theoretically
guarantee that the learned representations are provably invariant to important
symmetries such as rotation or translation, and simultaneously maintain
sufficient ability in distinguishing different geometric structures. The
strength of our proposed method is demonstrated through extensive experiments
on both synthetic and real-world datasets.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Content-Localization based System for Analyzing Sentiment and Hate Behaviors in Low-Resource Dialectal Arabic: English to Levantine and Gulf
Abstract: Even though online social movements can quickly become viral on social media,
languages can be a barrier to timely monitoring and analyzing the underlying
online social behaviors (OSB). This is especially true for under-resourced
languages on social media like dialectal Arabic; the primary language used by
Arabs on social media. Therefore, it is crucial to provide solutions to
efficiently exploit resources from high-resourced languages to solve
language-dependent OSB analysis in under-resourced languages. This paper
proposes to localize content of resources in high-resourced languages into
under-resourced Arabic dialects. Content localization goes beyond content
translation that converts text from one language to another; content
localization adapts culture, language nuances and regional preferences from one
language to a specific language/dialect. Automating understanding of the
natural and familiar day-to-day expressions in different regions, is the key to
achieve a wider analysis of OSB especially for smart cities. In this paper, we
utilize content-localization based neural machine translation to develop
sentiment and hate classifiers for two low-resourced Arabic dialects: Levantine
and Gulf. Not only this but we also leverage unsupervised learning to
facilitate the analysis of sentiment and hate predictions by inferring hidden
topics from the corresponding data and providing coherent interpretations of
those topics in their native language/dialects. The experimental evaluations
and proof-of-concept COVID-19 case study on real data have validated the
effectiveness of our proposed system in precisely distinguishing sentiments and
accurately identifying hate content in both Levantine and Gulf Arabic dialects.
Our findings shed light on the importance of considering the unique nature of
dialects within the same language and ignoring the dialectal aspect would lead
to misleading analysis.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Critical Role of Artificially Intelligent Conversational Chatbot
Abstract: Artificially intelligent chatbot, such as ChatGPT, represents a recent and
powerful advancement in the AI domain. Users prefer them for obtaining quick
and precise answers, avoiding the usual hassle of clicking through multiple
links in traditional searches. ChatGPT's conversational approach makes it
comfortable and accessible for finding answers quickly and in an organized
manner. However, it is important to note that these chatbots have limitations,
especially in terms of providing accurate answers as well as ethical concerns.
In this study, we explore various scenarios involving ChatGPT's ethical
implications within academic contexts, its limitations, and the potential
misuse by specific user groups. To address these challenges, we propose
architectural solutions aimed at preventing inappropriate use and promoting
responsible AI interactions.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Graph Prompt Learning: A Comprehensive Survey and Beyond
Abstract: Artificial General Intelligence (AGI) has revolutionized numerous fields, yet
its integration with graph data, a cornerstone in our interconnected world,
remains nascent. This paper presents a pioneering survey on the emerging domain
of graph prompts in AGI, addressing key challenges and opportunities in
harnessing graph data for AGI applications. Despite substantial advancements in
AGI across natural language processing and computer vision, the application to
graph data is relatively underexplored. This survey critically evaluates the
current landscape of AGI in handling graph data, highlighting the distinct
challenges in cross-modality, cross-domain, and cross-task applications
specific to graphs. Our work is the first to propose a unified framework for
understanding graph prompt learning, offering clarity on prompt tokens, token
structures, and insertion patterns in the graph domain. We delve into the
intrinsic properties of graph prompts, exploring their flexibility,
expressiveness, and interplay with existing graph models. A comprehensive
taxonomy categorizes over 100 works in this field, aligning them with
pre-training tasks across node-level, edge-level, and graph-level objectives.
Additionally, we present, ProG, a Python library, and an accompanying website,
to support and advance research in graph prompting. The survey culminates in a
discussion of current challenges and future directions, offering a roadmap for
research in graph prompting within AGI. Through this comprehensive analysis, we
aim to catalyze further exploration and practical applications of AGI in graph
data, underlining its potential to reshape AGI fields and beyond. ProG and the
website can be accessed by
\url{https://github.com/WxxShirley/Awesome-Graph-Prompt}, and
\url{https://github.com/sheldonresearch/ProG}, respectively.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: HuRef: HUman-REadable Fingerprint for Large Language Models
Abstract: Protecting the copyright of large language models (LLMs) has become crucial
due to their resource-intensive training and accompanying carefully designed
licenses. However, identifying the original base model of an LLM is challenging
due to potential parameter alterations through fine-tuning or continued
pretraining. In this study, we introduce HuRef, a human-readable fingerprint
for LLMs that uniquely identifies the base model without exposing model
parameters or interfering with training. We first observe that the vector
direction of LLM parameters remains stable after the model has converged during
pretraining, showing negligible perturbations through subsequent training
steps, including continued pretraining, supervised fine-tuning (SFT), and RLHF,
which makes it a sufficient condition to identify the base model. The necessity
is validated by continuing to train an LLM with an extra term to drive away the
model parameters' direction and the model becomes damaged. However, this
direction is vulnerable to simple attacks like dimension permutation or matrix
rotation, which significantly change it without affecting performance. To
address this, leveraging the Transformer structure, we systematically analyze
potential attacks and define three invariant terms that identify an LLM's base
model. We make these invariant terms human-readable by mapping them to a
Gaussian vector using a convolutional encoder and then converting it into a
natural image with StyleGAN2. Our method generates a dog image as an identity
fingerprint for an LLM, where the dog's appearance strongly indicates the LLM's
base model. Experimental results across various LLMs demonstrate the
effectiveness of our method, the generated dog image remains invariant to
different training steps, including SFT, RLHF, or even continued pretraining
with augmented vocabulary in a new language.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: STEER: Unified Style Transfer with Expert Reinforcement
Abstract: While text style transfer has many applications across natural language
processing, the core premise of transferring from a single source style is
unrealistic in a real-world setting. In this work, we focus on arbitrary style
transfer: rewriting a text from an arbitrary, unknown style to a target style.
  We propose STEER: Unified Style Transfer with Expert Reinforcement, a unified
frame-work developed to overcome the challenge of limited parallel data for
style transfer. STEER involves automatically generating a corpus of
style-transfer pairs using a product of experts during decoding. The generated
offline data is then used to pre-train an initial policy before switching to
online, off-policy reinforcement learning for further improvements via
fine-grained reward signals. STEER is unified and can transfer to multiple
target styles from an arbitrary, unknown source style, making it particularly
flexible and efficient.
  Experimental results on a challenging dataset with text from a diverse set of
styles demonstrate state-of-the-art results compared to competitive baselines.
Remarkably, STEER outperforms the 175B parameter instruction-tuned GPT-3 on
overall style transfer quality, despite being 226 times smaller in size. We
also show STEER is robust, maintaining its style transfer capabilities on
out-of-domain data, and surpassing nearly all baselines across various styles.
The success of our method highlights the potential of RL algorithms when
augmented with controllable decoding to overcome the challenge of limited data
supervision.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition
Abstract: Large-kernel convolutional neural networks (ConvNets) have recently received
extensive research attention, but there are two unresolved and critical issues
that demand further investigation. 1) The architectures of existing
large-kernel ConvNets largely follow the design principles of conventional
ConvNets or transformers, while the architectural design for large-kernel
ConvNets remains under-addressed. 2) As transformers have dominated multiple
modalities, it remains to be investigated whether ConvNets also have a strong
universal perception ability in domains beyond vision. In this paper, we
contribute from two aspects. 1) We propose four architectural guidelines for
designing large-kernel ConvNets, the core of which is to exploit the essential
characteristics of large kernels that distinguish them from small kernels -
they can see wide without going deep. Following such guidelines, our proposed
large-kernel ConvNet shows leading performance in image recognition. For
example, our models achieve an ImageNet accuracy of 88.0%, ADE20K mIoU of
55.6%, and COCO box AP of 56.4%, demonstrating better performance and higher
speed than a number of recently proposed powerful competitors. 2) We discover
that large kernels are the key to unlocking the exceptional performance of
ConvNets in domains where they were originally not proficient. With certain
modality-related preprocessing approaches, the proposed model achieves
state-of-the-art performance on time-series forecasting and audio recognition
tasks even without modality-specific customization to the architecture. Code
and all the models at https://github.com/AILab-CVC/UniRepLKNet.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: An Attentive Inductive Bias for Sequential Recommendation Beyond the Self-Attention
Abstract: Sequential recommendation (SR) models based on Transformers have achieved
remarkable successes. The self-attention mechanism of Transformers for computer
vision and natural language processing suffers from the oversmoothing problem,
i.e., hidden representations becoming similar to tokens. In the SR domain, we,
for the first time, show that the same problem occurs. We present pioneering
investigations that reveal the low-pass filtering nature of self-attention in
the SR, which causes oversmoothing. To this end, we propose a novel method
called Beyond Self-Attention for Sequential Recommendation (BSARec), which
leverages the Fourier transform to i) inject an inductive bias by considering
fine-grained sequential patterns and ii) integrate low and high-frequency
information to mitigate oversmoothing. Our discovery shows significant
advancements in the SR domain and is expected to bridge the gap for existing
Transformer-based SR models. We test our proposed approach through extensive
experiments on 6 benchmark datasets. The experimental results demonstrate that
our model outperforms 7 baseline methods in terms of recommendation
performance.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: DialogBench: Evaluating LLMs as Human-like Dialogue Systems
Abstract: Large language models (LLMs) have achieved remarkable breakthroughs in new
dialogue capabilities, refreshing human's impressions on dialogue systems. The
long-standing goal of dialogue systems is to be human-like enough to establish
long-term connections with users by satisfying the need for communication,
affection and social belonging. Therefore, there has been an urgent need to
evaluate LLMs as human-like dialogue systems. In this paper, we propose
DialogBench, a dialogue evaluation benchmark that currently contains $12$
dialogue tasks to assess the capabilities of LLMs as human-like dialogue
systems should have. Specifically, we prompt GPT-4 to generate evaluation
instances for each task. We first design the basic prompt based on widely-used
design principles and further mitigate the existing biases to generate
higher-quality evaluation instances. Our extensive test over $28$ LLMs
(including pre-trained and supervised instruction-tuning) shows that
instruction fine-tuning benefits improve the human likeness of LLMs to a
certain extent, but there is still much room to improve those capabilities for
most LLMs as human-like dialogue systems. In addition, experimental results
also indicate that LLMs perform differently in various abilities that
human-like dialogue systems should have. We will publicly release DialogBench,
along with the associated evaluation code for the broader research community.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval
Abstract: With the development of deep learning and natural language processing
techniques, pre-trained language models have been widely used to solve
information retrieval (IR) problems. Benefiting from the pre-training and
fine-tuning paradigm, these models achieve state-of-the-art performance. In
previous works, plain texts in Wikipedia have been widely used in the
pre-training stage. However, the rich structured information in Wikipedia, such
as the titles, abstracts, hierarchical heading (multi-level title) structure,
relationship between articles, references, hyperlink structures, and the
writing organizations, has not been fully explored. In this paper, we devise
four pre-training objectives tailored for IR tasks based on the structured
knowledge of Wikipedia. Compared to existing pre-training methods, our approach
can better capture the semantic knowledge in the training corpus by leveraging
the human-edited structured data from Wikipedia. Experimental results on
multiple IR benchmark datasets show the superior performance of our model in
both zero-shot and fine-tuning settings compared to existing strong retrieval
baselines. Besides, experimental results in biomedical and legal domains
demonstrate that our approach achieves better performance in vertical domains
compared to previous models, especially in scenarios where long text similarity
matching is needed.

Output: Information Retrieval


Instruction: What field is the article from?

Input:
Title: The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learning
Abstract: With the advance of the powerful heterogeneous, parallel and distributed
computing systems and ever increasing immense amount of data, machine learning
has become an indispensable part of cutting-edge technology, scientific
research and consumer products. In this study, we present a review of modern
machine and deep learning. We provide a high-level overview for the latest
advanced machine learning algorithms, applications, and frameworks. Our
discussion encompasses parallel distributed learning, deep learning as well as
federated learning. As a result, our work serves as an introductory text to the
vast field of modern machine learning.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendations
Abstract: Generative artificial intelligence (GenAI) offers promising potential for
advancing human-AI collaboration in qualitative research. However, existing
works focused on conventional machine-learning and pattern-based AI systems,
and little is known about how researchers interact with GenAI in qualitative
research. This work delves into researchers' perceptions of their collaboration
with GenAI, specifically ChatGPT. Through a user study involving ten
qualitative researchers, we found ChatGPT to be a valuable collaborator for
thematic analysis, enhancing coding efficiency, aiding initial data
exploration, offering granular quantitative insights, and assisting
comprehension for non-native speakers and non-experts. Yet, concerns about its
trustworthiness and accuracy, reliability and consistency, limited contextual
understanding, and broader acceptance within the research community persist. We
contribute five actionable design recommendations to foster effective human-AI
collaboration. These include incorporating transparent explanatory mechanisms,
enhancing interface and integration capabilities, prioritising contextual
understanding and customisation, embedding human-AI feedback loops and
iterative functionality, and strengthening trust through validation mechanisms.

Output: Human-Computer Interaction


Instruction: What field is the article from?

Input:
Title: Variational Annealing on Graphs for Combinatorial Optimization
Abstract: Several recent unsupervised learning methods use probabilistic approaches to
solve combinatorial optimization (CO) problems based on the assumption of
statistically independent solution variables. We demonstrate that this
assumption imposes performance limitations in particular on difficult problem
instances. Our results corroborate that an autoregressive approach which
captures statistical dependencies among solution variables yields superior
performance on many popular CO problems. We introduce subgraph tokenization in
which the configuration of a set of solution variables is represented by a
single token. This tokenization technique alleviates the drawback of the long
sequential sampling procedure which is inherent to autoregressive methods
without sacrificing expressivity. Importantly, we theoretically motivate an
annealed entropy regularization and show empirically that it is essential for
efficient and stable learning.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Exponentially Faster Language Modelling
Abstract: Language models only really need to use an exponential fraction of their
neurons for individual inferences. As proof, we present UltraFastBERT, a BERT
variant that uses 0.3% of its neurons during inference while performing on par
with similar BERT models. UltraFastBERT selectively engages just 12 out of 4095
neurons for each layer inference. This is achieved by replacing feedforward
networks with fast feedforward networks (FFFs). While no truly efficient
implementation currently exists to unlock the full acceleration potential of
conditional neural execution, we provide high-level CPU code achieving 78x
speedup over the optimized baseline feedforward implementation, and a PyTorch
implementation delivering 40x speedup over the equivalent batched feedforward
inference. We publish our training code, benchmarking setup, and model weights.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Continual Learning: Applications and the Road Forward
Abstract: Continual learning is a sub-field of machine learning, which aims to allow
machine learning models to continuously learn on new data, by accumulating
knowledge without forgetting what was learned in the past. In this work, we
take a step back, and ask: "Why should one care about continual learning in the
first place?". We set the stage by surveying recent continual learning papers
published at three major machine learning conferences, and show that
memory-constrained settings dominate the field. Then, we discuss five open
problems in machine learning, and even though they seem unrelated to continual
learning at first sight, we show that continual learning will inevitably be
part of their solution. These problems are model-editing, personalization,
on-device learning, faster (re-)training and reinforcement learning. Finally,
by comparing the desiderata from these unsolved problems and the current
assumptions in continual learning, we highlight and discuss four future
directions for continual learning research. We hope that this work offers an
interesting perspective on the future of continual learning, while displaying
its potential value and the paths we have to pursue in order to make it
successful. This work is the result of the many discussions the authors had at
the Dagstuhl seminar on Deep Continual Learning, in March 2023.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: A Principled Framework for Knowledge-enhanced Large Language Model
Abstract: Large Language Models (LLMs) are versatile, yet they often falter in tasks
requiring deep and reliable reasoning due to issues like hallucinations,
limiting their applicability in critical scenarios. This paper introduces a
rigorously designed framework for creating LLMs that effectively anchor
knowledge and employ a closed-loop reasoning process, enhancing their
capability for in-depth analysis. We dissect the framework to illustrate the
contribution of each component to the LLMs' performance, offering a theoretical
assurance of improved reasoning under well-defined assumptions.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Exploitation-Guided Exploration for Semantic Embodied Navigation
Abstract: In the recent progress in embodied navigation and sim-to-robot transfer,
modular policies have emerged as a de facto framework. However, there is more
to compositionality beyond the decomposition of the learning load into modular
components. In this work, we investigate a principled way to syntactically
combine these components. Particularly, we propose Exploitation-Guided
Exploration (XGX) where separate modules for exploration and exploitation come
together in a novel and intuitive manner. We configure the exploitation module
to take over in the deterministic final steps of navigation i.e. when the goal
becomes visible. Crucially, an exploitation module teacher-forces the
exploration module and continues driving an overridden policy optimization.
XGX, with effective decomposition and novel guidance, improves the
state-of-the-art performance on the challenging object navigation task from 70%
to 73%. Along with better accuracy, through targeted analysis, we show that XGX
is also more efficient at goal-conditioned exploration. Finally, we show
sim-to-real transfer to robot hardware and XGX performs over two-fold better
than the best baseline from simulation benchmarking. Project page:
xgxvisnav.github.io

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities
Abstract: Human interactions are deeply rooted in the interplay of thoughts, beliefs,
and desires made possible by Theory of Mind (ToM): our cognitive ability to
understand the mental states of ourselves and others. Although ToM may come
naturally to us, emulating it presents a challenge to even the most advanced
Large Language Models (LLMs). Recent improvements to LLMs' reasoning
capabilities from simple yet effective prompting techniques such as
Chain-of-Thought have seen limited applicability to ToM. In this paper, we turn
to the prominent cognitive science theory "Simulation Theory" to bridge this
gap. We introduce SimToM, a novel two-stage prompting framework inspired by
Simulation Theory's notion of perspective-taking. To implement this idea on
current ToM benchmarks, SimToM first filters context based on what the
character in question knows before answering a question about their mental
state. Our approach, which requires no additional training and minimal
prompt-tuning, shows substantial improvement over existing methods, and our
analysis reveals the importance of perspective-taking to Theory-of-Mind
capabilities. Our findings suggest perspective-taking as a promising direction
for future research into improving LLMs' ToM capabilities.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: HDMNet: A Hierarchical Matching Network with Double Attention for Large-scale Outdoor LiDAR Point Cloud Registration
Abstract: Outdoor LiDAR point clouds are typically large-scale and complexly
distributed. To achieve efficient and accurate registration, emphasizing the
similarity among local regions and prioritizing global local-to-local matching
is of utmost importance, subsequent to which accuracy can be enhanced through
cost-effective fine registration. In this paper, a novel hierarchical neural
network with double attention named HDMNet is proposed for large-scale outdoor
LiDAR point cloud registration. Specifically, A novel feature consistency
enhanced double-soft matching network is introduced to achieve two-stage
matching with high flexibility while enlarging the receptive field with high
efficiency in a patch-to patch manner, which significantly improves the
registration performance. Moreover, in order to further utilize the sparse
matching information from deeper layer, we develop a novel trainable embedding
mask to incorporate the confidence scores of correspondences obtained from pose
estimation of deeper layer, eliminating additional computations. The
high-confidence keypoints in the sparser point cloud of the deeper layer
correspond to a high-confidence spatial neighborhood region in shallower layer,
which will receive more attention, while the features of non-key regions will
be masked. Extensive experiments are conducted on two large-scale outdoor LiDAR
point cloud datasets to demonstrate the high accuracy and efficiency of the
proposed HDMNet.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Distribution Re-weighting and Voting Paradoxes
Abstract: We explore a specific type of distribution shift called domain expertise, in
which training is limited to a subset of all possible labels. This setting is
common among specialized human experts, or specific focused studies. We show
how the standard approach to distribution shift, which involves re-weighting
data, can result in paradoxical disagreements among differing domain expertise.
We also demonstrate how standard adjustments for causal inference lead to the
same paradox. We prove that the characteristics of these paradoxes exactly
mimic another set of paradoxes which arise among sets of voter preferences.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based Inference
Abstract: We present multimodal neural posterior estimation (MultiNPE), a method to
integrate heterogeneous data from different sources in simulation-based
inference with neural networks. Inspired by advances in attention-based deep
fusion learning, it empowers researchers to analyze data from different domains
and infer the parameters of complex mathematical models with increased
accuracy. We formulate different multimodal fusion approaches for MultiNPE
(early, late, and hybrid) and evaluate their performance in three challenging
numerical experiments. MultiNPE not only outperforms na\"ive baselines on a
benchmark model, but also achieves superior inference on representative
scientific models from neuroscience and cardiology. In addition, we
systematically investigate the impact of partially missing data on the
different fusion strategies. Across our different experiments, late and hybrid
fusion techniques emerge as the methods of choice for practical applications of
multimodal simulation-based inference.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
Abstract: In this study, we present an investigation into the anisotropy dynamics and
intrinsic dimension of embeddings in transformer architectures, focusing on the
dichotomy between encoders and decoders. Our findings reveal that the
anisotropy profile in transformer decoders exhibits a distinct bell-shaped
curve, with the highest anisotropy concentrations in the middle layers. This
pattern diverges from the more uniformly distributed anisotropy observed in
encoders. In addition, we found that the intrinsic dimension of embeddings
increases in the initial phases of training, indicating an expansion into
higher-dimensional space. Which is then followed by a compression phase towards
the end of training with dimensionality decrease, suggesting a refinement into
more compact representations. Our results provide fresh insights to the
understanding of encoders and decoders embedding properties.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Beyond Top-Class Agreement: Using Divergences to Forecast Performance under Distribution Shift
Abstract: Knowing if a model will generalize to data 'in the wild' is crucial for safe
deployment. To this end, we study model disagreement notions that consider the
full predictive distribution - specifically disagreement based on Hellinger
distance, Jensen-Shannon and Kullback-Leibler divergence. We find that
divergence-based scores provide better test error estimates and detection rates
on out-of-distribution data compared to their top-1 counterparts. Experiments
involve standard vision and foundation models.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Signed Binarization: Unlocking Efficiency Through Repetition-Sparsity Trade-Off
Abstract: Efficient inference of Deep Neural Networks (DNNs) on resource-constrained
edge devices is essential. Quantization and sparsity are key algorithmic
techniques that translate to repetition and sparsity within tensors at the
hardware-software interface. This paper introduces the concept of
repetition-sparsity trade-off that helps explain computational efficiency
during inference. We propose Signed Binarization, a unified co-design framework
that synergistically integrates hardware-software systems, quantization
functions, and representation learning techniques to address this trade-off.
Our results demonstrate that Signed Binarization is more accurate than
binarization with the same number of non-zero weights. Detailed analysis
indicates that signed binarization generates a smaller distribution of
effectual (non-zero) parameters nested within a larger distribution of total
parameters, both of the same type, for a DNN block. Finally, our approach
achieves a 26% speedup on real hardware, doubles energy efficiency, and reduces
density by 2.8x compared to binary methods for ResNet 18, presenting an
alternative solution for deploying efficient models in resource-limited
environments.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
Abstract: In recent years, modeling evolving knowledge over temporal knowledge graphs
(TKGs) has become a heated topic. Various methods have been proposed to
forecast links on TKGs. Most of them are embedding-based, where hidden
representations are learned to represent knowledge graph (KG) entities and
relations based on the observed graph contexts. Although these methods show
strong performance on traditional TKG forecasting (TKGF) benchmarks, they
naturally face a strong challenge when they are asked to model the unseen
zero-shot relations that has no prior graph context. In this paper, we try to
mitigate this problem as follows. We first input the text descriptions of KG
relations into large language models (LLMs) for generating relation
representations, and then introduce them into embedding-based TKGF methods.
LLM-empowered representations can capture the semantic information in the
relation descriptions. This makes the relations, whether seen or unseen, with
similar semantic meanings stay close in the embedding space, enabling TKGF
models to recognize zero-shot relations even without any observed graph
context. Experimental results show that our approach helps TKGF models to
achieve much better performance in forecasting the facts with previously unseen
relations, while still maintaining their ability in link forecasting regarding
seen relations.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
Abstract: Artificial intelligence, and specifically deep neural networks (DNNs), has
rapidly emerged in the past decade as the standard for several tasks from
specific advertising to object detection. The performance offered has led DNN
algorithms to become a part of critical embedded systems, requiring both
efficiency and reliability. In particular, DNNs are subject to malicious
examples designed in a way to fool the network while being undetectable to the
human observer: the adversarial examples. While previous studies propose
frameworks to implement such attacks in black box settings, those often rely on
the hypothesis that the attacker has access to the logits of the neural
network, breaking the assumption of the traditional black box. In this paper,
we investigate a real black box scenario where the attacker has no access to
the logits. In particular, we propose an architecture-agnostic attack which
solve this constraint by extracting the logits. Our method combines hardware
and software attacks, by performing a side-channel attack that exploits
electromagnetic leakages to extract the logits for a given input, allowing an
attacker to estimate the gradients and produce state-of-the-art adversarial
examples to fool the targeted neural network. Through this example of
adversarial attack, we demonstrate the effectiveness of logits extraction using
side-channel as a first step for more general attack frameworks requiring
either the logits or the confidence scores.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Pre-trained Recommender Systems: A Causal Debiasing Perspective
Abstract: Recent studies on pre-trained vision/language models have demonstrated the
practical benefit of a new, promising solution-building paradigm in AI where
models can be pre-trained on broad data describing a generic task space and
then adapted successfully to solve a wide range of downstream tasks, even when
training data is severely limited (e.g., in zero- or few-shot learning
scenarios). Inspired by such progress, we investigate in this paper the
possibilities and challenges of adapting such a paradigm to the context of
recommender systems, which is less investigated from the perspective of
pre-trained model. In particular, we propose to develop a generic recommender
that captures universal interaction patterns by training on generic user-item
interaction data extracted from different domains, which can then be fast
adapted to improve few-shot learning performance in unseen new domains (with
limited data).
  However, unlike vision/language data which share strong conformity in the
semantic space, universal patterns underlying recommendation data collected
across different domains (e.g., different countries or different E-commerce
platforms) are often occluded by both in-domain and cross-domain biases
implicitly imposed by the cultural differences in their user and item bases, as
well as their uses of different e-commerce platforms. As shown in our
experiments, such heterogeneous biases in the data tend to hinder the
effectiveness of the pre-trained model. To address this challenge, we further
introduce and formalize a causal debiasing perspective, which is substantiated
via a hierarchical Bayesian deep learning model, named PreRec. Our empirical
studies on real-world data show that the proposed model could significantly
improve the recommendation performance in zero- and few-shot learning settings
under both cross-market and cross-platform scenarios.

Output: Information Retrieval


Instruction: What field is the article from?

Input:
Title: Clash of the Explainers: Argumentation for Context-Appropriate Explanations
Abstract: Understanding when and why to apply any given eXplainable Artificial
Intelligence (XAI) technique is not a straightforward task. There is no single
approach that is best suited for a given context. This paper aims to address
the challenge of selecting the most appropriate explainer given the context in
which an explanation is required. For AI explainability to be effective,
explanations and how they are presented needs to be oriented towards the
stakeholder receiving the explanation. If -- in general -- no single
explanation technique surpasses the rest, then reasoning over the available
methods is required in order to select one that is context-appropriate. Due to
the transparency they afford, we propose employing argumentation techniques to
reach an agreement over the most suitable explainers from a given set of
possible explainers.
  In this paper, we propose a modular reasoning system consisting of a given
mental model of the relevant stakeholder, a reasoner component that solves the
argumentation problem generated by a multi-explainer component, and an AI model
that is to be explained suitably to the stakeholder of interest. By formalising
supporting premises -- and inferences -- we can map stakeholder characteristics
to those of explanation techniques. This allows us to reason over the
techniques and prioritise the best one for the given context, while also
offering transparency into the selection decision.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Federated Learning on Edge Sensing Devices: A Review
Abstract: The ability to monitor ambient characteristics, interact with them, and
derive information about the surroundings has been made possible by the rapid
proliferation of edge sensing devices like IoT, mobile, and wearable devices
and their measuring capabilities with integrated sensors. Even though these
devices are small and have less capacity for data storage and processing, they
produce vast amounts of data. Some example application areas where sensor data
is collected and processed include healthcare, environmental (including air
quality and pollution levels), automotive, industrial, aerospace, and
agricultural applications. These enormous volumes of sensing data collected
from the edge devices are analyzed using a variety of Machine Learning (ML) and
Deep Learning (DL) approaches. However, analyzing them on the cloud or a server
presents challenges related to privacy, hardware, and connectivity limitations.
Federated Learning (FL) is emerging as a solution to these problems while
preserving privacy by jointly training a model without sharing raw data. In
this paper, we review the FL strategies from the perspective of edge sensing
devices to get over the limitations of conventional machine learning
techniques. We focus on the key FL principles, software frameworks, and
testbeds. We also explore the current sensor technologies, properties of the
sensing devices and sensing applications where FL is utilized. We conclude with
a discussion on open issues and future research directions on FL for further
studies

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
Abstract: The field of Multi-Agent Reinforcement Learning (MARL) is currently facing a
reproducibility crisis. While solutions for standardized reporting have been
proposed to address the issue, we still lack a benchmarking tool that enables
standardization and reproducibility, while leveraging cutting-edge
Reinforcement Learning (RL) implementations. In this paper, we introduce
BenchMARL, the first MARL training library created to enable standardized
benchmarking across different algorithms, models, and environments. BenchMARL
uses TorchRL as its backend, granting it high performance and maintained
state-of-the-art implementations while addressing the broad community of MARL
PyTorch users. Its design enables systematic configuration and reporting, thus
allowing users to create and run complex benchmarks from simple one-line
inputs. BenchMARL is open-sourced on GitHub:
https://github.com/facebookresearch/BenchMARL

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Efficient Multimodal Diffusion Models Using Joint Data Infilling with Partially Shared U-Net
Abstract: Recently, diffusion models have been used successfully to fit distributions
for cross-modal data translation and multimodal data generation. However, these
methods rely on extensive scaling, overlooking the inefficiency and
interference between modalities. We develop Partially Shared U-Net (PS-U-Net)
architecture which is an efficient multimodal diffusion model that allows text
and image inputs to pass through dedicated layers and skip-connections for
preserving modality-specific fine-grained details. Inspired by image
inpainting, we also propose a new efficient multimodal sampling method that
introduces new scenarios for conditional generation while only requiring a
simple joint distribution to be learned. Our empirical exploration of the
MS-COCO dataset demonstrates that our method generates multimodal text and
image data with higher quality compared to existing multimodal diffusion models
while having a comparable size, faster training, faster multimodal sampling,
and more flexible generation.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents
Abstract: Large language models (LLMs) have attracted huge interest in practical
applications given their increasingly accurate responses and coherent reasoning
abilities. Given their nature as black-boxes using complex reasoning processes
on their inputs, it is inevitable that the demand for scalable and faithful
explanations for LLMs' generated content will continue to grow. There have been
major developments in the explainability of neural network models over the past
decade. Among them, post-hoc explainability methods, especially Shapley values,
have proven effective for interpreting deep learning models. However, there are
major challenges in scaling up Shapley values for LLMs, particularly when
dealing with long input contexts containing thousands of tokens and
autoregressively generated output sequences. Furthermore, it is often unclear
how to effectively utilize generated explanations to improve the performance of
LLMs. In this paper, we introduce TextGenSHAP, an efficient post-hoc
explanation method incorporating LM-specific techniques. We demonstrate that
this leads to significant increases in speed compared to conventional Shapley
value computations, reducing processing times from hours to minutes for
token-level explanations, and to just seconds for document-level explanations.
In addition, we demonstrate how real-time Shapley values can be utilized in two
important scenarios, providing better understanding of long-document question
answering by localizing important words and sentences; and improving existing
document retrieval systems through enhancing the accuracy of selected passages
and ultimately the final responses.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Multilingual Coarse Political Stance Classification of Media. The Editorial Line of a ChatGPT and Bard Newspaper
Abstract: Neutrality is difficult to achieve and, in politics, subjective. Traditional
media typically adopt an editorial line that can be used by their potential
readers as an indicator of the media bias. Several platforms currently rate
news outlets according to their political bias. The editorial line and the
ratings help readers in gathering a balanced view of news. But in the advent of
instruction-following language models, tasks such as writing a newspaper
article can be delegated to computers. Without imposing a biased persona, where
would an AI-based news outlet lie within the bias ratings? In this work, we use
the ratings of authentic news outlets to create a multilingual corpus of news
with coarse stance annotations (Left and Right) along with automatically
extracted topic annotations. We show that classifiers trained on this data are
able to identify the editorial line of most unseen newspapers in English,
German, Spanish and Catalan. We then apply the classifiers to 101
newspaper-like articles written by ChatGPT and Bard in the 4 languages at
different time periods. We observe that, similarly to traditional newspapers,
ChatGPT editorial line evolves with time and, being a data-driven system, the
stance of the generated articles differs among languages.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Synthesizing Efficiently Monitorable Formulas in Metric Temporal Logic
Abstract: In runtime verification, manually formalizing a specification for monitoring
system executions is a tedious and error-prone process. To address this issue,
we consider the problem of automatically synthesizing formal specifications
from system executions. To demonstrate our approach, we consider the popular
specification language Metric Temporal Logic (MTL), which is particularly
tailored towards specifying temporal properties for cyber-physical systems
(CPS). Most of the classical approaches for synthesizing temporal logic
formulas aim at minimizing the size of the formula. However, for efficiency in
monitoring, along with the size, the amount of "lookahead" required for the
specification becomes relevant, especially for safety-critical applications. We
formalize this notion and devise a learning algorithm that synthesizes concise
formulas having bounded lookahead. To do so, our algorithm reduces the
synthesis task to a series of satisfiability problems in Linear Real Arithmetic
(LRA) and generates MTL formulas from their satisfying assignments. The
reduction uses a novel encoding of a popular MTL monitoring procedure using
LRA. Finally, we implement our algorithm in a tool called TEAL and demonstrate
its ability to synthesize efficiently monitorable MTL formulas in a CPS
application.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Pseudo-Labeling for Domain-Agnostic Bangla Automatic Speech Recognition
Abstract: One of the major challenges for developing automatic speech recognition (ASR)
for low-resource languages is the limited access to labeled data with
domain-specific variations. In this study, we propose a pseudo-labeling
approach to develop a large-scale domain-agnostic ASR dataset. With the
proposed methodology, we developed a 20k+ hours labeled Bangla speech dataset
covering diverse topics, speaking styles, dialects, noisy environments, and
conversational scenarios. We then exploited the developed corpus to design a
conformer-based ASR system. We benchmarked the trained ASR with publicly
available datasets and compared it with other available models. To investigate
the efficacy, we designed and developed a human-annotated domain-agnostic test
set composed of news, telephony, and conversational data among others. Our
results demonstrate the efficacy of the model trained on psuedo-label data for
the designed test-set along with publicly-available Bangla datasets. The
experimental resources will be publicly
available.(https://github.com/hishab-nlp/Pseudo-Labeling-for-Domain-Agnostic-Bangla-ASR)

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: SPOT! Revisiting Video-Language Models for Event Understanding
Abstract: Understanding videos is an important research topic for multimodal learning.
Leveraging large-scale datasets of web-crawled video-text pairs as weak
supervision has become a pre-training paradigm for learning joint
representations and showcased remarkable potential in video understanding
tasks. However, videos can be multi-event and multi-grained, while these
video-text pairs usually contain only broad-level video captions. This raises a
question: with such weak supervision, can video representation in
video-language models gain the ability to distinguish even factual
discrepancies in textual description and understand fine-grained events? To
address this, we introduce SPOT Prober, to benchmark existing video-language
models's capacities of distinguishing event-level discrepancies as an indicator
of models' event understanding ability. Our approach involves extracting events
as tuples (<Subject, Predicate, Object, Attribute, Timestamps>) from videos and
generating false event tuples by manipulating tuple components systematically.
We reevaluate the existing video-language models with these positive and
negative captions and find they fail to distinguish most of the manipulated
events. Based on our findings, we propose to plug in these manipulated event
captions as hard negative samples and find them effective in enhancing models
for event understanding.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Spatial and Temporal Characteristics of Freight Tours: A Data-Driven Exploratory Analysis
Abstract: This paper presents a modeling approach to infer scheduling and routing
patterns from digital freight transport activity data for different freight
markets. We provide a complete modeling framework including a new
discrete-continuous decision tree approach for extracting rules from the
freight transport data. We apply these models to collected tour data for the
Netherlands to understand departure time patterns and tour strategies, also
allowing us to evaluate the effectiveness of the proposed algorithm. We find
that spatial and temporal characteristics are important to capture the types of
tours and time-of-day patterns of freight activities. Also, the empirical
evidence indicates that carriers in most of the transport markets are sensitive
to the level of congestion. Many of them adjust the type of tour, departure
time, and the number of stops per tour when facing a congested zone. The
results can be used by practitioners to get more grip on transport markets and
develop freight and traffic management measures.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: IMGTB: A Framework for Machine-Generated Text Detection Benchmarking
Abstract: In the era of large language models generating high quality texts, it is a
necessity to develop methods for detection of machine-generated text to avoid
harmful use or simply due to annotation purposes. It is, however, also
important to properly evaluate and compare such developed methods. Recently, a
few benchmarks have been proposed for this purpose; however, integration of
newest detection methods is rather challenging, since new methods appear each
month and provide slightly different evaluation pipelines. In this paper, we
present the IMGTB framework, which simplifies the benchmarking of
machine-generated text detection methods by easy integration of custom (new)
methods and evaluation datasets. Its configurability and flexibility makes
research and development of new detection methods easier, especially their
comparison to the existing state-of-the-art detectors. The default set of
analyses, metrics and visualizations offered by the tool follows the
established practices of machine-generated text detection benchmarking found in
state-of-the-art literature.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: On Diagnostics for Understanding Agent Training Behaviour in Cooperative MARL
Abstract: Cooperative multi-agent reinforcement learning (MARL) has made substantial
strides in addressing the distributed decision-making challenges. However, as
multi-agent systems grow in complexity, gaining a comprehensive understanding
of their behaviour becomes increasingly challenging. Conventionally, tracking
team rewards over time has served as a pragmatic measure to gauge the
effectiveness of agents in learning optimal policies. Nevertheless, we argue
that relying solely on the empirical returns may obscure crucial insights into
agent behaviour. In this paper, we explore the application of explainable AI
(XAI) tools to gain profound insights into agent behaviour. We employ these
diagnostics tools within the context of Level-Based Foraging and Multi-Robot
Warehouse environments and apply them to a diverse array of MARL algorithms. We
demonstrate how our diagnostics can enhance the interpretability and
explainability of MARL systems, providing a better understanding of agent
behaviour.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Cross-Dialect Sentence Transformation: A Comparative Analysis of Language Models for Adapting Sentences to British English
Abstract: This study explores linguistic distinctions among American, Indian, and Irish
English dialects and assesses various Language Models (LLMs) in their ability
to generate British English translations from these dialects. Using cosine
similarity analysis, the study measures the linguistic proximity between
original British English translations and those produced by LLMs for each
dialect. The findings reveal that Indian and Irish English translations
maintain notably high similarity scores, suggesting strong linguistic alignment
with British English. In contrast, American English exhibits slightly lower
similarity, reflecting its distinct linguistic traits. Additionally, the choice
of LLM significantly impacts translation quality, with Llama-2-70b consistently
demonstrating superior performance. The study underscores the importance of
selecting the right model for dialect translation, emphasizing the role of
linguistic expertise and contextual understanding in achieving accurate
translations.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Constrained Equation Learner Networks for Precision-Preserving Extrapolation of Robotic Skills
Abstract: In Programming by Demonstration, the robot learns novel skills from human
demonstrations. After learning, the robot should be able not only to reproduce
the skill, but also to generalize it to shifted domains without collecting new
training data. Adaptation to similar domains has been investigated in the
literature; however, an open problem is how to adapt learned skills to
different conditions that are outside of the data distribution, and, more
important, how to preserve the precision of the desired adaptations. This paper
presents a novel supervised learning framework called Constrained Equation
Learner Networks that addresses the trajectory adaptation problem in
Programming by Demonstrations from a constrained regression perspective. While
conventional approaches for constrained regression use one kind of basis
function, e.g., Gaussian, we exploit Equation Learner Networks to learn a set
of analytical expressions and use them as basis functions. These basis
functions are learned from demonstration with the objective to minimize
deviations from the training data while imposing constraints that represent the
desired adaptations, like new initial or final points or maintaining the
trajectory within given bounds. Our approach addresses three main difficulties
in adapting robotic trajectories: 1) minimizing the distortion of the
trajectory for new adaptations; 2) preserving the precision of the adaptations;
and 3) dealing with the lack of intuition about the structure of basis
functions. We validate our approach both in simulation and in real experiments
in a set of robotic tasks that require adaptation due to changes in the
environment, and we compare obtained results with two existing approaches.
Performed experiments show that Constrained Equation Learner Networks
outperform state of the art approaches by increasing generalization and
adaptability of robotic skills.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: AGRAMPLIFIER: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification
Abstract: The collaborative nature of federated learning (FL) poses a major threat in
the form of manipulation of local training data and local updates, known as the
Byzantine poisoning attack. To address this issue, many Byzantine-robust
aggregation rules (AGRs) have been proposed to filter out or moderate
suspicious local updates uploaded by Byzantine participants.
  This paper introduces a novel approach called AGRAMPLIFIER, aiming to
simultaneously improve the robustness, fidelity, and efficiency of the existing
AGRs. The core idea of AGRAMPLIFIER is to amplify the "morality" of local
updates by identifying the most repressive features of each gradient update,
which provides a clearer distinction between malicious and benign updates,
consequently improving the detection effect. To achieve this objective, two
approaches, namely AGRMP and AGRXAI, are proposed. AGRMP organizes local
updates into patches and extracts the largest value from each patch, while
AGRXAI leverages explainable AI methods to extract the gradient of the most
activated features. By equipping AGRAMPLIFIER with the existing
Byzantine-robust mechanisms, we successfully enhance the model's robustness,
maintaining its fidelity and improving overall efficiency.
  AGRAMPLIFIER is universally compatible with the existing Byzantine-robust
mechanisms. The paper demonstrates its effectiveness by integrating it with all
mainstream AGR mechanisms. Extensive evaluations conducted on seven datasets
from diverse domains against seven representative poisoning attacks
consistently show enhancements in robustness, fidelity, and efficiency, with
average gains of 40.08%, 39.18%, and 10.68%, respectively.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Lighter, yet More Faithful: Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization
Abstract: Despite their remarkable performance on abstractive summarization, large
language models (LLMs) face two significant challenges: their considerable size
and tendency to hallucinate. Hallucinations are concerning because they erode
the reliability of LLMs and raise safety issues. Pruning is a technique that
reduces model size by removing redundant weights to create sparse models that
enable more efficient inference. Pruned models yield comparable performance to
their counterpart full-sized models, making them ideal alternatives when
operating on a limited budget. However, the effect that pruning has upon
hallucinations in abstractive summarization with LLMs has yet to be explored.
In this paper, we provide an extensive empirical study on the hallucinations
produced by pruned models across three standard summarization tasks, two
pruning approaches, three instruction-tuned LLMs, and three hallucination
evaluation metrics. Surprisingly, we find that pruned LLMs hallucinate less
compared to their full-sized counterparts. Our follow-up analysis suggests that
pruned models tend to depend more on the source input and less on their
parametric knowledge from pre-training for generation. This greater dependency
on the source input leads to a higher lexical overlap between generated content
and the source input, which can be a reason for the reduction in
hallucinations.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
Abstract: Large Language Models (LLM) exhibit zero-shot mathematical reasoning capacity
as a behavior emergent with scale, commonly manifesting as chain-of-thoughts
(CoT) reasoning. However, multiple empirical findings suggest that this prowess
is exclusive to LLMs with exorbitant sizes (beyond 50 billion parameters).
Meanwhile, educational neuroscientists suggest that symbolic algebraic
manipulation be introduced around the same time as arithmetic word problems to
modularize language-to-formulation, symbolic manipulation of the formulation,
and endgame arithmetic. In this paper, we start with the hypothesis that much
smaller LMs, which are weak at multi-step reasoning, can achieve reasonable
arithmetic reasoning if arithmetic word problems are posed as a
formalize-then-solve task. In our architecture, which we call SYRELM, the LM
serves the role of a translator to map natural language arithmetic questions
into a formal language (FL) description. A symbolic solver then evaluates the
FL expression to obtain the answer. A small frozen LM, equipped with an
efficient low-rank adapter, is capable of generating FL expressions that
incorporate natural language descriptions of the arithmetic problem (e.g.,
variable names and their purposes, formal expressions combining variables,
etc.). We adopt policy-gradient reinforcement learning to train the adapted LM,
informed by the non-differentiable symbolic solver. This marks a sharp
departure from the recent development in tool-augmented LLMs, in which the
external tools (e.g., calculator, Web search, etc.) are essentially detached
from the learning phase of the LM. SYRELM shows massive improvements (e.g.,
+30.65 absolute point improvement in accuracy on the SVAMP dataset using GPT-J
6B model) over base LMs, while keeping our testbed easy to diagnose, interpret
and within reach of most researchers.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigation
Abstract: Large language models (LLMs) have brought significant advancements to code
generation, benefiting both novice and experienced developers. However, their
training using unsanitized data from open-source repositories, like GitHub,
introduces the risk of inadvertently propagating security vulnerabilities. To
effectively mitigate this concern, this paper presents a comprehensive study
focused on evaluating and enhancing code LLMs from a software security
perspective. We introduce SecuCoGen\footnote{SecuCoGen has been uploaded as
supplemental material and will be made publicly available after publication.},
a meticulously curated dataset targeting 21 critical vulnerability types.
SecuCoGen comprises 180 samples and serves as the foundation for conducting
experiments on three crucial code-related tasks: code generation, code repair
and vulnerability classification, with a strong emphasis on security. Our
experimental results reveal that existing models often overlook security
concerns during code generation, leading to the generation of vulnerable code.
To address this, we propose effective approaches to mitigate the security
vulnerabilities and enhance the overall robustness of code generated by LLMs.
Moreover, our study identifies weaknesses in existing models' ability to repair
vulnerable code, even when provided with vulnerability information.
Additionally, certain vulnerability types pose challenges for the models,
hindering their performance in vulnerability classification. Based on these
findings, we believe our study will have a positive impact on the software
engineering community, inspiring the development of improved methods for
training and utilizing LLMs, thereby leading to safer and more trustworthy
model deployment.

Output: Software Engineering


Instruction: What field is the article from?

Input:
Title: (Debiased) Contrastive Learning Loss for Recommendation (Technical Report)
Abstract: In this paper, we perform a systemic examination of the recommendation
losses, including listwise (softmax), pairwise(BPR), and pointwise
(mean-squared error, MSE, and Cosine Contrastive Loss, CCL) losses through the
lens of contrastive learning. We introduce and study both debiased InfoNCE and
mutual information neural estimator (MINE), for the first time, under the
recommendation setting. We also relate and differentiate these two losses with
the BPR loss through the lower bound analysis. Furthermore, we present the
debiased pointwise loss (for both MSE and CCL) and theoretically certify both
iALS and EASE, two of the most popular linear models, are inherently debiased.
The empirical experimental results demonstrate the effectiveness of the
debiased losses and newly introduced mutual-information losses outperform the
existing (biased) ones.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: NN-Steiner: A Mixed Neural-algorithmic Approach for the Rectilinear Steiner Minimum Tree Problem
Abstract: Recent years have witnessed rapid advances in the use of neural networks to
solve combinatorial optimization problems. Nevertheless, designing the "right"
neural model that can effectively handle a given optimization problem can be
challenging, and often there is no theoretical understanding or justification
of the resulting neural model. In this paper, we focus on the rectilinear
Steiner minimum tree (RSMT) problem, which is of critical importance in IC
layout design and as a result has attracted numerous heuristic approaches in
the VLSI literature. Our contributions are two-fold. On the methodology front,
we propose NN-Steiner, which is a novel mixed neural-algorithmic framework for
computing RSMTs that leverages the celebrated PTAS algorithmic framework of
Arora to solve this problem (and other geometric optimization problems). Our
NN-Steiner replaces key algorithmic components within Arora's PTAS by suitable
neural components. In particular, NN-Steiner only needs four neural network
(NN) components that are called repeatedly within an algorithmic framework.
Crucially, each of the four NN components is only of bounded size independent
of input size, and thus easy to train. Furthermore, as the NN component is
learning a generic algorithmic step, once learned, the resulting mixed
neural-algorithmic framework generalizes to much larger instances not seen in
training. Our NN-Steiner, to our best knowledge, is the first neural
architecture of bounded size that has capacity to approximately solve RSMT (and
variants). On the empirical front, we show how NN-Steiner can be implemented
and demonstrate the effectiveness of our resulting approach, especially in
terms of generalization, by comparing with state-of-the-art methods (both
neural or non-neural based).

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Analyzing the Impact of Companies on AI Research Based on Publications
Abstract: Artificial Intelligence (AI) is one of the most momentous technologies of our
time. Thus, it is of major importance to know which stakeholders influence AI
research. Besides researchers at universities and colleges, researchers in
companies have hardly been considered in this context. In this article, we
consider how the influence of companies on AI research can be made measurable
on the basis of scientific publishing activities. We compare academic- and
company-authored AI publications published in the last decade and use
scientometric data from multiple scholarly databases to look for differences
across these groups and to disclose the top contributing organizations. While
the vast majority of publications is still produced by academia, we find that
the citation count an individual publication receives is significantly higher
when it is (co-)authored by a company. Furthermore, using a variety of
altmetric indicators, we notice that publications with company participation
receive considerably more attention online. Finally, we place our analysis
results in a broader context and present targeted recommendations to safeguard
a harmonious balance between academia and industry in the realm of AI research.

Output: Artificial Intelligence