Instruction: What field is the article from?

Input:
Title: Removing Biases from Molecular Representations via Information Maximization
Abstract: High-throughput drug screening -- using cell imaging or gene expression
measurements as readouts of drug effect -- is a critical tool in biotechnology
to assess and understand the relationship between the chemical structure and
biological activity of a drug. Since large-scale screens have to be divided
into multiple experiments, a key difficulty is dealing with batch effects,
which can introduce systematic errors and non-biological associations in the
data. We propose InfoCORE, an Information maximization approach for COnfounder
REmoval, to effectively deal with batch effects and obtain refined molecular
representations. InfoCORE establishes a variational lower bound on the
conditional mutual information of the latent representations given a batch
identifier. It adaptively reweighs samples to equalize their implied batch
distribution. Extensive experiments on drug screening data reveal InfoCORE's
superior performance in a multitude of tasks including molecular property
prediction and molecule-phenotype retrieval. Additionally, we show results for
how InfoCORE offers a versatile framework and resolves general distribution
shifts and issues of data fairness by minimizing correlation with spurious
features or removing sensitive attributes. The code is available at
https://github.com/uhlerlab/InfoCORE.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Challenges of Radio Frequency Fingerprinting: From Data Collection to Deployment
Abstract: Radio Frequency Fingerprinting (RFF) techniques promise to authenticate
wireless devices at the physical layer based on inherent hardware imperfections
introduced during manufacturing. Such RF transmitter imperfections are
reflected into over-the-air signals, allowing receivers to accurately identify
the RF transmitting source. Recent advances in Machine Learning, particularly
in Deep Learning (DL), have improved the ability of RFF systems to extract and
learn complex features that make up the device-specific fingerprint. However,
integrating DL techniques with RFF and operating the system in real-world
scenarios presents numerous challenges. This article identifies and analyzes
these challenges while considering the three reference phases of any DL-based
RFF system: (i) data collection and preprocessing, (ii) training, and finally,
(iii) deployment. Our investigation points out the current open problems that
prevent real deployment of RFF while discussing promising future directions,
thus paving the way for further research in the area.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: PcLast: Discovering Plannable Continuous Latent States
Abstract: Goal-conditioned planning benefits from learned low-dimensional
representations of rich, high-dimensional observations. While compact latent
representations, typically learned from variational autoencoders or inverse
dynamics, enable goal-conditioned planning they ignore state affordances, thus
hampering their sample-efficient planning capabilities. In this paper, we learn
a representation that associates reachable states together for effective onward
planning. We first learn a latent representation with multi-step inverse
dynamics (to remove distracting information); and then transform this
representation to associate reachable states together in $\ell_2$ space. Our
proposals are rigorously tested in various simulation testbeds. Numerical
results in reward-based and reward-free settings show significant improvements
in sampling efficiency, and yields layered state abstractions that enable
computationally efficient hierarchical planning.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: CLadder: A Benchmark to Assess Causal Reasoning Capabilities of Language Models
Abstract: The ability to perform causal reasoning is widely considered a core feature
of intelligence. In this work, we investigate whether large language models
(LLMs) can coherently reason about causality. Much of the existing work in
natural language processing (NLP) focuses on evaluating commonsense causal
reasoning in LLMs, thus failing to assess whether a model can perform causal
inference in accordance with a set of well-defined formal rules. To address
this, we propose a new NLP task, causal inference in natural language, inspired
by the "causal inference engine" postulated by Judea Pearl et al. We compose a
large dataset, CLadder, with 10K samples: based on a collection of causal
graphs and queries (associational, interventional, and counterfactual), we
obtain symbolic questions and ground-truth answers, through an oracle causal
inference engine. These are then translated into natural language. We evaluate
multiple LLMs on our dataset, and we introduce and evaluate a bespoke
chain-of-thought prompting strategy, CausalCoT. We show that our task is highly
challenging for LLMs, and we conduct an in-depth analysis to gain deeper
insight into the causal reasoning abilities of LLMs. Our data is open-sourced
at https://huggingface.co/datasets/causalNLP/cladder, and our code can be found
at https://github.com/causalNLP/cladder.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: A Negative Result on Gradient Matching for Selective Backprop
Abstract: With increasing scale in model and dataset size, the training of deep neural
networks becomes a massive computational burden. One approach to speed up the
training process is Selective Backprop. For this approach, we perform a forward
pass to obtain a loss value for each data point in a minibatch. The backward
pass is then restricted to a subset of that minibatch, prioritizing high-loss
examples. We build on this approach, but seek to improve the subset selection
mechanism by choosing the (weighted) subset which best matches the mean
gradient over the entire minibatch. We use the gradients w.r.t. the model's
last layer as a cheap proxy, resulting in virtually no overhead in addition to
the forward pass. At the same time, for our experiments we add a simple random
selection baseline which has been absent from prior work. Surprisingly, we find
that both the loss-based as well as the gradient-matching strategy fail to
consistently outperform the random baseline.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
Abstract: Recent work by Power et al. (2022) highlighted a surprising "grokking"
phenomenon in learning arithmetic tasks: a neural net first "memorizes" the
training set, resulting in perfect training accuracy but near-random test
accuracy, and after training for sufficiently longer, it suddenly transitions
to perfect test accuracy. This paper studies the grokking phenomenon in
theoretical setups and shows that it can be induced by a dichotomy of early and
late phase implicit biases. Specifically, when training homogeneous neural nets
with large initialization and small weight decay on both classification and
regression tasks, we prove that the training process gets trapped at a solution
corresponding to a kernel predictor for a long time, and then a very sharp
transition to min-norm/max-margin predictors occurs, leading to a dramatic
change in test accuracy.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
Abstract: In practice, preference learning from human feedback depends on incomplete
data with hidden context. Hidden context refers to data that affects the
feedback received, but which is not represented in the data used to train a
preference model. This captures common issues of data collection, such as
having human annotators with varied preferences, cognitive processes that
result in seemingly irrational behavior, and combining data labeled according
to different criteria. We prove that standard applications of preference
learning, including reinforcement learning from human feedback (RLHF),
implicitly aggregate over hidden contexts according to a well-known voting rule
called Borda count. We show this can produce counter-intuitive results that are
very different from other methods which implicitly aggregate via expected
utility. Furthermore, our analysis formalizes the way that preference learning
from users with diverse values tacitly implements a social choice function. A
key implication of this result is that annotators have an incentive to
misreport their preferences in order to influence the learned model, leading to
vulnerabilities in the deployment of RLHF. As a step towards mitigating these
problems, we introduce a class of methods called distributional preference
learning (DPL). DPL methods estimate a distribution of possible score values
for each alternative in order to better account for hidden context.
Experimental results indicate that applying DPL to RLHF for LLM chatbots
identifies hidden context in the data and significantly reduces subsequent
jailbreak vulnerability. Our code and data are available at
https://github.com/cassidylaidlaw/hidden-context

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Improving Activation Steering in Language Models with Mean-Centring
Abstract: Recent work in activation steering has demonstrated the potential to better
control the outputs of Large Language Models (LLMs), but it involves finding
steering vectors. This is difficult because engineers do not typically know how
features are represented in these models. We seek to address this issue by
applying the idea of mean-centring to steering vectors. We find that taking the
average of activations associated with a target dataset, and then subtracting
the mean of all training activations, results in effective steering vectors. We
test this method on a variety of models on natural language tasks by steering
away from generating toxic text, and steering the completion of a story towards
a target genre. We also apply mean-centring to extract function vectors, more
effectively triggering the execution of a range of natural language tasks by a
significant margin (compared to previous baselines). This suggests that
mean-centring can be used to easily improve the effectiveness of activation
steering in a wide range of contexts.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Multiscale Feature Attribution for Outliers
Abstract: Machine learning techniques can automatically identify outliers in massive
datasets, much faster and more reproducible than human inspection ever could.
But finding such outliers immediately leads to the question: which features
render this input anomalous? We propose a new feature attribution method,
Inverse Multiscale Occlusion, that is specifically designed for outliers, for
which we have little knowledge of the type of features we want to identify and
expect that the model performance is questionable because anomalous test data
likely exceed the limits of the training data. We demonstrate our method on
outliers detected in galaxy spectra from the Dark Energy Survey Instrument and
find its results to be much more interpretable than alternative attribution
approaches.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Recurrent Linear Transformers
Abstract: The self-attention mechanism in the transformer architecture is capable of
capturing long-range dependencies and it is the main reason behind its
effectiveness in processing sequential data. Nevertheless, despite their
success, transformers have two significant drawbacks that still limit their
broader applicability: (1) In order to remember past information, the
self-attention mechanism requires access to the whole history to be provided as
context. (2) The inference cost in transformers is expensive. In this paper we
introduce recurrent alternatives to the transformer self-attention mechanism
that offer a context-independent inference cost, leverage long-range
dependencies effectively, and perform well in practice. We evaluate our
approaches in reinforcement learning problems where the aforementioned
computational limitations make the application of transformers nearly
infeasible. We quantify the impact of the different components of our
architecture in a diagnostic environment and assess performance gains in 2D and
3D pixel-based partially-observable environments. When compared to a
state-of-the-art architecture, GTrXL, inference in our approach is at least 40%
cheaper while reducing memory use in more than 50%. Our approach either
performs similarly or better than GTrXL, improving more than 37% upon GTrXL
performance on harder tasks.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: An Empirical Study of Automated Mislabel Detection in Real World Vision Datasets
Abstract: Major advancements in computer vision can primarily be attributed to the use
of labeled datasets. However, acquiring labels for datasets often results in
errors which can harm model performance. Recent works have proposed methods to
automatically identify mislabeled images, but developing strategies to
effectively implement them in real world datasets has been sparsely explored.
Towards improved data-centric methods for cleaning real world vision datasets,
we first conduct more than 200 experiments carefully benchmarking recently
developed automated mislabel detection methods on multiple datasets under a
variety of synthetic and real noise settings with varying noise levels. We
compare these methods to a Simple and Efficient Mislabel Detector (SEMD) that
we craft, and find that SEMD performs similarly to or outperforms prior
mislabel detection approaches. We then apply SEMD to multiple real world
computer vision datasets and test how dataset size, mislabel removal strategy,
and mislabel removal amount further affect model performance after retraining
on the cleaned data. With careful design of the approach, we find that mislabel
removal leads per-class performance improvements of up to 8% of a retrained
classifier in smaller data regimes.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Griffon: Spelling out All Object Locations at Any Granularity with Large Language Models
Abstract: Replicating the innate human ability to detect all objects based on free-form
texts at any granularity remains a formidable challenge for Vision-Language
models. Current Large Vision Language Models (LVLMs) are predominantly
constrained to grounding a single, pre-existing object, relying solely on data
from Referring Expression Comprehension tasks. The limitation leads to a
compromise in model design, necessitating the introduction of visual expert
models or the integration of customized head structures. Beyond these
constraints, our research delves into the untapped potential of LVLMs and
uncover their inherent capability for basic object perception, allowing them to
accurately identify and locate objects of interest. Building on this insight,
we introduce a novel language-prompted localization dataset designed to fully
unleash the capabilities of LVLMs in integrating fine-grained object perception
with precise location awareness. More importantly, we present
$\textbf{Griffon}$, a purely LVLM-based baseline, which does not require the
introduction of any special tokens, expert models, or additional detection
modules. It simply maintains a consistent structure with popular LVLMs by
unifying data formats across various localization-related scenarios and is
trained end-to-end through a well-designed pipeline. Comprehensive experiments
demonstrate that $\textbf{Griffon}$ not only achieves state-of-the-art
performance on the fine-grained RefCOCO series but also approaches the
capabilities of the expert model Faster RCNN on the detection benchmark MSCOCO.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Physical Reasoning and Object Planning for Household Embodied Agents
Abstract: In this study, we explore the sophisticated domain of task planning for
robust household embodied agents, with a particular emphasis on the intricate
task of selecting substitute objects. We introduce the CommonSense Object
Affordance Task (COAT), a novel framework designed to analyze reasoning
capabilities in commonsense scenarios. This approach is centered on
understanding how these agents can effectively identify and utilize alternative
objects when executing household tasks, thereby offering insights into the
complexities of practical decision-making in real-world environments.Drawing
inspiration from human decision-making, we explore how large language models
tackle this challenge through three meticulously crafted commonsense
question-and-answer datasets, featuring refined rules and human annotations.
Our evaluation of state-of-the-art language models on these datasets sheds
light on three pivotal considerations: 1) aligning an object's inherent utility
with the task at hand, 2) navigating contextual dependencies (societal norms,
safety, appropriateness, and efficiency), and 3) accounting for the current
physical state of the object. To maintain accessibility, we introduce five
abstract variables reflecting an object's physical condition, modulated by
human insights to simulate diverse household scenarios. Our contributions
include insightful Object-Utility mappings addressing the first consideration
and two extensive QA datasets (15k and 130k questions) probing the intricacies
of contextual dependencies and object states. The datasets, along with our
findings, are accessible at: \url{https://github.com/com-phy-affordance/COAT}.
This research not only advances our understanding of physical commonsense
reasoning in language models but also paves the way for future improvements in
household agent intelligence.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: CQM: Curriculum Reinforcement Learning with a Quantized World Model
Abstract: Recent curriculum Reinforcement Learning (RL) has shown notable progress in
solving complex tasks by proposing sequences of surrogate tasks. However, the
previous approaches often face challenges when they generate curriculum goals
in a high-dimensional space. Thus, they usually rely on manually specified goal
spaces. To alleviate this limitation and improve the scalability of the
curriculum, we propose a novel curriculum method that automatically defines the
semantic goal space which contains vital information for the curriculum
process, and suggests curriculum goals over it. To define the semantic goal
space, our method discretizes continuous observations via vector
quantized-variational autoencoders (VQ-VAE) and restores the temporal relations
between the discretized observations by a graph. Concurrently, ours suggests
uncertainty and temporal distance-aware curriculum goals that converges to the
final goals over the automatically composed goal space. We demonstrate that the
proposed method allows efficient explorations in an uninformed environment with
raw goal examples only. Also, ours outperforms the state-of-the-art curriculum
RL methods on data efficiency and performance, in various goal-reaching tasks
even with ego-centric visual inputs.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Neuro-Inspired Fragmentation and Recall to Overcome Catastrophic Forgetting in Curiosity
Abstract: Deep reinforcement learning methods exhibit impressive performance on a range
of tasks but still struggle on hard exploration tasks in large environments
with sparse rewards. To address this, intrinsic rewards can be generated using
forward model prediction errors that decrease as the environment becomes known,
and incentivize an agent to explore novel states. While prediction-based
intrinsic rewards can help agents solve hard exploration tasks, they can suffer
from catastrophic forgetting and actually increase at visited states. We first
examine the conditions and causes of catastrophic forgetting in grid world
environments. We then propose a new method FARCuriosity, inspired by how humans
and animals learn. The method depends on fragmentation and recall: an agent
fragments an environment based on surprisal, and uses different local curiosity
modules (prediction-based intrinsic reward functions) for each fragment so that
modules are not trained on the entire environment. At each fragmentation event,
the agent stores the current module in long-term memory (LTM) and either
initializes a new module or recalls a previously stored module based on its
match with the current state. With fragmentation and recall, FARCuriosity
achieves less forgetting and better overall performance in games with varied
and heterogeneous environments in the Atari benchmark suite of tasks. Thus,
this work highlights the problem of catastrophic forgetting in prediction-based
curiosity methods and proposes a solution.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: TaskBench: Benchmarking Large Language Models for Task Automation
Abstract: Recently, the incredible progress of large language models (LLMs) has ignited
the spark of task automation, which decomposes the complex tasks described by
user instructions into sub-tasks, and invokes external tools to execute them,
and plays a central role in autonomous agents. However, there lacks a
systematic and standardized benchmark to foster the development of LLMs in task
automation. To this end, we introduce TaskBench to evaluate the capability of
LLMs in task automation. Specifically, task automation can be formulated into
three critical stages: task decomposition, tool invocation, and parameter
prediction to fulfill user intent. This complexity makes data collection and
evaluation more challenging compared to common NLP tasks. To generate
high-quality evaluation datasets, we introduce the concept of Tool Graph to
represent the decomposed tasks in user intent, and adopt a back-instruct method
to simulate user instruction and annotations. Furthermore, we propose TaskEval
to evaluate the capability of LLMs from different aspects, including task
decomposition, tool invocation, and parameter prediction. Experimental results
demonstrate that TaskBench can effectively reflects the capability of LLMs in
task automation. Benefiting from the mixture of automated data construction and
human verification, TaskBench achieves a high consistency compared to the human
evaluation, which can be utilized as a comprehensive and faithful benchmark for
LLM-based autonomous agents.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: ChatAnything: Facetime Chat with LLM-Enhanced Personas
Abstract: In this technical report, we target generating anthropomorphized personas for
LLM-based characters in an online manner, including visual appearance,
personality and tones, with only text descriptions. To achieve this, we first
leverage the in-context learning capability of LLMs for personality generation
by carefully designing a set of system prompts. We then propose two novel
concepts: the mixture of voices (MoV) and the mixture of diffusers (MoD) for
diverse voice and appearance generation. For MoV, we utilize the text-to-speech
(TTS) algorithms with a variety of pre-defined tones and select the most
matching one based on the user-provided text description automatically. For
MoD, we combine the recent popular text-to-image generation techniques and
talking head algorithms to streamline the process of generating talking
objects. We termed the whole framework as ChatAnything. With it, users could be
able to animate anything with any personas that are anthropomorphic using just
a few text inputs. However, we have observed that the anthropomorphic objects
produced by current generative models are often undetectable by pre-trained
face landmark detectors, leading to failure of the face motion generation, even
if these faces possess human-like appearances because those images are nearly
seen during the training (e.g., OOD samples). To address this issue, we
incorporate pixel-level guidance to infuse human face landmarks during the
image generation phase. To benchmark these metrics, we have built an evaluation
dataset. Based on it, we verify that the detection rate of the face landmark is
significantly increased from 57.0% to 92.5% thus allowing automatic face
animation based on generated speech content. The code and more results can be
found at https://chatanything.github.io/.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Toward Robust Imperceptible Perturbation against Unauthorized Text-to-image Diffusion-based Synthesis
Abstract: Text-to-image diffusion models allow seamless generation of personalized
images from scant reference photos. Yet, these tools, in the wrong hands, can
fabricate misleading or harmful content, endangering individuals. To address
this problem, existing poisoning-based approaches perturb user images in an
imperceptible way to render them "unlearnable" from malicious uses. We identify
two limitations of these defending approaches: i) sub-optimal due to the
hand-crafted heuristics for solving the intractable bilevel optimization and
ii) lack of robustness against simple data transformations like Gaussian
filtering. To solve these challenges, we propose MetaCloak, which solves the
bi-level poisoning problem with a meta-learning framework with an additional
transformation sampling process to craft transferable and robust perturbation.
Specifically, we employ a pool of surrogate diffusion models to craft
transferable and model-agnostic perturbation. Furthermore, by incorporating an
additional transformation process, we design a simple denoising-error
maximization loss that is sufficient for causing transformation-robust semantic
distortion and degradation in a personalized generation. Extensive experiments
on the VGGFace2 and CelebA-HQ datasets show that MetaCloak outperforms existing
approaches. Notably, MetaCloak can successfully fool online training services
like Replicate, in a black-box manner, demonstrating the effectiveness of
MetaCloak in real-world scenarios. Our code is available at
https://github.com/liuyixin-louis/MetaCloak.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Integrating AI and Learning Analytics for Data-Driven Pedagogical Decisions and Personalized Interventions in Education
Abstract: This research study delves into the conceptualization, development, and
deployment of an innovative learning analytics tool, leveraging the
capabilities of OpenAI's GPT-4 model. This tool is designed to quantify student
engagement, map learning progression, and evaluate the efficacy of diverse
instructional strategies within an educational context. Through the analysis of
various critical data points such as students' stress levels, curiosity,
confusion, agitation, topic preferences, and study methods, the tool offers a
rich, multi-dimensional view of the learning environment. Furthermore, it
employs Bloom's taxonomy as a framework to gauge the cognitive levels addressed
by students' questions, thereby elucidating their learning progression. The
information gathered from these measurements can empower educators by providing
valuable insights to enhance teaching methodologies, pinpoint potential areas
for improvement, and craft personalized interventions for individual students.
The study articulates the design intricacies, implementation strategy, and
thorough evaluation of the learning analytics tool, underscoring its
prospective contributions to enhancing educational outcomes and bolstering
student success. Moreover, the practicalities of integrating the tool within
existing educational platforms and the requisite robust, secure, and scalable
technical infrastructure are addressed. This research opens avenues for
harnessing AI's potential in shaping the future of education, facilitating
data-driven pedagogical decisions, and ultimately fostering a more conducive,
personalized learning environment.

Output: Computers and Society


Instruction: What field is the article from?

Input:
Title: PEFT-MedAware: Large Language Model for Medical Awareness
Abstract: Chat models are capable of answering a wide range of questions, however, the
accuracy of their responses is highly uncertain. In this research, we propose a
specialized PEFT-MedAware model where we utilize parameter-efficient
fine-tuning (PEFT) to enhance the Falcon-1b large language model on specialized
MedQuAD data consisting of 16,407 medical QA pairs, leveraging only 0.44% of
its trainable parameters to enhance computational efficiency. The paper adopts
data preprocessing and PEFT to optimize model performance, complemented by a
BitsAndBytesConfig for efficient transformer training. The resulting model was
capable of outperforming other LLMs in medical question-answering tasks in
specific domains with greater accuracy utilizing limited computational
resources making it suitable for deployment in resource-constrained
environments. We propose further improvements through expanded datasets, larger
models, and feedback mechanisms for sustained medical relevancy. Our work
highlights the efficiency gains and specialized capabilities of PEFT in medical
AI, outpacing standard models in precision without extensive resource demands.
The proposed model and data are released for research purposes only.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Typhoon Intensity Prediction with Vision Transformer
Abstract: Predicting typhoon intensity accurately across space and time is crucial for
issuing timely disaster warnings and facilitating emergency response. This has
vast potential for minimizing life losses and property damages as well as
reducing economic and environmental impacts. Leveraging satellite imagery for
scenario analysis is effective but also introduces additional challenges due to
the complex relations among clouds and the highly dynamic context. Existing
deep learning methods in this domain rely on convolutional neural networks
(CNNs), which suffer from limited per-layer receptive fields. This limitation
hinders their ability to capture long-range dependencies and global contextual
knowledge during inference. In response, we introduce a novel approach, namely
"Typhoon Intensity Transformer" (Tint), which leverages self-attention
mechanisms with global receptive fields per layer. Tint adopts a
sequence-to-sequence feature representation learning perspective. It begins by
cutting a given satellite image into a sequence of patches and recursively
employs self-attention operations to extract both local and global contextual
relations between all patch pairs simultaneously, thereby enhancing per-patch
feature representation learning. Extensive experiments on a publicly available
typhoon benchmark validate the efficacy of Tint in comparison with both
state-of-the-art deep learning and conventional meteorological methods. Our
code is available at https://github.com/chen-huanxin/Tint.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Adaptive Language-based Mental Health Assessment with Item-Response Theory
Abstract: Mental health issues widely vary across individuals - the manifestations of
signs and symptoms can be fairly heterogeneous. Recently, language-based
depression and anxiety assessments have shown promise for capturing this
heterogeneous nature by evaluating a patient's own language, but such
approaches require a large sample of words per person to be accurate. In this
work, we introduce adaptive language-based assessment - the task of iteratively
estimating an individual's psychological score based on limited language
responses to questions that the model also decides to ask. To this end, we
explore two statistical learning-based approaches for measurement/scoring:
classical test theory (CTT) and item response theory (IRT). We find that using
adaptive testing in general can significantly reduce the number of questions
required to achieve high validity (r ~ 0.7) with standardized tests, bringing
down from 11 total questions down to 3 for depression and 5 for anxiety. Given
the combinatorial nature of the problem, we empirically evaluate multiple
strategies for both the ordering and scoring objectives, introducing two new
methods: a semi-supervised item response theory based method (ALIRT), and a
supervised actor-critic based model. While both of the models achieve
significant improvements over random and fixed orderings, we find ALIRT to be a
scalable model that achieves the highest accuracy with lower numbers of
questions (e.g. achieves Pearson r ~ 0.93 after only 3 questions versus asking
all 11 questions). Overall, ALIRT allows prompting a reduced number of
questions without compromising accuracy or overhead computational costs.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: HungerGist: An Interpretable Predictive Model for Food Insecurity
Abstract: The escalating food insecurity in Africa, caused by factors such as war,
climate change, and poverty, demonstrates the critical need for advanced early
warning systems. Traditional methodologies, relying on expert-curated data
encompassing climate, geography, and social disturbances, often fall short due
to data limitations, hindering comprehensive analysis and potential discovery
of new predictive factors. To address this, this paper introduces "HungerGist",
a multi-task deep learning model utilizing news texts and NLP techniques. Using
a corpus of over 53,000 news articles from nine African countries over four
years, we demonstrate that our model, trained solely on news data, outperforms
the baseline method trained on both traditional risk factors and human-curated
keywords. In addition, our method has the ability to detect critical texts that
contain interpretable signals known as "gists." Moreover, our examination of
these gists indicates that this approach has the potential to reveal latent
factors that would otherwise remain concealed in unstructured texts.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Models
Abstract: In the current era, a multitude of language models has emerged to cater to
user inquiries. Notably, the GPT-3.5 Turbo language model has gained
substantial attention as the underlying technology for ChatGPT. Leveraging
extensive parameters, this model adeptly responds to a wide range of questions.
However, due to its reliance on internal knowledge, the accuracy of responses
may not be absolute. This article scrutinizes ChatGPT as a Question Answering
System (QAS), comparing its performance to other existing QASs. The primary
focus is on evaluating ChatGPT's proficiency in extracting responses from
provided paragraphs, a core QAS capability. Additionally, performance
comparisons are made in scenarios without a surrounding passage. Multiple
experiments, exploring response hallucination and considering question
complexity, were conducted on ChatGPT. Evaluation employed well-known Question
Answering (QA) datasets, including SQuAD, NewsQA, and PersianQuAD, across
English and Persian languages. Metrics such as F-score, exact match, and
accuracy were employed in the assessment. The study reveals that, while ChatGPT
demonstrates competence as a generative model, it is less effective in question
answering compared to task-specific models. Providing context improves its
performance, and prompt engineering enhances precision, particularly for
questions lacking explicit answers in provided paragraphs. ChatGPT excels at
simpler factual questions compared to "how" and "why" question types. The
evaluation highlights occurrences of hallucinations, where ChatGPT provides
responses to questions without available answers in the provided context.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: A Sparse Cross Attention-based Graph Convolution Network with Auxiliary Information Awareness for Traffic Flow Prediction
Abstract: Deep graph convolution networks (GCNs) have recently shown excellent
performance in traffic prediction tasks. However, they face some challenges.
First, few existing models consider the influence of auxiliary information,
i.e., weather and holidays, which may result in a poor grasp of
spatial-temporal dynamics of traffic data. Second, both the construction of a
dynamic adjacent matrix and regular graph convolution operations have quadratic
computation complexity, which restricts the scalability of GCN-based models. To
address such challenges, this work proposes a deep encoder-decoder model
entitled AIMSAN. It contains an auxiliary information-aware module (AIM) and
sparse cross attention-based graph convolution network (SAN). The former learns
multi-attribute auxiliary information and obtains its embedded presentation of
different time-window sizes. The latter uses a cross-attention mechanism to
construct dynamic adjacent matrices by fusing traffic data and embedded
auxiliary data. Then, SAN applies diffusion GCN on traffic data to mine rich
spatial-temporal dynamics. Furthermore, AIMSAN considers and uses the spatial
sparseness of traffic nodes to reduce the quadratic computation complexity.
Experimental results on three public traffic datasets demonstrate that the
proposed method outperforms other counterparts in terms of various performance
indices. Specifically, the proposed method has competitive performance with the
state-of-the-art algorithms but saves 35.74% of GPU memory usage, 42.25% of
training time, and 45.51% of validation time on average.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Integrated Drill Boom Hole-Seeking Control via Reinforcement Learning
Abstract: Intelligent drill boom hole-seeking is a promising technology for enhancing
drilling efficiency, mitigating potential safety hazards, and relieving human
operators. Most existing intelligent drill boom control methods rely on a
hierarchical control framework based on inverse kinematics. However, these
methods are generally time-consuming due to the computational complexity of
inverse kinematics and the inefficiency of the sequential execution of multiple
joints. To tackle these challenges, this study proposes an integrated drill
boom control method based on Reinforcement Learning (RL). We develop an
integrated drill boom control framework that utilizes a parameterized policy to
directly generate control inputs for all joints at each time step, taking
advantage of joint posture and target hole information. By formulating the
hole-seeking task as a Markov decision process, contemporary mainstream RL
algorithms can be directly employed to learn a hole-seeking policy, thus
eliminating the need for inverse kinematics solutions and promoting cooperative
multi-joint control. To enhance the drilling accuracy throughout the entire
drilling process, we devise a state representation that combines
Denavit-Hartenberg joint information and preview hole-seeking discrepancy data.
Simulation results show that the proposed method significantly outperforms
traditional methods in terms of hole-seeking accuracy and time efficiency.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: Spatio-Temporal Graph Neural Point Process for Traffic Congestion Event Prediction
Abstract: Traffic congestion event prediction is an important yet challenging task in
intelligent transportation systems. Many existing works about traffic
prediction integrate various temporal encoders and graph convolution networks
(GCNs), called spatio-temporal graph-based neural networks, which focus on
predicting dense variables such as flow, speed and demand in time snapshots,
but they can hardly forecast the traffic congestion events that are sparsely
distributed on the continuous time axis. In recent years, neural point process
(NPP) has emerged as an appropriate framework for event prediction in
continuous time scenarios. However, most conventional works about NPP cannot
model the complex spatio-temporal dependencies and congestion evolution
patterns. To address these limitations, we propose a spatio-temporal graph
neural point process framework, named STGNPP for traffic congestion event
prediction. Specifically, we first design the spatio-temporal graph learning
module to fully capture the long-range spatio-temporal dependencies from the
historical traffic state data along with the road network. The extracted
spatio-temporal hidden representation and congestion event information are then
fed into a continuous gated recurrent unit to model the congestion evolution
patterns. In particular, to fully exploit the periodic information, we also
improve the intensity function calculation of the point process with a periodic
gated mechanism. Finally, our model simultaneously predicts the occurrence time
and duration of the next congestion. Extensive experiments on two real-world
datasets demonstrate that our method achieves superior performance in
comparison to existing state-of-the-art approaches.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Combating the effects of speed and delays in end-to-end self-driving
Abstract: In the behavioral cloning approach to end-to-end driving, a dataset of expert
driving is collected and the model learns to guess what the expert would do in
different situations. Situations are summarized in observations and the outputs
are low or mid-level commands (e.g. brake, throttle, and steering; or
trajectories). The models learn to match observations at time T to actions
recorded at T or as simultaneously as possible. However, when deploying the
models to the real world (or to an asynchronous simulation), the action
predicted based on observations at time T gets applied at T + $\Delta$ T. In a
variety of cases, $\Delta$ T can be considerable and significantly influence
performance.
  We first demonstrate that driving at two different speeds is effectively two
different tasks. Delays partially cause this difference and linearly amplify
it. Even without computational delays, actuator delays and slipping due to
inertia result in the need to perform actions preemptively when driving fast.
The function mapping observations to commands becomes different compared to
slow driving. We experimentally show that models trained to drive fast cannot
perform the seemingly easier task of driving slow and vice-versa. Good driving
models may be judged to be poor due to testing them at "a safe low speed", a
task they cannot perform.
  Secondly, we show how to counteract the effect of delays in end-to-end
networks by changing the target labels. This is in contrast to the approaches
attempting to minimize the delays, i.e. the cause, not the effect. To exemplify
the problems and solutions in the real world, we use 1:10 scale minicars with
limited computing power, using behavioral cloning for end-to-end driving. Some
of the ideas discussed here may be transferable to the wider context of
self-driving, to vehicles with more compute power and end-to-mid or modular
approaches.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: Continual Instruction Tuning for Large Multimodal Models
Abstract: Instruction tuning is now a widely adopted approach to aligning large
multimodal models (LMMs) to follow human intent. It unifies the data format of
vision-language tasks, enabling multi-task joint training. However,
vision-language tasks are constantly being created in practice. Instead of
always re-training LMMs when new tasks arrive, continual learning offers
flexibility for models to continually and efficiently exploit the evolving
data. This work aims to explore the following two questions: 1) Do LMMs still
suffer from catastrophic forgetting in continual instruction tuning? 2) Are the
existing three classes of continual learning methods still applicable to the
continual instruction tuning of LMMs? An extensive study is conducted to
address the above questions. First, we establish the first benchmark in this
setting and reveal that catastrophic forgetting is still observed when
continually instruction-tuning LMMs. However, the multi-task joint instruction
tuning can facilitate the model's continual learning ability and mitigate
forgetting. Second, we integrate and adapt classic continual learning methods
to our context, demonstrating the efficacy of data replay and model expansion
strategies across diverse scenarios. In contrast, regularization-based methods
only perform well on models that have been jointly instruction-tuned on
multiple tasks. Third, we delve into the correlation and forgetting dynamics
between vision-language task pairs and propose task-similarity-informed
regularization and model expansion methods for continual instruction tuning of
LMMs. Experimental results show that our approach consistently boosts the
model's performance.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Using Think-Aloud Data to Understand Relations between Self-Regulation Cycle Characteristics and Student Performance in Intelligent Tutoring Systems
Abstract: Numerous studies demonstrate the importance of self-regulation during
learning by problem-solving. Recent work in learning analytics has largely
examined students' use of SRL concerning overall learning gains. Limited
research has related SRL to in-the-moment performance differences among
learners. The present study investigates SRL behaviors in relationship to
learners' moment-by-moment performance while working with intelligent tutoring
systems for stoichiometry chemistry. We demonstrate the feasibility of labeling
SRL behaviors based on AI-generated think-aloud transcripts, identifying the
presence or absence of four SRL categories (processing information, planning,
enacting, and realizing errors) in each utterance. Using the SRL codes, we
conducted regression analyses to examine how the use of SRL in terms of
presence, frequency, cyclical characteristics, and recency relate to student
performance on subsequent steps in multi-step problems. A model considering
students' SRL cycle characteristics outperformed a model only using
in-the-moment SRL assessment. In line with theoretical predictions, students'
actions during earlier, process-heavy stages of SRL cycles exhibited lower
moment-by-moment correctness during problem-solving than later SRL cycle
stages. We discuss system re-design opportunities to add SRL support during
stages of processing and paths forward for using machine learning to speed
research depending on the assessment of SRL based on transcription of
think-aloud data.

Output: Computers and Society


Instruction: What field is the article from?

Input:
Title: Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs
Abstract: Despite Graph neural networks' significant performance gain over many classic
techniques in various graph-related downstream tasks, their successes are
restricted in shallow models due to over-smoothness and the difficulties of
optimizations among many other issues. In this paper, to alleviate the
over-smoothing issue, we propose a soft graph normalization method to preserve
the diversities of node embeddings and prevent indiscrimination due to possible
over-closeness. Combined with residual connections, we analyze the reason why
the method can effectively capture the knowledge in both input graph structures
and node features even with deep networks. Additionally, inspired by Curriculum
Learning that learns easy examples before the hard ones, we propose a novel
label-smoothing-based learning framework to enhance the optimization of deep
GNNs, which iteratively smooths labels in an auxiliary graph and constructs
many gradual non-smooth tasks for extracting increasingly complex knowledge and
gradually discriminating nodes from coarse to fine. The method arguably reduces
the risk of overfitting and generalizes better results. Finally, extensive
experiments are carried out to demonstrate the effectiveness and potential of
the proposed model and learning framework through comparison with twelve
existing baselines including the state-of-the-art methods on twelve real-world
node classification benchmarks.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions
Abstract: Domain invariant learning aims to learn models that extract invariant
features over various training domains, resulting in better generalization to
unseen target domains. Recently, Bayesian Neural Networks have achieved
promising results in domain invariant learning, but most works concentrate on
aligning features distributions rather than parameter distributions. Inspired
by the principle of Bayesian Neural Network, we attempt to directly learn the
domain invariant posterior distribution of network parameters. We first propose
a theorem to show that the invariant posterior of parameters can be implicitly
inferred by aggregating posteriors on different training domains. Our
assumption is more relaxed and allows us to extract more domain invariant
information. We also propose a simple yet effective method, named PosTerior
Generalization (PTG), that can be used to estimate the invariant parameter
distribution. PTG fully exploits variational inference to approximate parameter
distributions, including the invariant posterior and the posteriors on training
domains. Furthermore, we develop a lite version of PTG for widespread
applications. PTG shows competitive performance on various domain
generalization benchmarks on DomainBed. Additionally, PTG can use any existing
domain generalization methods as its prior, and combined with previous
state-of-the-art method the performance can be further improved. Code will be
made public.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: SOccDPT: Semi-Supervised 3D Semantic Occupancy from Dense Prediction Transformers trained under memory constraints
Abstract: We present SOccDPT, a memory-efficient approach for 3D semantic occupancy
prediction from monocular image input using dense prediction transformers. To
address the limitations of existing methods trained on structured traffic
datasets, we train our model on unstructured datasets including the Indian
Driving Dataset and Bengaluru Driving Dataset. Our semi-supervised training
pipeline allows SOccDPT to learn from datasets with limited labels by reducing
the requirement for manual labelling by substituting it with pseudo-ground
truth labels to produce our Bengaluru Semantic Occupancy Dataset. This broader
training enhances our model's ability to handle unstructured traffic scenarios
effectively. To overcome memory limitations during training, we introduce
patch-wise training where we select a subset of parameters to train each epoch,
reducing memory usage during auto-grad graph construction. In the context of
unstructured traffic and memory-constrained training and inference, SOccDPT
outperforms existing disparity estimation approaches as shown by the RMSE score
of 9.1473, achieves a semantic segmentation IoU score of 46.02% and operates at
a competitive frequency of 69.47 Hz. We make our code and semantic occupancy
dataset public.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: High-throughput Biomedical Relation Extraction for Semi-Structured Web Articles Empowered by Large Language Models
Abstract: Objective: To develop a high-throughput biomedical relation extraction system
that takes advantage of the large language models' (LLMs) reading comprehension
ability and biomedical world knowledge in a scalable and evidential manner.
Methods: We formulate the relation extraction task as a simple binary
classification problem for large language models such as ChatGPT. Specifically,
LLMs make the decision based on the external corpus and its world knowledge,
giving the reason for the judgment to factual verification. This method is
tailored for semi-structured web articles, wherein we designate the main title
as the tail entity and explicitly incorporate it into the context, and the
potential head entities are matched based on a biomedical thesaurus. Moreover,
lengthy contents are sliced into text chunks, embedded, and retrieved with
additional embedding models, ensuring compatibility with the context window
size constraints of available open-source LLMs. Results: Using an open-source
LLM, we extracted 304315 relation triplets of three distinct relation types
from four reputable biomedical websites. To assess the efficacy of the basic
pipeline employed for biomedical relation extraction, we curated a benchmark
dataset annotated by a medical expert. Evaluation results indicate that the
pipeline exhibits performance comparable to that of GPT-4. Case studies further
illuminate challenges faced by contemporary LLMs in the context of biomedical
relation extraction for semi-structured web articles. Conclusion: The proposed
method has demonstrated its effectiveness in leveraging the strengths of LLMs
for high-throughput biomedical relation extraction. Its adaptability is
evident, as it can be seamlessly extended to diverse semi-structured biomedical
websites, facilitating the extraction of various types of biomedical relations
with ease.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Federated Learning for Short Text Clustering
Abstract: Short text clustering has been popularly studied for its significance in
mining valuable insights from many short texts. In this paper, we focus on the
federated short text clustering (FSTC) problem, i.e., clustering short texts
that are distributed in different clients, which is a realistic problem under
privacy requirements. Compared with the centralized short text clustering
problem that short texts are stored on a central server, the FSTC problem has
not been explored yet. To fill this gap, we propose a Federated Robust Short
Text Clustering (FSTC) framework. FSTC includes two main modules, i.e., robust
short text clustering module and federated cluster center aggregation module.
The robust short text clustering module aims to train an effective short text
clustering model with local data in each client. We innovatively combine
optimal transport to generate pseudo-labels with Gaussian-uniform mixture model
to ensure the reliability of the pseudo-supervised data. The federated cluster
center aggregation module aims to exchange knowledge across clients without
sharing local raw data in an efficient way. The server aggregates the local
cluster centers from different clients and then sends the global centers back
to all clients in each communication round. Our empirical studies on three
short text clustering datasets demonstrate that FSTC significantly outperforms
the federated short text clustering baselines.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation
Abstract: Recent advances in image tokenizers, such as VQ-VAE, have enabled
text-to-image generation using auto-regressive methods, similar to language
modeling. However, these methods have yet to leverage pre-trained language
models, despite their adaptability to various downstream tasks. In this work,
we explore this gap by adapting a pre-trained language model for
auto-regressive text-to-image generation, and find that pre-trained language
models offer limited help. We provide a two-fold explanation by analyzing
tokens from each modality. First, we demonstrate that image tokens possess
significantly different semantics compared to text tokens, rendering
pre-trained language models no more effective in modeling them than randomly
initialized ones. Second, the text tokens in the image-text datasets are too
simple compared to normal language model pre-training data, which causes the
catastrophic degradation of language models' capability.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text
Abstract: We present an overview of the ArAIEval shared task, organized as part of the
first ArabicNLP 2023 conference co-located with EMNLP 2023. ArAIEval offers two
tasks over Arabic text: (i) persuasion technique detection, focusing on
identifying persuasion techniques in tweets and news articles, and (ii)
disinformation detection in binary and multiclass setups over tweets. A total
of 20 teams participated in the final evaluation phase, with 14 and 16 teams
participating in Tasks 1 and 2, respectively. Across both tasks, we observed
that fine-tuning transformer models such as AraBERT was at the core of the
majority of the participating systems. We provide a description of the task
setup, including a description of the dataset construction and the evaluation
setup. We further give a brief overview of the participating systems. All
datasets and evaluation scripts from the shared task are released to the
research community. (https://araieval.gitlab.io/) We hope this will enable
further research on these important tasks in Arabic.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Comparing Photorealistic and Animated Embodied Conversational Agents in Serious Games: An Empirical Study on User Experience
Abstract: Embodied conversational agents (ECAs) are paradigms of conversational user
interfaces in the form of embodied characters. While ECAs offer various
manipulable features, this paper focuses on a study conducted to explore two
distinct levels of presentation realism. The two agent versions are
photorealistic and animated. The study aims to provide insights and design
suggestions for speech-enabled ECAs within serious game environments. A
within-subjects, two-by-two factorial design was employed for this research
with a cohort of 36 participants balanced for gender. The results showed that
both the photorealistic and the animated versions were perceived as highly
usable, with overall mean scores of 5.76 and 5.71, respectively. However, 69.4
per cent of the participants stated they preferred the photorealistic version,
25 per cent stated they preferred the animated version and 5.6 per cent had no
stated preference. The photorealistic agents were perceived as more realistic
and human-like, while the animated characters made the task feel more like a
game. Even though the agents' realism had no significant effect on usability,
it positively influenced participants' perceptions of the agent. This research
aims to lay the groundwork for future studies on ECA realism's impact in
serious games across diverse contexts.

Output: Human-Computer Interaction


Instruction: What field is the article from?

Input:
Title: Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification
Abstract: Recent technological advancements have led to a large number of patents in a
diverse range of domains, making it challenging for human experts to analyze
and manage. State-of-the-art methods for multi-label patent classification rely
on deep neural networks (DNNs), which are complex and often considered
black-boxes due to their opaque decision-making processes. In this paper, we
propose a novel deep explainable patent classification framework by introducing
layer-wise relevance propagation (LRP) to provide human-understandable
explanations for predictions. We train several DNN models, including Bi-LSTM,
CNN, and CNN-BiLSTM, and propagate the predictions backward from the output
layer up to the input layer of the model to identify the relevance of words for
individual predictions. Considering the relevance score, we then generate
explanations by visualizing relevant words for the predicted patent class.
Experimental results on two datasets comprising two-million patent texts
demonstrate high performance in terms of various evaluation measures. The
explanations generated for each prediction highlight important relevant words
that align with the predicted class, making the prediction more understandable.
Explainable systems have the potential to facilitate the adoption of complex
AI-enabled methods for patent classification in real-world applications.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs
Abstract: We introduce Lumos, a novel framework for training language agents that
employs a unified data format and a modular architecture based on open-source
large language models (LLMs). Lumos consists of three distinct modules:
planning, grounding, and execution. The planning module breaks down a task into
a series of high-level, tool-agnostic subgoals, which are then made specific by
the grounding module through a set of low-level actions. These actions are
subsequently executed by the execution module, utilizing a range of
off-the-shelf tools and APIs. In order to train these modules effectively,
high-quality annotations of subgoals and actions were collected and are made
available for fine-tuning open-source LLMs for various tasks such as complex
question answering, web tasks, and math problems. Leveraging this unified data
and modular design, Lumos not only achieves comparable or superior performance
to current, state-of-the-art agents, but also exhibits several key advantages:
(1) Lumos surpasses GPT-4/3.5-based agents in complex question answering and
web tasks, while equalling the performance of significantly larger LLM agents
on math tasks; (2) Lumos outperforms open-source agents created through
conventional training methods and those using chain-of-thoughts training; and
(3) Lumos is capable of effectively generalizing to unseen interactive tasks,
outperforming larger LLM-based agents and even exceeding performance of
specialized agents.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method
Abstract: Large Language Models (LLMs) have shown great potential in Natural Language
Processing (NLP) tasks. However, recent literature reveals that LLMs generate
nonfactual responses intermittently, which impedes the LLMs' reliability for
further utilization. In this paper, we propose a novel self-detection method to
detect which questions that a LLM does not know that are prone to generate
nonfactual results. Specifically, we first diversify the textual expressions
for a given question and collect the corresponding answers. Then we examine the
divergencies between the generated answers to identify the questions that the
model may generate falsehoods. All of the above steps can be accomplished by
prompting the LLMs themselves without referring to any other external
resources. We conduct comprehensive experiments and demonstrate the
effectiveness of our method on recently released LLMs, e.g., Vicuna, ChatGPT,
and GPT-4.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
Abstract: Recent advancements in large language models (LLMs) have significantly
expanded their functionality and skills as tool agents. In this paper, we argue
that a waveform pattern in the model's attention allocation has an impact on
the tool use performance, which degrades when the position of essential
information hits the trough zone. To address this issue, we propose a novel
inference method named Attention Buckets. This approach enables LLMs to handle
context by conducting parallel processes, each featuring a unique RoPE angle
base that shapes the attention waveform. Attention Buckets ensures that an
attention trough of a particular process can be compensated with an attention
peak of another run, reducing the risk of the LLM missing essential information
residing within the attention trough. Our extensive experiments on the widely
recognized tool use benchmark demonstrate the efficacy of our approach, where a
7B-parameter open-source model enhanced by Attention Buckets achieves SOTA
performance on par with GPT-4.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: BanglaBait: Semi-Supervised Adversarial Approach for Clickbait Detection on Bangla Clickbait Dataset
Abstract: Intentionally luring readers to click on a particular content by exploiting
their curiosity defines a title as clickbait. Although several studies focused
on detecting clickbait titles in English articles, low resource language like
Bangla has not been given adequate attention. To tackle clickbait titles in
Bangla, we have constructed the first Bangla clickbait detection dataset
containing 15,056 labeled news articles and 65,406 unlabelled news articles
extracted from clickbait dense news sites. Each article has been labeled by
three expert linguists and includes an article's title, body, and other
metadata. By incorporating labeled and unlabelled data, we finetune a
pretrained Bangla transformer model in an adversarial fashion using Semi
Supervised Generative Adversarial Networks (SS GANs). The proposed model acts
as a good baseline for this dataset, outperforming traditional neural network
models (LSTM, GRU, CNN) and linguistic feature based models. We expect that
this dataset and the detailed analysis and comparison of these clickbait
detection models will provide a fundamental basis for future research into
detecting clickbait titles in Bengali articles. We have released the
corresponding code and dataset.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Toward a Critical Toponymy Framework for Named Entity Recognition: A Case Study of Airbnb in New York City
Abstract: Critical toponymy examines the dynamics of power, capital, and resistance
through place names and the sites to which they refer. Studies here have
traditionally focused on the semantic content of toponyms and the top-down
institutional processes that produce them. However, they have generally ignored
the ways in which toponyms are used by ordinary people in everyday discourse,
as well as the other strategies of geospatial description that accompany and
contextualize toponymic reference. Here, we develop computational methods to
measure how cultural and economic capital shape the ways in which people refer
to places, through a novel annotated dataset of 47,440 New York City Airbnb
listings from the 2010s. Building on this dataset, we introduce a new named
entity recognition (NER) model able to identify important discourse categories
integral to the characterization of place. Our findings point toward new
directions for critical toponymy and to a range of previously understudied
linguistic signals relevant to research on neighborhood status, housing and
tourism markets, and gentrification.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Analyze the robustness of three NMF algorithms (Robust NMF with L1 norm, L2-1 norm NMF, L2 NMF)
Abstract: Non-negative matrix factorization (NMF) and its variants have been widely
employed in clustering and classification tasks (Long, & Jian , 2021). However,
noises can seriously affect the results of our experiments. Our research is
dedicated to investigating the noise robustness of non-negative matrix
factorization (NMF) in the face of different types of noise. Specifically, we
adopt three different NMF algorithms, namely L1 NMF, L2 NMF, and L21 NMF, and
use the ORL and YaleB data sets to simulate a series of experiments with
salt-and-pepper noise and Block-occlusion noise separately. In the experiment,
we use a variety of evaluation indicators, including root mean square error
(RMSE), accuracy (ACC), and normalized mutual information (NMI), to evaluate
the performance of different NMF algorithms in noisy environments. Through
these indicators, we quantify the resistance of NMF algorithms to noise and
gain insights into their feasibility in practical applications.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: On the Opportunities of Green Computing: A Survey
Abstract: Artificial Intelligence (AI) has achieved significant advancements in
technology and research with the development over several decades, and is
widely used in many areas including computing vision, natural language
processing, time-series analysis, speech synthesis, etc. During the age of deep
learning, especially with the arise of Large Language Models, a large majority
of researchers' attention is paid on pursuing new state-of-the-art (SOTA)
results, resulting in ever increasing of model size and computational
complexity. The needs for high computing power brings higher carbon emission
and undermines research fairness by preventing small or medium-sized research
institutions and companies with limited funding in participating in research.
To tackle the challenges of computing resources and environmental impact of AI,
Green Computing has become a hot research topic. In this survey, we give a
systematic overview of the technologies used in Green Computing. We propose the
framework of Green Computing and devide it into four key components: (1)
Measures of Greenness, (2) Energy-Efficient AI, (3) Energy-Efficient Computing
Systems and (4) AI Use Cases for Sustainability. For each components, we
discuss the research progress made and the commonly used techniques to optimize
the AI efficiency. We conclude that this new research direction has the
potential to address the conflicts between resource constraints and AI
development. We encourage more researchers to put attention on this direction
and make AI more environmental friendly.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Anonymous Jamming Detection in 5G with Bayesian Network Model Based Inference Analysis
Abstract: Jamming and intrusion detection are critical in 5G research, aiming to
maintain reliability, prevent user experience degradation, and avoid
infrastructure failure. This paper introduces an anonymous jamming detection
model for 5G based on signal parameters from the protocol stacks. The system
uses supervised and unsupervised learning for real-time, high-accuracy
detection of jamming, including unknown types. Supervised models reach an AUC
of 0.964 to 1, compared to LSTM models with an AUC of 0.923 to 1. However, the
need for data annotation limits the supervised approach. To address this, an
unsupervised auto-encoder-based anomaly detection is presented with an AUC of
0.987. The approach is resistant to adversarial training samples. For
transparency and domain knowledge injection, a Bayesian network-based causation
analysis is introduced.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: MPCNN: A Novel Matrix Profile Approach for CNN-based Sleep Apnea Classification
Abstract: Sleep apnea (SA) is a significant respiratory condition that poses a major
global health challenge. Previous studies have investigated several machine and
deep learning models for electrocardiogram (ECG)-based SA diagnoses. Despite
these advancements, conventional feature extractions derived from ECG signals,
such as R-peaks and RR intervals, may fail to capture crucial information
encompassed within the complete PQRST segments. In this study, we propose an
innovative approach to address this diagnostic gap by delving deeper into the
comprehensive segments of the ECG signal. The proposed methodology draws
inspiration from Matrix Profile algorithms, which generate an Euclidean
distance profile from fixed-length signal subsequences. From this, we derived
the Min Distance Profile (MinDP), Max Distance Profile (MaxDP), and Mean
Distance Profile (MeanDP) based on the minimum, maximum, and mean of the
profile distances, respectively. To validate the effectiveness of our approach,
we use the modified LeNet-5 architecture as the primary CNN model, along with
two existing lightweight models, BAFNet and SE-MSCNN, for ECG classification
tasks. Our extensive experimental results on the PhysioNet Apnea-ECG dataset
revealed that with the new feature extraction method, we achieved a per-segment
accuracy up to 92.11 \% and a per-recording accuracy of 100\%. Moreover, it
yielded the highest correlation compared to state-of-the-art methods, with a
correlation coefficient of 0.989. By introducing a new feature extraction
method based on distance relationships, we enhanced the performance of certain
lightweight models, showing potential for home sleep apnea test (HSAT) and SA
detection in IoT devices. The source code for this work is made publicly
available in GitHub: https://github.com/vinuni-vishc/MPCNN-Sleep-Apnea.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: DeepCache: Accelerating Diffusion Models for Free
Abstract: Diffusion models have recently gained unprecedented attention in the field of
image synthesis due to their remarkable generative capabilities.
Notwithstanding their prowess, these models often incur substantial
computational costs, primarily attributed to the sequential denoising process
and cumbersome model size. Traditional methods for compressing diffusion models
typically involve extensive retraining, presenting cost and feasibility
challenges. In this paper, we introduce DeepCache, a novel training-free
paradigm that accelerates diffusion models from the perspective of model
architecture. DeepCache capitalizes on the inherent temporal redundancy
observed in the sequential denoising steps of diffusion models, which caches
and retrieves features across adjacent denoising stages, thereby curtailing
redundant computations. Utilizing the property of the U-Net, we reuse the
high-level features while updating the low-level features in a very cheap way.
This innovative strategy, in turn, enables a speedup factor of 2.3$\times$ for
Stable Diffusion v1.5 with only a 0.05 decline in CLIP Score, and 4.1$\times$
for LDM-4-G with a slight decrease of 0.22 in FID on ImageNet. Our experiments
also demonstrate DeepCache's superiority over existing pruning and distillation
methods that necessitate retraining and its compatibility with current sampling
techniques. Furthermore, we find that under the same throughput, DeepCache
effectively achieves comparable or even marginally improved results with DDIM
or PLMS. The code is available at https://github.com/horseee/DeepCache

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: On learning spatial sequences with the movement of attention
Abstract: In this paper we start with a simple question, how is it possible that humans
can recognize different movements over skin with only a prior visual experience
of them? Or in general, what is the representation of spatial sequences that
are invariant to scale, rotation, and translation across different modalities?
To answer, we rethink the mathematical representation of spatial sequences,
argue against the minimum description length principle, and focus on the
movements of attention. We advance the idea that spatial sequences must be
represented on different levels of abstraction, this adds redundancy but is
necessary for recognition and generalization. To address the open question of
how these abstractions are formed we propose two hypotheses: the first invites
exploring selectionism learning, instead of finding parameters in some models;
the second proposes to find new data structures, not neural network
architectures, to efficiently store and operate over redundant features to be
further selected. Movements of attention are central to human cognition and
lessons should be applied to new better learning algorithms.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Harnessing Discrete Representations For Continual Reinforcement Learning
Abstract: Reinforcement learning (RL) agents make decisions using nothing but
observations from the environment, and consequently, heavily rely on the
representations of those observations. Though some recent breakthroughs have
used vector-based categorical representations of observations, often referred
to as discrete representations, there is little work explicitly assessing the
significance of such a choice. In this work, we provide a thorough empirical
investigation of the advantages of representing observations as vectors of
categorical values within the context of reinforcement learning. We perform
evaluations on world-model learning, model-free RL, and ultimately continual RL
problems, where the benefits best align with the needs of the problem setting.
We find that, when compared to traditional continuous representations, world
models learned over discrete representations accurately model more of the world
with less capacity, and that agents trained with discrete representations learn
better policies with less data. In the context of continual RL, these benefits
translate into faster adapting agents. Additionally, our analysis suggests that
the observed performance improvements can be attributed to the information
contained within the latent vectors and potentially the encoding of the
discrete representation itself.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Weight-Entanglement Meets Gradient-Based Neural Architecture Search
Abstract: Weight sharing is a fundamental concept in neural architecture search (NAS),
enabling gradient-based methods to explore cell-based architecture spaces
significantly faster than traditional blackbox approaches. In parallel, weight
\emph{entanglement} has emerged as a technique for intricate parameter sharing
among architectures within macro-level search spaces. %However, the macro
structure of such spaces poses compatibility challenges for gradient-based NAS
methods. %As a result, blackbox optimization methods have been commonly
employed, particularly in conjunction with supernet training, to maintain
search efficiency. %Due to the inherent differences in the structure of these
search spaces, these Since weight-entanglement poses compatibility challenges
for gradient-based NAS methods, these two paradigms have largely developed
independently in parallel sub-communities. This paper aims to bridge the gap
between these sub-communities by proposing a novel scheme to adapt
gradient-based methods for weight-entangled spaces. This enables us to conduct
an in-depth comparative assessment and analysis of the performance of
gradient-based NAS in weight-entangled search spaces. Our findings reveal that
this integration of weight-entanglement and gradient-based NAS brings forth the
various benefits of gradient-based methods (enhanced performance, improved
supernet training properties and superior any-time performance), while
preserving the memory efficiency of weight-entangled spaces. The code for our
work is openly accessible
\href{https://anonymous.4open.science/r/TangleNAS-527C}{here}

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Prototype of deployment of Federated Learning with IoT devices
Abstract: In the age of technology, data is an increasingly important resource. This
importance is growing in the field of Artificial Intelligence (AI), where sub
fields such as Machine Learning (ML) need more and more data to achieve better
results. Internet of Things (IoT) is the connection of sensors and smart
objects to collect and exchange data, in addition to achieving many other
tasks. A huge amount of the resource desired, data, is stored in mobile
devices, sensors and other Internet of Things (IoT) devices, but remains there
due to data protection restrictions. At the same time these devices do not have
enough data or computational capacity to train good models. Moreover,
transmitting, storing and processing all this data on a centralised server is
problematic. Federated Learning (FL) provides an innovative solution that
allows devices to learn in a collaborative way. More importantly, it
accomplishes this without violating data protection laws. FL is currently
growing, and there are several solutions that implement it. This article
presents a prototype of a FL solution where the IoT devices used were raspberry
pi boards. The results compare the performance of a solution of this type with
those obtained in traditional approaches. In addition, the FL solution
performance was tested in a hostile environment. A convolutional neural network
(CNN) and a image data set were used. The results show the feasibility and
usability of these techniques, although in many cases they do not reach the
performance of traditional approaches.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning
Abstract: Machine Unlearning (MU) algorithms have become increasingly critical due to
the imperative adherence to data privacy regulations. The primary objective of
MU is to erase the influence of specific data samples on a given model without
the need to retrain it from scratch. Accordingly, existing methods focus on
maximizing user privacy protection. However, there are different degrees of
privacy regulations for each real-world web-based application. Exploring the
full spectrum of trade-offs between privacy, model utility, and runtime
efficiency is critical for practical unlearning scenarios. Furthermore,
designing the MU algorithm with simple control of the aforementioned trade-off
is desirable but challenging due to the inherent complex interaction. To
address the challenges, we present Controllable Machine Unlearning (ConMU), a
novel framework designed to facilitate the calibration of MU. The ConMU
framework contains three integral modules: an important data selection module
that reconciles the runtime efficiency and model generalization, a progressive
Gaussian mechanism module that balances privacy and model generalization, and
an unlearning proxy that controls the trade-offs between privacy and runtime
efficiency. Comprehensive experiments on various benchmark datasets have
demonstrated the robust adaptability of our control mechanism and its
superiority over established unlearning methods. ConMU explores the full
spectrum of the Privacy-Utility-Efficiency trade-off and allows practitioners
to account for different real-world regulations. Source code available at:
https://github.com/guangyaodou/ConMU.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words
Abstract: Training on multiple modalities of input can augment the capabilities of a
language model. Here, we ask whether such a training regime can improve the
quality and efficiency of these systems as well. We focus on text--audio and
introduce Whisbert, which is inspired by the text--image approach of FLAVA
(Singh et al., 2022). In accordance with Babylm guidelines (Warstadt et al.,
2023), we pretrain Whisbert on a dataset comprising only 100 million words plus
their corresponding speech from the word-aligned version of the People's Speech
dataset (Galvez et al., 2021). To assess the impact of multimodality, we
compare versions of the model that are trained on text only and on both audio
and text simultaneously. We find that while Whisbert is able to perform well on
multimodal masked modeling and surpasses the Babylm baselines in most benchmark
tasks, it struggles to optimize its complex objective and outperform its
text-only Whisbert baseline.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Empirical evaluation of Uncertainty Quantification in Retrieval-Augmented Language Models for Science
Abstract: Large language models (LLMs) have shown remarkable achievements in natural
language processing tasks, producing high-quality outputs. However, LLMs still
exhibit limitations, including the generation of factually incorrect
information. In safety-critical applications, it is important to assess the
confidence of LLM-generated content to make informed decisions. Retrieval
Augmented Language Models (RALMs) is relatively a new area of research in NLP.
RALMs offer potential benefits for scientific NLP tasks, as retrieved
documents, can serve as evidence to support model-generated content. This
inclusion of evidence enhances trustworthiness, as users can verify and explore
the retrieved documents to validate model outputs. Quantifying uncertainty in
RALM generations further improves trustworthiness, with retrieved text and
confidence scores contributing to a comprehensive and reliable model for
scientific applications. However, there is limited to no research on UQ for
RALMs, particularly in scientific contexts. This study aims to address this gap
by conducting a comprehensive evaluation of UQ in RALMs, focusing on scientific
tasks. This research investigates how uncertainty scores vary when scientific
knowledge is incorporated as pretraining and retrieval data and explores the
relationship between uncertainty scores and the accuracy of model-generated
outputs. We observe that an existing RALM finetuned with scientific knowledge
as the retrieval data tends to be more confident in generating predictions
compared to the model pretrained only with scientific knowledge. We also found
that RALMs are overconfident in their predictions, making inaccurate
predictions more confidently than accurate ones. Scientific knowledge provided
either as pretraining or retrieval corpus does not help alleviate this issue.
We released our code, data and dashboards at https://github.com/pnnl/EXPERT2.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Exploring Large Language Models for Human Mobility Prediction under Public Events
Abstract: Public events, such as concerts and sports games, can be major attractors for
large crowds, leading to irregular surges in travel demand. Accurate human
mobility prediction for public events is thus crucial for event planning as
well as traffic or crowd management. While rich textual descriptions about
public events are commonly available from online sources, it is challenging to
encode such information in statistical or machine learning models. Existing
methods are generally limited in incorporating textual information, handling
data sparsity, or providing rationales for their predictions. To address these
challenges, we introduce a framework for human mobility prediction under public
events (LLM-MPE) based on Large Language Models (LLMs), leveraging their
unprecedented ability to process textual data, learn from minimal examples, and
generate human-readable explanations. Specifically, LLM-MPE first transforms
raw, unstructured event descriptions from online sources into a standardized
format, and then segments historical mobility data into regular and
event-related components. A prompting strategy is designed to direct LLMs in
making and rationalizing demand predictions considering historical mobility and
event features. A case study is conducted for Barclays Center in New York City,
based on publicly available event information and taxi trip data. Results show
that LLM-MPE surpasses traditional models, particularly on event days, with
textual data significantly enhancing its accuracy. Furthermore, LLM-MPE offers
interpretable insights into its predictions. Despite the great potential of
LLMs, we also identify key challenges including misinformation and high costs
that remain barriers to their broader adoption in large-scale human mobility
analysis.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
Abstract: Large Language Models (LLMs) are frequently used for multi-faceted language
generation and evaluation tasks that involve satisfying intricate user
constraints or taking into account multiple aspects and criteria. However,
their performance can fall short, due to the model's lack of coherence and
inability to plan and decompose the problem. We propose Branch-Solve-Merge
(BSM), a Large Language Model program (Schlag et al., 2023) for tackling such
challenging natural language tasks. It consists of branch, solve, and merge
modules that are parameterized with specific prompts to the base LLM. These
three modules plan a decomposition of the task into multiple parallel
sub-tasks, independently solve them, and fuse the solutions to the sub-tasks.
We apply our method to the tasks of LLM response evaluation and constrained
text generation and evaluate its effectiveness with multiple LLMs, including
Vicuna, LLaMA-2-chat, and GPT-4. BSM improves the evaluation correctness and
consistency for each LLM by enhancing human-LLM agreement by up to 26%,
reducing length and pairwise position biases by up to 50%, and allowing
LLaMA-2-chat to match or outperform GPT-4 on most domains. On the constraint
story generation task, BSM improves the coherence of the stories while also
improving constraint satisfaction by 12%.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: LiPar: A Lightweight Parallel Learning Model for Practical In-Vehicle Network Intrusion Detection
Abstract: With the development of intelligent transportation systems, vehicles are
exposed to a complex network environment. As the main network of in-vehicle
networks, the controller area network (CAN) has many potential security
hazards, resulting in higher requirements for intrusion detection systems to
ensure safety. Among intrusion detection technologies, methods based on deep
learning work best without prior expert knowledge. However, they all have a
large model size and rely on cloud computing, and are therefore not suitable to
be installed on the in-vehicle network. Therefore, we propose a lightweight
parallel neural network structure, LiPar, to allocate task loads to multiple
electronic control units (ECU). The LiPar model consists of multi-dimensional
branch convolution networks, spatial and temporal feature fusion learning, and
a resource adaptation algorithm. Through experiments, we prove that LiPar has
great detection performance, running efficiency, and lightweight model size,
which can be well adapted to the in-vehicle environment practically and protect
the in-vehicle CAN bus security.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Real-Time Vibration-Based Bearing Fault Diagnosis Under Time-Varying Speed Conditions
Abstract: Detection of rolling-element bearing faults is crucial for implementing
proactive maintenance strategies and for minimizing the economic and
operational consequences of unexpected failures. However, many existing
techniques are developed and tested under strictly controlled conditions,
limiting their adaptability to the diverse and dynamic settings encountered in
practical applications. This paper presents an efficient real-time
convolutional neural network (CNN) for diagnosing multiple bearing faults under
various noise levels and time-varying rotational speeds. Additionally, we
propose a novel Fisher-based spectral separability analysis (SSA) method to
elucidate the effectiveness of the designed CNN model. We conducted experiments
on both healthy bearings and bearings afflicted with inner race, outer race,
and roller ball faults. The experimental results show the superiority of our
model over the current state-of-the-art approach in three folds: it achieves
substantial accuracy gains of up to 15.8%, it is robust to noise with high
performance across various signal-to-noise ratios, and it runs in real-time
with processing durations five times less than acquisition. Additionally, by
using the proposed SSA technique, we offer insights into the model's
performance and underscore its effectiveness in tackling real-world challenges.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: L2T-DLN: Learning to Teach with Dynamic Loss Network
Abstract: With the concept of teaching being introduced to the machine learning
community, a teacher model start using dynamic loss functions to teach the
training of a student model. The dynamic intends to set adaptive loss functions
to different phases of student model learning. In existing works, the teacher
model 1) merely determines the loss function based on the present states of the
student model, i.e., disregards the experience of the teacher; 2) only utilizes
the states of the student model, e.g., training iteration number and
loss/accuracy from training/validation sets, while ignoring the states of the
loss function. In this paper, we first formulate the loss adjustment as a
temporal task by designing a teacher model with memory units, and, therefore,
enables the student learning to be guided by the experience of the teacher
model. Then, with a dynamic loss network, we can additionally use the states of
the loss to assist the teacher learning in enhancing the interactions between
the teacher and the student model. Extensive experiments demonstrate our
approach can enhance student learning and improve the performance of various
deep models on real-world tasks, including classification, objective detection,
and semantic segmentation scenarios.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification
Abstract: Diffusion-based purification defenses leverage diffusion models to remove
crafted perturbations of adversarial examples and achieve state-of-the-art
robustness. Recent studies show that even advanced attacks cannot break such
defenses effectively, since the purification process induces an extremely deep
computational graph which poses the potential problem of gradient obfuscation,
high memory cost, and unbounded randomness. In this paper, we propose a unified
framework DiffAttack to perform effective and efficient attacks against
diffusion-based purification defenses, including both DDPM and score-based
approaches. In particular, we propose a deviated-reconstruction loss at
intermediate diffusion steps to induce inaccurate density gradient estimation
to tackle the problem of vanishing/exploding gradients. We also provide a
segment-wise forwarding-backwarding algorithm, which leads to memory-efficient
gradient backpropagation. We validate the attack effectiveness of DiffAttack
compared with existing adaptive attacks on CIFAR-10 and ImageNet. We show that
DiffAttack decreases the robust accuracy of models compared with SOTA attacks
by over 20% on CIFAR-10 under $\ell_\infty$ attack $(\epsilon=8/255)$, and over
10% on ImageNet under $\ell_\infty$ attack $(\epsilon=4/255)$. We conduct a
series of ablations studies, and we find 1) DiffAttack with the
deviated-reconstruction loss added over uniformly sampled time steps is more
effective than that added over only initial/final steps, and 2) diffusion-based
purification with a moderate diffusion length is more robust under DiffAttack.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Expressivity of ReLU-Networks under Convex Relaxations
Abstract: Convex relaxations are a key component of training and certifying provably
safe neural networks. However, despite substantial progress, a wide and poorly
understood accuracy gap to standard networks remains, raising the question of
whether this is due to fundamental limitations of convex relaxations. Initial
work investigating this question focused on the simple and widely used IBP
relaxation. It revealed that some univariate, convex, continuous piecewise
linear (CPWL) functions cannot be encoded by any ReLU network such that its
IBP-analysis is precise. To explore whether this limitation is shared by more
advanced convex relaxations, we conduct the first in-depth study on the
expressive power of ReLU networks across all commonly used convex relaxations.
We show that: (i) more advanced relaxations allow a larger class of univariate
functions to be expressed as precisely analyzable ReLU networks, (ii) more
precise relaxations can allow exponentially larger solution spaces of ReLU
networks encoding the same functions, and (iii) even using the most precise
single-neuron relaxations, it is impossible to construct precisely analyzable
ReLU networks that express multivariate, convex, monotone CPWL functions.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: An HCAI Methodological Framework: Putting It Into Action to Enable Human-Centered AI
Abstract: Human-centered AI (HCAI), as a design philosophy, advocates prioritizing
humans in designing, developing, and deploying intelligent systems, aiming to
maximize the benefits of AI technology to humans and avoid its potential
adverse effects. While HCAI has gained momentum, the lack of guidance on
methodology in its implementation makes its adoption challenging. After
assessing the needs for a methodological framework for HCAI, this paper first
proposes a comprehensive and interdisciplinary HCAI methodological framework
integrated with seven components, including design goals, design principles,
implementation approaches, design paradigms, interdisciplinary teams, methods,
and processes. THe implications of the framework are also discussed. This paper
also presents a "three-layer" approach to facilitate the implementation of the
framework. We believe the proposed framework is systematic and executable,
which can overcome the weaknesses in current frameworks and the challenges
currently faced in implementing HCAI. Thus, the framework can help put it into
action to develop, transfer, and implement HCAI in practice, eventually
enabling the design, development, and deployment of HCAI-based intelligent
systems.

Output: Human-Computer Interaction