Instruction: What field is the article from?

Input:
Title: Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimation
Abstract: Although 3D human pose estimation has gained impressive development in recent
years, only a few works focus on infants, that have different bone lengths and
also have limited data. Directly applying adult pose estimation models
typically achieves low performance in the infant domain and suffers from
out-of-distribution issues. Moreover, the limitation of infant pose data
collection also heavily constrains the efficiency of learning-based models to
lift 2D poses to 3D. To deal with the issues of small datasets, domain
adaptation and data augmentation are commonly used techniques. Following this
paradigm, we take advantage of an optimization-based method that utilizes
generative priors to predict 3D infant keypoints from 2D keypoints without the
need of large training data. We further apply a guided diffusion model to
domain adapt 3D adult pose to infant pose to supplement small datasets.
Besides, we also prove that our method, ZeDO-i, could attain efficient domain
adaptation, even if only a small number of data is given. Quantitatively, we
claim that our model attains state-of-the-art MPJPE performance of 43.6 mm on
the SyRIP dataset and 21.2 mm on the MINI-RGBD dataset.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Learning Dynamic Selection and Pricing of Out-of-Home Deliveries
Abstract: Home delivery failures, traffic congestion, and relatively large handling
times have a negative impact on the profitability of last-mile logistics. These
external factors contribute to up to $28\%$ of the overall costs and $25\%$ of
emissions for the home delivery supply chain. A potential solution, showing
annual growth rates up to $36\%$, is the delivery to parcel lockers or parcel
shops, denoted by out-of-home (OOH) delivery. In the academic literature,
models of customer behavior with respect to OOH delivery were so far limited to
deterministic settings, contrasting with the stochastic nature of actual
customer choices. We model the sequential decision-making problem of which OOH
location to offer against what incentive for each incoming customer, taking
into account future customer arrivals and choices. We propose Dynamic Selection
and Pricing of OOH (DSPO), an algorithmic pipeline that uses a novel
spatial-temporal state encoding as input to a convolutional neural network. We
demonstrate the performance of our method by benchmarking it against three
state-of-the-art approaches. Our extensive numerical study, guided by
real-world data, reveals that DSPO can save $20.8\%$ in costs compared to a
situation without OOH locations, $8.1\%$ compared to a static selection and
pricing policy, and $4.6\%$ compared to a state-of-the-art demand management
benchmark. We provide comprehensive insights into the complex interplay between
OOH delivery dynamics and customer behavior influenced by pricing strategies.
The implications of our findings suggest practitioners to adopt dynamic
selection and pricing policies as OOH delivery gains a larger market share.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: RIDE: Real-time Intrusion Detection via Explainable Machine Learning Implemented in a Memristor Hardware Architecture
Abstract: Deep Learning (DL) based methods have shown great promise in network
intrusion detection by identifying malicious network traffic behavior patterns
with high accuracy, but their applications to real-time, packet-level
detections in high-speed communication networks are challenging due to the high
computation time and resource requirements of Deep Neural Networks (DNNs), as
well as lack of explainability. To this end, we propose a packet-level network
intrusion detection solution that makes novel use of Recurrent Autoencoders to
integrate an arbitrary-length sequence of packets into a more compact joint
feature embedding, which is fed into a DNN-based classifier. To enable
explainability and support real-time detections at micro-second speed, we
further develop a Software-Hardware Co-Design approach to efficiently realize
the proposed solution by converting the learned detection policies into
decision trees and implementing them using an emerging architecture based on
memristor devices. By jointly optimizing associated software and hardware
constraints, we show that our approach leads to an extremely efficient,
real-time solution with high detection accuracy at the packet level. Evaluation
results on real-world datasets (e.g., UNSW and CIC-IDS datasets) demonstrate
nearly three-nines detection accuracy with a substantial speedup of nearly four
orders of magnitude.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Efficient IoT Inference via Context-Awareness
Abstract: While existing strategies to execute deep learning-based classification on
low-power platforms assume the models are trained on all classes of interest,
this paper posits that adopting context-awareness i.e. narrowing down a
classification task to the current deployment context consisting of only recent
inference queries can substantially enhance performance in resource-constrained
environments. We propose a new paradigm, CACTUS, for scalable and efficient
context-aware classification where a micro-classifier recognizes a small set of
classes relevant to the current context and, when context change happens (e.g.,
a new class comes into the scene), rapidly switches to another suitable
micro-classifier. CACTUS features several innovations, including optimizing the
training cost of context-aware classifiers, enabling on-the-fly context-aware
switching between classifiers, and balancing context switching costs and
performance gains via simple yet effective switching policies. We show that
CACTUS achieves significant benefits in accuracy, latency, and compute budget
across a range of datasets and IoT platforms.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)
Abstract: Inspired by the success of contrastive learning, we systematically examine
recommendation losses, including listwise (softmax), pairwise (BPR), and
pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+, an
optimized generalization of InfoNCE with balance coefficients, and highlight
its performance advantages, particularly when aligned with our new decoupled
contrastive loss, MINE+. We also leverage debiased InfoNCE to debias pointwise
recommendation loss (CCL) as Debiased CCL. Interestingly, our analysis reveals
that linear models like iALS and EASE are inherently debiased. Empirical
results demonstrates the effectiveness of MINE+ and Debiased-CCL.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions
Abstract: Theory of mind (ToM) evaluations currently focus on testing models using
passive narratives that inherently lack interactivity. We introduce FANToM, a
new benchmark designed to stress-test ToM within information-asymmetric
conversational contexts via question answering. Our benchmark draws upon
important theoretical requisites from psychology and necessary empirical
considerations when evaluating large language models (LLMs). In particular, we
formulate multiple types of questions that demand the same underlying reasoning
to identify illusory or false sense of ToM capabilities in LLMs. We show that
FANToM is challenging for state-of-the-art LLMs, which perform significantly
worse than humans even with chain-of-thought reasoning or fine-tuning.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Mini but Mighty: Finetuning ViTs with Mini Adapters
Abstract: Vision Transformers (ViTs) have become one of the dominant architectures in
computer vision, and pre-trained ViT models are commonly adapted to new tasks
via fine-tuning. Recent works proposed several parameter-efficient transfer
learning methods, such as adapters, to avoid the prohibitive training and
storage cost of finetuning. In this work, we observe that adapters perform
poorly when the dimension of adapters is small, and we propose MiMi, a training
framework that addresses this issue. We start with large adapters which can
reach high performance, and iteratively reduce their size. To enable automatic
estimation of the hidden dimension of every adapter, we also introduce a new
scoring function, specifically designed for adapters, that compares the neuron
importance across layers. Our method outperforms existing methods in finding
the best trade-off between accuracy and trained parameters across the three
dataset benchmarks DomainNet, VTAB, and Multi-task, for a total of 29 datasets.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: SkyMath: Technical Report
Abstract: Large language models (LLMs) have shown great potential to solve varieties of
natural language processing (NLP) tasks, including mathematical reasoning. In
this work, we present SkyMath, a large language model for mathematics with 13
billion parameters. By applying self-compare fine-tuning, we have enhanced
mathematical reasoning abilities of Skywork-13B-Base remarkably. On GSM8K,
SkyMath outperforms all known open-source models of similar size and has
established a new SOTA performance.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Optimizing and Fine-tuning Large Language Model for Urban Renewal
Abstract: This study aims to innovatively explore adaptive applications of large
language models (LLM) in urban renewal. It also aims to improve its performance
and text generation quality for knowledge question-answering (QA) tasks. Based
on the ChatGLM, we automatically generate QA datasets using urban renewal
scientific literature corpora in a self-instruct manner and then conduct joint
fine-tuning training on the model using the Prefix and LoRA fine-tuning methods
to create an LLM for urban renewal. By guiding the LLM to automatically
generate QA data based on prompt words and given text, it is possible to
quickly obtain datasets in the urban renewal field and provide data support for
the fine-tuning training of LLMs. The experimental results show that the joint
fine-tuning training method proposed in this study can significantly improve
the performance of LLM on the QA tasks. Compared with LoRA fine-tuning, the
method improves the Bleu and Rouge metrics on the test by about 5%; compared
with the model before fine-tuning, the method improves the Bleu and Rouge
metrics by about 15%-20%. This study demonstrates the effectiveness and
superiority of the joint fine-tuning method using Prefix and LoRA for ChatGLM
in the urban renewal knowledge QA tasks. It provides a new approach for
fine-tuning LLMs on urban renewal-related tasks.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Towards an Automatic AI Agent for Reaction Condition Recommendation in Chemical Synthesis
Abstract: Artificial intelligence (AI) for reaction condition optimization has become
an important topic in the pharmaceutical industry, given that a data-driven AI
model can assist drug discovery and accelerate reaction design. However,
existing AI models lack the chemical insights and real-time knowledge
acquisition abilities of experienced human chemists. This paper proposes a
Large Language Model (LLM) empowered AI agent to bridge this gap. We put forth
a novel three-phase paradigm and applied advanced intelligence-enhancement
methods like in-context learning and multi-LLM debate so that the AI agent can
borrow human insight and update its knowledge by searching the latest chemical
literature. Additionally, we introduce a novel Coarse-label Contrastive
Learning (CCL) based chemical fingerprint that greatly enhances the agent's
performance in optimizing the reaction condition. With the above efforts, the
proposed AI agent can autonomously generate the optimal reaction condition
recommendation without any human interaction. Further, the agent is highly
professional in terms of chemical reactions. It demonstrates close-to-human
performance and strong generalization capability in both dry-lab and wet-lab
experiments. As the first attempt in the chemical AI agent, this work goes a
step further in the field of "AI for chemistry" and opens up new possibilities
for computer-aided synthesis planning.

Output: Information Retrieval


Instruction: What field is the article from?

Input:
Title: Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards
Abstract: Community Question-Answering (CQA) portals serve as a valuable tool for
helping users within an organization. However, making them accessible to
non-English-speaking users continues to be a challenge. Translating questions
can broaden the community's reach, benefiting individuals with similar
inquiries in various languages. Translating questions using Neural Machine
Translation (NMT) poses more challenges, especially in noisy environments,
where the grammatical correctness of the questions is not monitored. These
questions may be phrased as statements by non-native speakers, with incorrect
subject-verb order and sometimes even missing question marks. Creating a
synthetic parallel corpus from such data is also difficult due to its noisy
nature. To address this issue, we propose a training methodology that
fine-tunes the NMT system only using source-side data. Our approach balances
adequacy and fluency by utilizing a loss function that combines BERTScore and
Masked Language Model (MLM) Score. Our method surpasses the conventional
Maximum Likelihood Estimation (MLE) based fine-tuning approach, which relies on
synthetic target data, by achieving a 1.9 BLEU score improvement. Our model
exhibits robustness while we add noise to our baseline, and still achieve 1.1
BLEU improvement and large improvements on TER and BLEURT metrics. Our proposed
methodology is model-agnostic and is only necessary during the training phase.
We make the codes and datasets publicly available at
\url{https://www.iitp.ac.in/~ai-nlp-ml/resources.html#DomainAdapt} for
facilitating further research.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: One Style is All you Need to Generate a Video
Abstract: In this paper, we propose a style-based conditional video generative model.
We introduce a novel temporal generator based on a set of learned sinusoidal
bases. Our method learns dynamic representations of various actions that are
independent of image content and can be transferred between different actors.
Beyond the significant enhancement of video quality compared to prevalent
methods, we demonstrate that the disentangled dynamic and content permit their
independent manipulation, as well as temporal GAN-inversion to retrieve and
transfer a video motion from one content or identity to another without further
preprocessing such as landmark points.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Deep Learning in Computed Tomography Pulmonary Angiography Imaging: A Dual-Pronged Approach for Pulmonary Embolism Detection
Abstract: The increasing reliance on Computed Tomography Pulmonary Angiography for
Pulmonary Embolism (PE) diagnosis presents challenges and a pressing need for
improved diagnostic solutions. The primary objective of this study is to
leverage deep learning techniques to enhance the Computer Assisted Diagnosis of
PE. In this study, we propose a classifier-guided detection approach that
effectively leverages the classifier's probabilistic inference to direct the
detection predictions, marking a novel contribution in the domain of automated
PE diagnosis. Our end-to-end classification framework introduces an
Attention-Guided Convolutional Neural Network (AG-CNN) that leverages local
context by utilizing an attention mechanism. This approach emulates the
attention of a human expert by looking at both global appearances and local
lesion regions before forming a conclusive decision. The classifier achieves a
notable AUROC, sensitivity, specificity and F1-score of 0.927, 0.862, 0.879 and
0.805 respectively on the FUMPE dataset with Inception-v3 backbone
architecture. Moreover, AG-CNN outperforms the baseline DenseNet-121 model,
achieving an 8.1% AUROC gain. While prior studies have primarily focused on PE
detection in main arteries, our utilization of state-of-the-art object
detection models and ensembling techniques significantly enhances detection
accuracy for small embolisms in the peripheral arteries. Finally, our proposed
classifier-guided detection approach further refines the detection metrics
contributing new state-of-the-art to the community: mAP$_{50}$, sensitivity and
F1-score of 0.846, 0.901 and 0.779 respectively outperforming the former
benchmark with a significant 3.7% improvement in mAP$_{50}$. Our research aims
to elevate PE patient care by integrating AI solutions into clinical workflows,
highlighting the potential of human-AI collaboration in medical diagnostics.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Conditional Unscented Autoencoders for Trajectory Prediction
Abstract: The \ac{CVAE} is one of the most widely-used models in trajectory prediction
for \ac{AD}. It captures the interplay between a driving context and its
ground-truth future into a probabilistic latent space and uses it to produce
predictions. In this paper, we challenge key components of the CVAE. We
leverage recent advances in the space of the VAE, the foundation of the CVAE,
which show that a simple change in the sampling procedure can greatly benefit
performance. We find that unscented sampling, which draws samples from any
learned distribution in a deterministic manner, can naturally be better suited
to trajectory prediction than potentially dangerous random sampling. We go
further and offer additional improvements, including a more structured mixture
latent space, as well as a novel, potentially more expressive way to do
inference with CVAEs. We show wide applicability of our models by evaluating
them on the INTERACTION prediction dataset, outperforming the state of the art,
as well as at the task of image modeling on the CelebA dataset, outperforming
the baseline vanilla CVAE. Code is available at
https://github.com/boschresearch/cuae-prediction.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: Continuous Training and Fine-tuning for Domain-Specific Language Models in Medical Question Answering
Abstract: Large language models exhibit promising general capabilities but often lack
specialized knowledge for domain-specific tasks. Developing domain experts from
a base model enables a range of applications without prohibitive training
costs. This work demonstrates a method using continuous training and
instruction fine-tuning to rapidly adapt Llama 2 base models to the Chinese
medical domain. We first conduct continuous training on 1B tokens from Chinese
medical references to teach relevant vocabulary and knowledge. The models are
then fine-tuned on 54K examples sourced from the Chinese National Medical
Licensing Examination. Experiments on Chinese medical data confirm the
effectiveness of this approach, producing a model comparable to GPT-3.5-turbo
while using way less computational resource. The resulting domain-specific
model could be useful for various Chinese medical applications. More broadly,
this provides a template for domain-specific training of large language models
in areas where pre-trained models lack the required expertise, such as law,
science, and engineering.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Combining Behaviors with the Successor Features Keyboard
Abstract: The Option Keyboard (OK) was recently proposed as a method for transferring
behavioral knowledge across tasks. OK transfers knowledge by adaptively
combining subsets of known behaviors using Successor Features (SFs) and
Generalized Policy Improvement (GPI). However, it relies on hand-designed
state-features and task encodings which are cumbersome to design for every new
environment. In this work, we propose the "Successor Features Keyboard" (SFK),
which enables transfer with discovered state-features and task encodings. To
enable discovery, we propose the "Categorical Successor Feature Approximator"
(CSFA), a novel learning algorithm for estimating SFs while jointly discovering
state-features and task encodings. With SFK and CSFA, we achieve the first
demonstration of transfer with SFs in a challenging 3D environment where all
the necessary representations are discovered. We first compare CSFA against
other methods for approximating SFs and show that only CSFA discovers
representations compatible with SF&GPI at this scale. We then compare SFK
against transfer learning baselines and show that it transfers most quickly to
long-horizon tasks.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: A General Framework for Robust G-Invariance in G-Equivariant Networks
Abstract: We introduce a general method for achieving robust group-invariance in
group-equivariant convolutional neural networks ($G$-CNNs), which we call the
$G$-triple-correlation ($G$-TC) layer. The approach leverages the theory of the
triple-correlation on groups, which is the unique, lowest-degree polynomial
invariant map that is also complete. Many commonly used invariant maps - such
as the max - are incomplete: they remove both group and signal structure. A
complete invariant, by contrast, removes only the variation due to the actions
of the group, while preserving all information about the structure of the
signal. The completeness of the triple correlation endows the $G$-TC layer with
strong robustness, which can be observed in its resistance to invariance-based
adversarial attacks. In addition, we observe that it yields measurable
improvements in classification accuracy over standard Max $G$-Pooling in
$G$-CNN architectures. We provide a general and efficient implementation of the
method for any discretized group, which requires only a table defining the
group's product structure. We demonstrate the benefits of this method for
$G$-CNNs defined on both commutative and non-commutative groups - $SO(2)$,
$O(2)$, $SO(3)$, and $O(3)$ (discretized as the cyclic $C8$, dihedral $D16$,
chiral octahedral $O$ and full octahedral $O_h$ groups) - acting on
$\mathbb{R}^2$ and $\mathbb{R}^3$ on both $G$-MNIST and $G$-ModelNet10
datasets.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: In-Context Learning Dynamics with Random Binary Sequences
Abstract: Large language models (LLMs) trained on huge corpora of text datasets
demonstrate intriguing capabilities, achieving state-of-the-art performance on
tasks they were not explicitly trained for. The precise nature of LLM
capabilities is often mysterious, and different prompts can elicit different
capabilities through in-context learning. We propose a framework that enables
us to analyze in-context learning dynamics to understand latent concepts
underlying LLMs' behavioral patterns. This provides a more nuanced
understanding than success-or-failure evaluation benchmarks, but does not
require observing internal activations as a mechanistic interpretation of
circuits would. Inspired by the cognitive science of human randomness
perception, we use random binary sequences as context and study dynamics of
in-context learning by manipulating properties of context data, such as
sequence length. In the latest GPT-3.5+ models, we find emergent abilities to
generate seemingly random numbers and learn basic formal languages, with
striking in-context learning dynamics where model outputs transition sharply
from seemingly random behaviors to deterministic repetition.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Safer-Instruct: Aligning Language Models with Automated Preference Data
Abstract: Reinforcement Learning from Human Feedback (RLHF) is a vital strategy for
enhancing model safety in language models. However, annotating preference data
for RLHF is a resource-intensive and creativity-demanding process, while
automatic generation methods face limitations in data diversity and quality. In
response, we present Safer-Instruct, a novel pipeline for semi-automatically
constructing large-scale preference datasets. Our approach leverages reversed
instruction tuning, instruction induction, and expert model evaluation to
efficiently generate high-quality preference data without human annotators. We
evaluate Safer-Instruct using LLaMA for instruction induction and GPT-4 as an
expert model, generating approximately 10K preference samples. Finetuning an
Alpaca model on this dataset demonstrates improved harmlessness while
maintaining competitive performance on conversation and downstream tasks.
Safer-Instruct addresses the challenges in preference data acquisition,
advancing the development of safer and more responsible AI systems. Our code
and data are available at https://github.com/uscnlp-lime/safer-instruct

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Deep Unlearning: Fast and Efficient Training-free Approach to Controlled Forgetting
Abstract: Machine unlearning has emerged as a prominent and challenging area of
interest, driven in large part by the rising regulatory demands for industries
to delete user data upon request and the heightened awareness of privacy.
Existing approaches either retrain models from scratch or use several
finetuning steps for every deletion request, often constrained by computational
resource limitations and restricted access to the original training data. In
this work, we introduce a novel class unlearning algorithm designed to
strategically eliminate an entire class or a group of classes from the learned
model. To that end, our algorithm first estimates the Retain Space and the
Forget Space, representing the feature or activation spaces for samples from
classes to be retained and unlearned, respectively. To obtain these spaces, we
propose a novel singular value decomposition-based technique that requires
layer wise collection of network activations from a few forward passes through
the network. We then compute the shared information between these spaces and
remove it from the forget space to isolate class-discriminatory feature space
for unlearning. Finally, we project the model weights in the orthogonal
direction of the class-discriminatory space to obtain the unlearned model. We
demonstrate our algorithm's efficacy on ImageNet using a Vision Transformer
with only $\sim$1.5% drop in retain accuracy compared to the original model
while maintaining under 1% accuracy on the unlearned class samples. Further,
our algorithm consistently performs well when subject to Membership Inference
Attacks showing 7.8% improvement on average across a variety of image
classification datasets and network architectures, as compared to other
baselines while being $\sim$6x more computationally efficient.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Open Datasheets: Machine-readable Documentation for Open Datasets and Responsible AI Assessments
Abstract: This paper introduces a no-code, machine-readable documentation framework for
open datasets, with a focus on Responsible AI (RAI) considerations. The
framework aims to improve the accessibility, comprehensibility, and usability
of open datasets, facilitating easier discovery and use, better understanding
of content and context, and evaluation of dataset quality and accuracy. The
proposed framework is designed to streamline the evaluation of datasets,
helping researchers, data scientists, and other open data users quickly
identify datasets that meet their needs and/or organizational policies or
regulations. The paper also discusses the implementation of the framework and
provides recommendations to maximize its potential. The framework is expected
to enhance the quality and reliability of data used in research and
decision-making, fostering the development of more responsible and trustworthy
AI systems.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Just-in-Time Security Patch Detection -- LLM At the Rescue for Data Augmentation
Abstract: In the face of growing vulnerabilities found in open-source software, the
need to identify {discreet} security patches has become paramount. The lack of
consistency in how software providers handle maintenance often leads to the
release of security patches without comprehensive advisories, leaving users
vulnerable to unaddressed security risks. To address this pressing issue, we
introduce a novel security patch detection system, LLMDA, which capitalizes on
Large Language Models (LLMs) and code-text alignment methodologies for patch
review, data enhancement, and feature combination. Within LLMDA, we initially
utilize LLMs for examining patches and expanding data of PatchDB and SPI-DB,
two security patch datasets from recent literature. We then use labeled
instructions to direct our LLMDA, differentiating patches based on security
relevance. Following this, we apply a PTFormer to merge patches with code,
formulating hybrid attributes that encompass both the innate details and the
interconnections between the patches and the code. This distinctive combination
method allows our system to capture more insights from the combined context of
patches and code, hence improving detection precision. Finally, we devise a
probabilistic batch contrastive learning mechanism within batches to augment
the capability of the our LLMDA in discerning security patches. The results
reveal that LLMDA significantly surpasses the start of the art techniques in
detecting security patches, underscoring its promise in fortifying software
maintenance.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Adaptive Shortcut Debiasing for Online Continual Learning
Abstract: We propose a novel framework DropTop that suppresses the shortcut bias in
online continual learning (OCL) while being adaptive to the varying degree of
the shortcut bias incurred by continuously changing environment. By the
observed high-attention property of the shortcut bias, highly-activated
features are considered candidates for debiasing. More importantly, resolving
the limitation of the online environment where prior knowledge and auxiliary
data are not ready, two novel techniques -- feature map fusion and adaptive
intensity shifting -- enable us to automatically determine the appropriate
level and proportion of the candidate shortcut features to be dropped.
Extensive experiments on five benchmark datasets demonstrate that, when
combined with various OCL algorithms, DropTop increases the average accuracy by
up to 10.4% and decreases the forgetting by up to 63.2%.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Contextual Confidence and Generative AI
Abstract: Generative AI models perturb the foundations of effective human
communication. They present new challenges to contextual confidence, disrupting
participants' ability to identify the authentic context of communication and
their ability to protect communication from reuse and recombination outside its
intended context. In this paper, we describe strategies--tools, technologies
and policies--that aim to stabilize communication in the face of these
challenges. The strategies we discuss fall into two broad categories.
Containment strategies aim to reassert context in environments where it is
currently threatened--a reaction to the context-free expectations and norms
established by the internet. Mobilization strategies, by contrast, view the
rise of generative AI as an opportunity to proactively set new and higher
expectations around privacy and authenticity in mediated communication.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Distribution of Action Movements (DAM): A Descriptor for Human Action Recognition
Abstract: Human action recognition from skeletal data is an important and active area
of research in which the state of the art has not yet achieved near-perfect
accuracy on many well-known datasets. In this paper, we introduce the
Distribution of Action Movements Descriptor, a novel action descriptor based on
the distribution of the directions of the motions of the joints between frames,
over the set of all possible motions in the dataset. The descriptor is computed
as a normalized histogram over a set of representative directions of the
joints, which are in turn obtained via clustering. While the descriptor is
global in the sense that it represents the overall distribution of movement
directions of an action, it is able to partially retain its temporal structure
by applying a windowing scheme.
  The descriptor, together with a standard classifier, outperforms several
state-of-the-art techniques on many well-known datasets.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Regional Correlation Aided Mobile Traffic Prediction with Spatiotemporal Deep Learning
Abstract: Mobile traffic data in urban regions shows differentiated patterns during
different hours of the day. The exploitation of these patterns enables highly
accurate mobile traffic prediction for proactive network management. However,
recent Deep Learning (DL) driven studies have only exploited spatiotemporal
features and have ignored the geographical correlations, causing high
complexity and erroneous mobile traffic predictions. This paper addresses these
limitations by proposing an enhanced mobile traffic prediction scheme that
combines the clustering strategy of daily mobile traffic peak time and novel
multi Temporal Convolutional Network with a Long Short Term Memory (multi
TCN-LSTM) model. The mobile network cells that exhibit peak traffic during the
same hour of the day are clustered together. Our experiments on large-scale
real-world mobile traffic data show up to 28% performance improvement compared
to state-of-the-art studies, which confirms the efficacy and viability of the
proposed approach.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Explaining the Decisions of Deep Policy Networks for Robotic Manipulations
Abstract: Deep policy networks enable robots to learn behaviors to solve various
real-world complex tasks in an end-to-end fashion. However, they lack
transparency to provide the reasons of actions. Thus, such a black-box model
often results in low reliability and disruptive actions during the deployment
of the robot in practice. To enhance its transparency, it is important to
explain robot behaviors by considering the extent to which each input feature
contributes to determining a given action. In this paper, we present an
explicit analysis of deep policy models through input attribution methods to
explain how and to what extent each input feature affects the decisions of the
robot policy models. To this end, we present two methods for applying input
attribution methods to robot policy networks: (1) we measure the importance
factor of each joint torque to reflect the influence of the motor torque on the
end-effector movement, and (2) we modify a relevance propagation method to
handle negative inputs and outputs in deep policy networks properly. To the
best of our knowledge, this is the first report to identify the dynamic changes
of input attributions of multi-modal sensor inputs in deep policy networks
online for robotic manipulation.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: A Fuzzy Time Series-Based Model Using Particle Swarm Optimization and Weighted Rules
Abstract: During the last decades, a myriad of fuzzy time series models have been
proposed in scientific literature. Among the most accurate models found in
fuzzy time series, the high-order ones are the most accurate. The research
described in this paper tackles three potential limitations associated with the
application of high-order fuzzy time series models. To begin with, the adequacy
of forecast rules lacks consistency. Secondly, as the model's order increases,
data utilization diminishes. Thirdly, the uniformity of forecast rules proves
to be highly contingent on the chosen interval partitions. To address these
likely drawbacks, we introduce a novel model based on fuzzy time series that
amalgamates the principles of particle swarm optimization (PSO) and weighted
summation. Our results show that our approach models accurately the time series
in comparison with previous methods.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Neural Network Pruning by Gradient Descent
Abstract: The rapid increase in the parameters of deep learning models has led to
significant costs, challenging computational efficiency and model
interpretability. In this paper, we introduce a novel and straightforward
neural network pruning framework that incorporates the Gumbel-Softmax
technique. This framework enables the simultaneous optimization of a network's
weights and topology in an end-to-end process using stochastic gradient
descent. Empirical results demonstrate its exceptional compression capability,
maintaining high accuracy on the MNIST dataset with only 0.15\% of the original
network parameters. Moreover, our framework enhances neural network
interpretability, not only by allowing easy extraction of feature importance
directly from the pruned network but also by enabling visualization of feature
symmetry and the pathways of information propagation from features to outcomes.
Although the pruning strategy is learned through deep learning, it is
surprisingly intuitive and understandable, focusing on selecting key
representative features and exploiting data patterns to achieve extreme sparse
pruning. We believe our method opens a promising new avenue for deep learning
pruning and the creation of interpretable machine learning systems.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months
Abstract: We analyze VeLO (versatile learned optimizer), the largest scale attempt to
train a general purpose "foundational" optimizer to date. VeLO was trained on
thousands of machine learning tasks using over 4000 TPU months with the goal of
producing an optimizer capable of generalizing to new problems while being
hyperparameter free, and outperforming industry standards such as Adam. We
independently evaluate VeLO on the MLCommons optimizer benchmark suite. We find
that, contrary to initial claims: (1) VeLO has a critical hyperparameter that
needs problem-specific tuning, (2) VeLO does not necessarily outperform
competitors in quality of solution found, and (3) VeLO is not faster than
competing optimizers at reducing the training loss. These observations call
into question VeLO's generality and the value of the investment in training it.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: RMS: Redundancy-Minimizing Point Cloud Sampling for Real-Time Pose Estimation in Degenerated Environments
Abstract: The typical point cloud sampling methods used in state estimation for mobile
robots preserve a high level of point redundancy. The point redundancy slows
down the estimation pipeline and can make real-time estimation drift in
geometrically symmetrical and structureless environments. We propose a novel
point cloud sampling method that is capable of lowering the effects of
geometrical degeneracies by minimizing redundancy within the cloud. The
proposed method is an alternative to the commonly used sparsification methods
that normalize the density of points to comply with the constraints on the
real-time capabilities of a robot. In contrast to density normalization, our
method builds on the fact that linear and planar surfaces contain a high level
of redundancy propagated into iterative estimation pipelines. We define the
concept of gradient flow quantifying the surface underlying a point. We also
show that maximizing the entropy of the gradient flow minimizes point
redundancy for robot ego-motion estimation. We integrate the proposed method
into the point-based KISS-ICP and feature-based LOAM odometry pipelines and
evaluate it experimentally on KITTI, Hilti-Oxford, and custom datasets from
multirotor UAVs. The experiments show that the proposed sampling technique
outperforms state-of-the-art methods in well-conditioned as well as in
geometrically-degenerated settings, in both accuracy and speed.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: In Search of the Long-Tail: Systematic Generation of Long-Tail Knowledge via Logical Rule Guided Search
Abstract: Since large language models have approached human-level performance on many
tasks, it has become increasingly harder for researchers to find tasks that are
still challenging to the models. Failure cases usually come from the long-tail
distribution - data that an oracle language model could assign a probability on
the lower end of its distribution. Current methodology such as prompt
engineering or crowdsourcing are insufficient for creating long-tail examples
because humans are constrained by cognitive bias. We propose a
Logic-Induced-Knowledge-Search (LINK) framework for systematically generating
long-tail knowledge statements. Grounded by a symbolic rule, we search for
long-tail values for each variable of the rule by first prompting a LLM, then
verifying the correctness of the values with a critic, and lastly pushing for
the long-tail distribution with a reranker. With this framework we construct a
dataset, Logic-Induced-Long-Tail (LINT), consisting of 200 symbolic rules and
50K knowledge statements spanning across four domains. Human annotations find
that 84% of the statements in LINT are factually correct. In contrast, ChatGPT
and GPT4 struggle with directly generating long-tail statements under the
guidance of logic rules, each only getting 56% and 78% of their statements
correct. Moreover, their "long-tail" generations in fact fall into the higher
likelihood range, and thus are not really long-tail. Our findings suggest that
LINK is effective for generating data in the long-tail distribution while
enforcing quality. LINT can be useful for systematically evaluating LLMs'
capabilities in the long-tail distribution. We challenge the models with a
simple entailment classification task using samples from LINT. We find that
ChatGPT and GPT4's capability in identifying incorrect knowledge drop by ~3% in
the long-tail distribution compared to head distribution.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: A new fuzzy multi-attribute group decision-making method based on TOPSIS and optimization models
Abstract: In this paper, a new method based on TOPSIS and optimization models is
proposed for multi-attribute group decision-making in the environment of
interval-valued intuitionistic fuzzy sets.Firstly, by minimizing the sum of
differences between individual evaluations and the overallconsistent
evaluations of all experts, a new optimization model is established for
determining expert weights. Secondly, based on TOPSIS method, the improved
closeness index for evaluating each alternative is obtained. Finally, the
attribute weight is determined by establishing an optimization model with the
goal of maximizing the closeness of each alternative, and it is brought into
the closeness index so that the alternatives can be ranked. Combining all these
together, the complete fuzzy multi-attribute group decision-making algorithm is
formulated, which can give full play to the advantages of subjective and
objective weighting methods. In the end, the feasibility and effectiveness of
the provided method are verified by a real case study.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: A Representative Study on Human Detection of Artificially Generated Media Across Countries
Abstract: AI-generated media has become a threat to our digital society as we know it.
These forgeries can be created automatically and on a large scale based on
publicly available technology. Recognizing this challenge, academics and
practitioners have proposed a multitude of automatic detection strategies to
detect such artificial media. However, in contrast to these technical advances,
the human perception of generated media has not been thoroughly studied yet.
  In this paper, we aim at closing this research gap. We perform the first
comprehensive survey into people's ability to detect generated media, spanning
three countries (USA, Germany, and China) with 3,002 participants across audio,
image, and text media. Our results indicate that state-of-the-art forgeries are
almost indistinguishable from "real" media, with the majority of participants
simply guessing when asked to rate them as human- or machine-generated. In
addition, AI-generated media receive is voted more human like across all media
types and all countries. To further understand which factors influence people's
ability to detect generated media, we include personal variables, chosen based
on a literature review in the domains of deepfake and fake news research. In a
regression analysis, we found that generalized trust, cognitive reflection, and
self-reported familiarity with deepfakes significantly influence participant's
decision across all media categories.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: LLamol: A Dynamic Multi-Conditional Generative Transformer for De Novo Molecular Design
Abstract: Generative models have demonstrated substantial promise in Natural Language
Processing (NLP) and have found application in designing molecules, as seen in
General Pretrained Transformer (GPT) models. In our efforts to develop such a
tool for exploring the organic chemical space in search of potentially
electro-active compounds, we present "LLamol", a single novel generative
transformer model based on the LLama 2 architecture, which was trained on a 13M
superset of organic compounds drawn from diverse public sources. To allow for a
maximum flexibility in usage and robustness in view of potentially incomplete
data, we introduce "Stochastic Context Learning" as a new training procedure.
We demonstrate that the resulting model adeptly handles single- and
multi-conditional organic molecule generation with up to four conditions, yet
more are possible. The model generates valid molecular structures in SMILES
notation while flexibly incorporating three numerical and/or one token sequence
into the generative process, just as requested. The generated compounds are
very satisfactory in all scenarios tested. In detail, we showcase the model's
capability to utilize token sequences for conditioning, either individually or
in combination with numerical properties, making LLamol a potent tool for de
novo molecule design, easily expandable with new properties.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Synthesizing Black-box Anti-forensics DeepFakes with High Visual Quality
Abstract: DeepFake, an AI technology for creating facial forgeries, has garnered global
attention. Amid such circumstances, forensics researchers focus on developing
defensive algorithms to counter these threats. In contrast, there are
techniques developed for enhancing the aggressiveness of DeepFake, e.g.,
through anti-forensics attacks, to disrupt forensic detectors. However, such
attacks often sacrifice image visual quality for improved undetectability. To
address this issue, we propose a method to generate novel adversarial
sharpening masks for launching black-box anti-forensics attacks. Unlike many
existing arts, with such perturbations injected, DeepFakes could achieve high
anti-forensics performance while exhibiting pleasant sharpening visual effects.
After experimental evaluations, we prove that the proposed method could
successfully disrupt the state-of-the-art DeepFake detectors. Besides, compared
with the images processed by existing DeepFake anti-forensics methods, the
visual qualities of anti-forensics DeepFakes rendered by the proposed method
are significantly refined.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Cerbero-7B: A Leap Forward in Language-Specific LLMs Through Enhanced Chat Corpus Generation and Evaluation
Abstract: This study introduces a novel approach for generating high-quality,
language-specific chat corpora using a self-chat mechanism. We combine a
generator LLM for creating new samples and an embedder LLM to ensure diversity.
A new Masked Language Modelling (MLM) model-based quality assessment metric is
proposed for evaluating and filtering the corpora. Utilizing the llama2-70b as
the generator and a multilingual sentence transformer as embedder, we generate
an Italian chat corpus and refine the Fauno corpus, which is based on
translated English ChatGPT self-chat data. The refinement uses structural
assertions and Natural Language Processing techniques. Both corpora undergo a
comprehensive quality evaluation using the proposed MLM model-based quality
metric. The Italian LLM fine-tuned with these corpora demonstrates
significantly enhanced language comprehension and question-answering skills.
The resultant model, cerbero-7b, establishes a new state-of-the-art for Italian
LLMs. This approach marks a substantial advancement in the development of
language-specific LLMs, with a special emphasis on augmenting corpora for
underrepresented languages like Italian.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need
Abstract: Vision Transformers have been rapidly uprising in computer vision thanks to
their outstanding scaling trends, and gradually replacing convolutional neural
networks (CNNs). Recent works on self-supervised learning (SSL) introduce
siamese pre-training tasks, on which Transformer backbones continue to
demonstrate ever stronger results than CNNs. People come to believe that
Transformers or self-attention modules are inherently more suitable than CNNs
in the context of SSL. However, it is noteworthy that most if not all prior
arts of SSL with CNNs chose the standard ResNets as their backbones, whose
architecture effectiveness is known to already lag behind advanced Vision
Transformers. Therefore, it remains unclear whether the self-attention
operation is crucial for the recent advances in SSL - or CNNs can deliver the
same excellence with more advanced designs, too? Can we close the SSL
performance gap between Transformers and CNNs? To answer these intriguing
questions, we apply self-supervised pre-training to the recently proposed,
stronger lager-kernel CNN architecture and conduct an apple-to-apple comparison
with Transformers, in their SSL performance. Our results show that we are able
to build pure CNN SSL architectures that perform on par with or better than the
best SSL-trained Transformers, by just scaling up convolutional kernel sizes
besides other small tweaks. Impressively, when transferring to the downstream
tasks \texttt{MS COCO} detection and segmentation, our SSL pre-trained CNN
model (trained in 100 epochs) achieves the same good performance as the
300-epoch pre-trained Transformer counterpart. We hope this work can help to
better understand what is essential (or not) for self-supervised learning
backbones.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives
Abstract: Mental health conversational agents (a.k.a. chatbots) are widely studied for
their potential to offer accessible support to those experiencing mental health
challenges. Previous surveys on the topic primarily consider papers published
in either computer science or medicine, leading to a divide in understanding
and hindering the sharing of beneficial knowledge between both domains. To
bridge this gap, we conduct a comprehensive literature review using the PRISMA
framework, reviewing 534 papers published in both computer science and
medicine. Our systematic review reveals 136 key papers on building mental
health-related conversational agents with diverse characteristics of modeling
and experimental design techniques. We find that computer science papers focus
on LLM techniques and evaluating response quality using automated metrics with
little attention to the application while medical papers use rule-based
conversational agents and outcome metrics to measure the health outcomes of
participants. Based on our findings on transparency, ethics, and cultural
heterogeneity in this review, we provide a few recommendations to help bridge
the disciplinary divide and enable the cross-disciplinary development of mental
health conversational agents.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: How good are Large Language Models on African Languages?
Abstract: Recent advancements in natural language processing have led to the
proliferation of large language models (LLMs). These models have been shown to
yield good performance, using in-context learning, even on unseen tasks and
languages. Additionally, they have been widely adopted as
language-model-as-a-service commercial APIs like GPT-4 API. However, their
performance on African languages is largely unknown. We present an analysis of
three popular large language models (mT0, LLaMa 2, and GPT-4) on five tasks
(news topic classification, sentiment classification, machine translation,
question answering, and named entity recognition) across 30 African languages,
spanning different language families and geographical regions. Our results
suggest that all LLMs produce below-par performance on African languages, and
there is a large gap in performance compared to high-resource languages like
English most tasks. We find that GPT-4 has an average or impressive performance
on classification tasks but very poor results on generative tasks like machine
translation. Surprisingly, we find that mT0 had the best overall on
cross-lingual QA, better than the state-of-the-art supervised model (i.e.
fine-tuned mT5) and GPT-4 on African languages. Overall, LLaMa 2 records the
worst performance due to its limited multilingual capabilities and
English-centric pre-training corpus. In general, our findings present a
call-to-action to ensure African languages are well represented in large
language models, given their growing popularity.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Learning-driven Zero Trust in Distributed Computing Continuum Systems
Abstract: Converging Zero Trust (ZT) with learning techniques can solve various
operational and security challenges in Distributed Computing Continuum Systems
(DCCS). Implementing centralized ZT architecture is seen as unsuitable for the
computing continuum (e.g., computing entities with limited connectivity and
visibility, etc.). At the same time, implementing decentralized ZT in the
computing continuum requires understanding infrastructure limitations and novel
approaches to enhance resource access management decisions. To overcome such
challenges, we present a novel learning-driven ZT conceptual architecture
designed for DCCS. We aim to enhance ZT architecture service quality by
incorporating lightweight learning strategies such as Representation Learning
(ReL) and distributing ZT components across the computing continuum. The ReL
helps to improve the decision-making process by predicting threats or untrusted
requests. Through an illustrative example, we show how the learning process
detects and blocks the requests, enhances resource access control, and reduces
network and computation overheads. Lastly, we discuss the conceptual
architecture, processes, and provide a research agenda.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: AIOps-Driven Enhancement of Log Anomaly Detection in Unsupervised Scenarios
Abstract: Artificial intelligence operations (AIOps) play a pivotal role in
identifying, mitigating, and analyzing anomalous system behaviors and alerts.
However, the research landscape in this field remains limited, leaving
significant gaps unexplored. This study introduces a novel hybrid framework
through an innovative algorithm that incorporates an unsupervised strategy.
This strategy integrates Principal Component Analysis (PCA) and Artificial
Neural Networks (ANNs) and uses a custom loss function to substantially enhance
the effectiveness of log anomaly detection. The proposed approach encompasses
the utilization of both simulated and real-world datasets, including logs from
SockShop and Hadoop Distributed File System (HDFS). The experimental results
are highly promising, demonstrating significant reductions in pseudo-positives.
Moreover, this strategy offers notable advantages, such as the ability to
process logs in their raw, unprocessed form, and the potential for further
enhancements. The successful implementation of this approach showcases a
remarkable reduction in anomalous logs, thus unequivocally establishing the
efficacy of the proposed methodology. Ultimately, this study makes a
substantial contribution to the advancement of log anomaly detection within
AIOps platforms, addressing the critical need for effective and efficient log
analysis in modern and complex systems.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Dynamics Harmonic Analysis of Robotic Systems: Application in Data-Driven Koopman Modelling
Abstract: We introduce the use of harmonic analysis to decompose the state space of
symmetric robotic systems into orthogonal isotypic subspaces. These are
lower-dimensional spaces that capture distinct, symmetric, and synergistic
motions. For linear dynamics, we characterize how this decomposition leads to a
subdivision of the dynamics into independent linear systems on each subspace, a
property we term dynamics harmonic analysis (DHA). To exploit this property, we
use Koopman operator theory to propose an equivariant deep-learning
architecture that leverages the properties of DHA to learn a global linear
model of system dynamics. Our architecture, validated on synthetic systems and
the dynamics of locomotion of a quadrupedal robot, demonstrates enhanced
generalization, sample efficiency, and interpretability, with less trainable
parameters and computational costs.

Output: Robotics


Instruction: What field is the article from?

Input:
Title: Grokking Group Multiplication with Cosets
Abstract: We use the group Fourier transform over the symmetric group $S_n$ to reverse
engineer a 1-layer feedforward network that has "grokked" the multiplication of
$S_5$ and $S_6$. Each model discovers the true subgroup structure of the full
group and converges on circuits that decompose the group multiplication into
the multiplication of the group's conjugate subgroups. We demonstrate the value
of using the symmetries of the data and models to understand their mechanisms
and hold up the ``coset circuit'' that the model uses as a fascinating example
of the way neural networks implement computations. We also draw attention to
current challenges in conducting mechanistic interpretability research by
comparing our work to Chughtai et al. [6] which alleges to find a different
algorithm for this same problem.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: MAST: Model-Agnostic Sparsified Training
Abstract: We introduce a novel optimization problem formulation that departs from the
conventional way of minimizing machine learning model loss as a black-box
function. Unlike traditional formulations, the proposed approach explicitly
incorporates an initially pre-trained model and random sketch operators,
allowing for sparsification of both the model and gradient during training. We
establish insightful properties of the proposed objective function and
highlight its connections to the standard formulation. Furthermore, we present
several variants of the Stochastic Gradient Descent (SGD) method adapted to the
new problem formulation, including SGD with general sampling, a distributed
version, and SGD with variance reduction techniques. We achieve tighter
convergence rates and relax assumptions, bridging the gap between theoretical
principles and practical applications, covering several important techniques
such as Dropout and Sparse training. This work presents promising opportunities
to enhance the theoretical understanding of model training through a
sparsification-aware optimization approach.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Graph Metanetworks for Processing Diverse Neural Architectures
Abstract: Neural networks efficiently encode learned information within their
parameters. Consequently, many tasks can be unified by treating neural networks
themselves as input data. When doing so, recent studies demonstrated the
importance of accounting for the symmetries and geometry of parameter spaces.
However, those works developed architectures tailored to specific networks such
as MLPs and CNNs without normalization layers, and generalizing such
architectures to other types of networks can be challenging. In this work, we
overcome these challenges by building new metanetworks - neural networks that
take weights from other neural networks as input. Put simply, we carefully
build graphs representing the input neural networks and process the graphs
using graph neural networks. Our approach, Graph Metanetworks (GMNs),
generalizes to neural architectures where competing methods struggle, such as
multi-head attention layers, normalization layers, convolutional layers, ResNet
blocks, and group-equivariant linear layers. We prove that GMNs are expressive
and equivariant to parameter permutation symmetries that leave the input neural
network functions unchanged. We validate the effectiveness of our method on
several metanetwork tasks over diverse neural network architectures.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: DiFace: Cross-Modal Face Recognition through Controlled Diffusion
Abstract: Diffusion probabilistic models (DPMs) have exhibited exceptional proficiency
in generating visual media of outstanding quality and realism. Nonetheless,
their potential in non-generative domains, such as face recognition, has yet to
be thoroughly investigated. Meanwhile, despite the extensive development of
multi-modal face recognition methods, their emphasis has predominantly centered
on visual modalities. In this context, face recognition through textual
description presents a unique and promising solution that not only transcends
the limitations from application scenarios but also expands the potential for
research in the field of cross-modal face recognition. It is regrettable that
this avenue remains unexplored and underutilized, a consequence from the
challenges mainly associated with three aspects: 1) the intrinsic imprecision
of verbal descriptions; 2) the significant gaps between texts and images; and
3) the immense hurdle posed by insufficient databases.To tackle this problem,
we present DiFace, a solution that effectively achieves face recognition via
text through a controllable diffusion process, by establishing its theoretical
connection with probability transport. Our approach not only unleashes the
potential of DPMs across a broader spectrum of tasks but also achieves, to the
best of our knowledge, a significant accuracy in text-to-image face recognition
for the first time, as demonstrated by our experiments on verification and
identification.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning
Abstract: For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows
the generative paradigm and learns to reconstruct masked graph edges or node
features. Contrastive Learning (CL) maximizes the similarity between augmented
views of the same graph and is widely used for GSSL. However, MAE and CL are
considered separately in existing works for GSSL. We observe that the MAE and
CL paradigms are complementary and propose the graph contrastive masked
autoencoder (GCMAE) framework to unify them. Specifically, by focusing on local
edges or node features, MAE cannot capture global information of the graph and
is sensitive to particular edges and features. On the contrary, CL excels in
extracting global information because it considers the relation between graphs.
As such, we equip GCMAE with an MAE branch and a CL branch, and the two
branches share a common encoder, which allows the MAE branch to exploit the
global information extracted by the CL branch. To force GCMAE to capture global
graph structures, we train it to reconstruct the entire adjacency matrix
instead of only the masked edges as in existing works. Moreover, a
discrimination loss is proposed for feature reconstruction, which improves the
disparity between node embeddings rather than reducing the reconstruction error
to tackle the feature smoothing problem of MAE. We evaluate GCMAE on four
popular graph tasks (i.e., node classification, node clustering, link
prediction, and graph classification) and compare with 14 state-of-the-art
baselines. The results show that GCMAE consistently provides good accuracy
across these tasks, and the maximum accuracy improvement is up to 3.2% compared
with the best-performing baseline.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Exploring Prompting Large Language Models as Explainable Metrics
Abstract: This paper describes the IUST NLP Lab submission to the Prompting Large
Language Models as Explainable Metrics Shared Task at the Eval4NLP 2023
Workshop on Evaluation & Comparison of NLP Systems. We have proposed a
zero-shot prompt-based strategy for explainable evaluation of the summarization
task using Large Language Models (LLMs). The conducted experiments demonstrate
the promising potential of LLMs as evaluation metrics in Natural Language
Processing (NLP), particularly in the field of summarization. Both few-shot and
zero-shot approaches are employed in these experiments. The performance of our
best provided prompts achieved a Kendall correlation of 0.477 with human
evaluations in the text summarization task on the test data. Code and results
are publicly available on GitHub.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Attention-Based Real-Time Defenses for Physical Adversarial Attacks in Vision Applications
Abstract: Deep neural networks exhibit excellent performance in computer vision tasks,
but their vulnerability to real-world adversarial attacks, achieved through
physical objects that can corrupt their predictions, raises serious security
concerns for their application in safety-critical domains. Existing defense
methods focus on single-frame analysis and are characterized by high
computational costs that limit their applicability in multi-frame scenarios,
where real-time decisions are crucial.
  To address this problem, this paper proposes an efficient attention-based
defense mechanism that exploits adversarial channel-attention to quickly
identify and track malicious objects in shallow network layers and mask their
adversarial effects in a multi-frame setting. This work advances the state of
the art by enhancing existing over-activation techniques for real-world
adversarial attacks to make them usable in real-time applications. It also
introduces an efficient multi-frame defense framework, validating its efficacy
through extensive experiments aimed at evaluating both defense performance and
computational cost.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: TaskMet: Task-Driven Metric Learning for Model Learning
Abstract: Deep learning models are often deployed in downstream tasks that the training
procedure may not be aware of. For example, models solely trained to achieve
accurate predictions may struggle to perform well on downstream tasks because
seemingly small prediction errors may incur drastic task errors. The standard
end-to-end learning approach is to make the task loss differentiable or to
introduce a differentiable surrogate that the model can be trained on. In these
settings, the task loss needs to be carefully balanced with the prediction loss
because they may have conflicting objectives. We propose take the task loss
signal one level deeper than the parameters of the model and use it to learn
the parameters of the loss function the model is trained on, which can be done
by learning a metric in the prediction space. This approach does not alter the
optimal prediction model itself, but rather changes the model learning to
emphasize the information important for the downstream task. This enables us to
achieve the best of both worlds: a prediction model trained in the original
prediction space while also being valuable for the desired downstream task. We
validate our approach through experiments conducted in two main settings: 1)
decision-focused model learning scenarios involving portfolio optimization and
budget allocation, and 2) reinforcement learning in noisy environments with
distracting states. The source code to reproduce our experiments is available
at https://github.com/facebookresearch/taskmet

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Leveraging Generative AI for Clinical Evidence Summarization Needs to Achieve Trustworthiness
Abstract: Evidence-based medicine aims to improve the quality of healthcare by
empowering medical decisions and practices with the best available evidence.
The rapid growth of medical evidence, which can be obtained from various
sources, poses a challenge in collecting, appraising, and synthesizing the
evidential information. Recent advancements in generative AI, exemplified by
large language models, hold promise in facilitating the arduous task. However,
developing accountable, fair, and inclusive models remains a complicated
undertaking. In this perspective, we discuss the trustworthiness of generative
AI in the context of automated summarization of medical evidence.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: A Bayesian approach for prompt optimization in pre-trained language models
Abstract: A prompt is a sequence of symbol or tokens, selected from a vocabulary
according to some rule, which is prepended/concatenated to a textual query. A
key problem is how to select the sequence of tokens: in this paper we formulate
it as a combinatorial optimization problem. The high dimensionality of the
token space com-pounded by the length of the prompt sequence requires a very
efficient solution. In this paper we propose a Bayesian optimization method,
executed in a continuous em-bedding of the combinatorial space. In this paper
we focus on hard prompt tuning (HPT) which directly searches for discrete
tokens to be added to the text input with-out requiring access to the large
language model (LLM) and can be used also when LLM is available only as a
black-box. This is critically important if LLMs are made available in the Model
as a Service (MaaS) manner as in GPT-4. The current manu-script is focused on
the optimization of discrete prompts for classification tasks. The discrete
prompts give rise to difficult combinatorial optimization problem which easily
become intractable given the dimension of the token space in realistic
applications. The optimization method considered in this paper is Bayesian
optimization (BO) which has become the dominant approach in black-box
optimization for its sample efficiency along with its modular structure and
versatility. In this paper we use BoTorch, a library for Bayesian optimization
research built on top of pyTorch. Albeit preliminary and obtained using a
'vanilla' version of BO, the experiments on RoB-ERTa on six benchmarks, show a
good performance across a variety of tasks and enable an analysis of the
tradeoff between size of the search space, accuracy and wall clock time.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Electronic Communication Data Link Encryption Simulation Based on Wireless Communication
Abstract: In order to improve the simulation effect of electronic communication data
link encryption, the author proposes a solution based on wireless
communication. The main content of this technology is based on the research of
wireless communication, improve the elliptic curve cryptographic algorithm to
build a system encryption model, obtain legal and valid node private keys,
evaluate and analyze the relevant security attributes of the system, verify the
security of the keys, and realize the encryption optimization of wireless
network communication. Experimental results show that: Using the improved
elliptic curve to simulate the system data chain encryption under the
certificateless public key cryptosystem in network communication, the time is
only 2.31 milliseconds, which is lower than other algorithms. Conclusion: It is
proved that the technology research based on wireless communication can
effectively improve the encryption simulation effect of electronic
communication data link.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Comparable Demonstrations are Important in In-Context Learning: A Novel Perspective on Demonstration Selection
Abstract: In-Context Learning (ICL) is an important paradigm for adapting Large
Language Models (LLMs) to downstream tasks through a few demonstrations.
Despite the great success of ICL, the limitation of the demonstration number
may lead to demonstration bias, i.e. the input-label mapping induced by LLMs
misunderstands the task's essence. Inspired by human experience, we attempt to
mitigate such bias through the perspective of the inter-demonstration
relationship. Specifically, we construct Comparable Demonstrations (CDs) by
minimally editing the texts to flip the corresponding labels, in order to
highlight the task's essence and eliminate potential spurious correlations
through the inter-demonstration comparison. Through a series of experiments on
CDs, we find that (1) demonstration bias does exist in LLMs, and CDs can
significantly reduce such bias; (2) CDs exhibit good performance in ICL,
especially in out-of-distribution scenarios. In summary, this study explores
the ICL mechanisms from a novel perspective, providing a deeper insight into
the demonstration selection strategy for ICL.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Client Orchestration and Cost-Efficient Joint Optimization for NOMA-Enabled Hierarchical Federated Learning
Abstract: Hierarchical federated learning (HFL) shows great advantages over
conventional two-layer federated learning (FL) in reducing network overhead and
interaction latency while still retaining the data privacy of distributed FL
clients. However, the communication and energy overhead still pose a bottleneck
for HFL performance, especially as the number of clients raises dramatically.
To tackle this issue, we propose a non-orthogonal multiple access (NOMA)
enabled HFL system under semi-synchronous cloud model aggregation in this
paper, aiming to minimize the total cost of time and energy at each HFL global
round. Specifically, we first propose a novel fuzzy logic based client
orchestration policy considering client heterogenerity in multiple aspects,
including channel quality, data quantity and model staleness. Subsequently,
given the fuzzy based client-edge association, a joint edge server scheduling
and resource allocation problem is formulated. Utilizing problem decomposition,
we firstly derive the closed-form solution for the edge server scheduling
subproblem via the penalty dual decomposition (PDD) method. Next, a deep
deterministic policy gradient (DDPG) based algorithm is proposed to tackle the
resource allocation subproblem considering time-varying environments. Finally,
extensive simulations demonstrate that the proposed scheme outperforms the
considered benchmarks regarding HFL performance improvement and total cost
reduction.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Selective Visual Representations Improve Convergence and Generalization for Embodied AI
Abstract: Embodied AI models often employ off the shelf vision backbones like CLIP to
encode their visual observations. Although such general purpose representations
encode rich syntactic and semantic information about the scene, much of this
information is often irrelevant to the specific task at hand. This introduces
noise within the learning process and distracts the agent's focus from
task-relevant visual cues. Inspired by selective attention in humans-the
process through which people filter their perception based on their
experiences, knowledge, and the task at hand-we introduce a parameter-efficient
approach to filter visual stimuli for embodied AI. Our approach induces a
task-conditioned bottleneck using a small learnable codebook module. This
codebook is trained jointly to optimize task reward and acts as a
task-conditioned selective filter over the visual observation. Our experiments
showcase state-of-the-art performance for object goal navigation and object
displacement across 5 benchmarks, ProcTHOR, ArchitecTHOR, RoboTHOR, AI2-iTHOR,
and ManipulaTHOR. The filtered representations produced by the codebook are
also able generalize better and converge faster when adapted to other
simulation environments such as Habitat. Our qualitative analyses show that
agents explore their environments more effectively and their representations
retain task-relevant information like target object recognition while ignoring
superfluous information about other objects. Code and pretrained models are
available at our project website: https://embodied-codebook.github.io.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Creating a Discipline-specific Commons for Infectious Disease Epidemiology
Abstract: Objective: To create a commons for infectious disease (ID) epidemiology in
which epidemiologists, public health officers, data producers, and software
developers can not only share data and software, but receive assistance in
improving their interoperability. Materials and Methods: We represented 586
datasets, 54 software, and 24 data formats in OWL 2 and then used logical
queries to infer potentially interoperable combinations of software and
datasets, as well as statistics about the FAIRness of the collection. We
represented the objects in DATS 2.2 and a software metadata schema of our own
design. We used these representations as the basis for the Content, Search,
FAIR-o-meter, and Workflow pages that constitute the MIDAS Digital Commons.
Results: Interoperability was limited by lack of standardization of input and
output formats of software. When formats existed, they were human-readable
specifications (22/24; 92%); only 3 formats (13%) had machine-readable
specifications. Nevertheless, logical search of a triple store based on named
data formats was able to identify scores of potentially interoperable
combinations of software and datasets. Discussion: We improved the findability
and availability of a sample of software and datasets and developed metrics for
assessing interoperability. The barriers to interoperability included poor
documentation of software input/output formats and little attention to
standardization of most types of data in this field. Conclusion: Centralizing
and formalizing the representation of digital objects within a commons promotes
FAIRness, enables its measurement over time and the identification of
potentially interoperable combinations of data and software.

Output: Software Engineering


Instruction: What field is the article from?

Input:
Title: Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
Abstract: Creating large-scale and well-annotated datasets to train AI algorithms is
crucial for automated tumor detection and localization. However, with limited
resources, it is challenging to determine the best type of annotations when
annotating massive amounts of unlabeled data. To address this issue, we focus
on polyps in colonoscopy videos and pancreatic tumors in abdominal CT scans;
both applications require significant effort and time for pixel-wise annotation
due to the high dimensional nature of the data, involving either temporary or
spatial dimensions. In this paper, we develop a new annotation strategy, termed
Drag&Drop, which simplifies the annotation process to drag and drop. This
annotation strategy is more efficient, particularly for temporal and volumetric
imaging, than other types of weak annotations, such as per-pixel, bounding
boxes, scribbles, ellipses, and points. Furthermore, to exploit our Drag&Drop
annotations, we develop a novel weakly supervised learning method based on the
watershed algorithm. Experimental results show that our method achieves better
detection and localization performance than alternative weak annotations and,
more importantly, achieves similar performance to that trained on detailed
per-pixel annotations. Interestingly, we find that, with limited resources,
allocating weak annotations from a diverse patient population can foster models
more robust to unseen images than allocating per-pixel annotations for a small
set of images. In summary, this research proposes an efficient annotation
strategy for tumor detection and localization that is less accurate than
per-pixel annotations but useful for creating large-scale datasets for
screening tumors in various medical modalities.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Skywork: A More Open Bilingual Foundation Model
Abstract: In this technical report, we present Skywork-13B, a family of large language
models (LLMs) trained on a corpus of over 3.2 trillion tokens drawn from both
English and Chinese texts. This bilingual foundation model is the most
extensively trained and openly published LLMs of comparable size to date. We
introduce a two-stage training methodology using a segmented corpus, targeting
general purpose training and then domain-specific enhancement training,
respectively. We show that our model not only excels on popular benchmarks, but
also achieves \emph{state of the art} performance in Chinese language modeling
on diverse domains. Furthermore, we propose a novel leakage detection method,
demonstrating that test data contamination is a pressing issue warranting
further investigation by the LLM community. To spur future research, we release
Skywork-13B along with checkpoints obtained during intermediate stages of the
training process. We are also releasing part of our SkyPile corpus, a
collection of over 150 billion tokens of web text, which is the largest high
quality open Chinese pre-training corpus to date. We hope Skywork-13B and our
open corpus will serve as a valuable open-source resource to democratize access
to high-quality LLMs.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: The Utility of "Even if..." Semifactual Explanation to Optimise Positive Outcomes
Abstract: When users receive either a positive or negative outcome from an automated
system, Explainable AI (XAI) has almost exclusively focused on how to mutate
negative outcomes into positive ones by crossing a decision boundary using
counterfactuals (e.g., \textit{"If you earn 2k more, we will accept your loan
application"}). Here, we instead focus on \textit{positive} outcomes, and take
the novel step of using XAI to optimise them (e.g., \textit{"Even if you wish
to half your down-payment, we will still accept your loan application"}).
Explanations such as these that employ "even if..." reasoning, and do not cross
a decision boundary, are known as semifactuals. To instantiate semifactuals in
this context, we introduce the concept of \textit{Gain} (i.e., how much a user
stands to benefit from the explanation), and consider the first causal
formalisation of semifactuals. Tests on benchmark datasets show our algorithms
are better at maximising gain compared to prior work, and that causality is
important in the process. Most importantly however, a user study supports our
main hypothesis by showing people find semifactual explanations more useful
than counterfactuals when they receive the positive outcome of a loan
acceptance.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques
Abstract: Recent advances in language models (LMs), have demonstrated significant
efficacy in tasks related to the arts and humanities. While LMs have exhibited
exceptional performance across a wide range of natural language processing
tasks, there are notable challenges associated with their utilization on small
datasets and their ability to replicate more creative human capacities. In this
study, we aim to address these challenges by training a Persian classical
poetry generation model using a transformer architecture on a specialized
dataset with no pretraining. Additionally, we propose a novel decoding method
to enhance coherence and meaningfulness in the generated poetry, effectively
managing the tradeoff between diversity and quality. Furthermore, the results
of our training approach and the proposed decoding method are evaluated through
comprehensive set of automatic and human evaluations and showed its superior
capability to generate coherent and meaningful poetry in compare to other
decoding methods and an existing Persian large language model (LLM).

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Boosting Summarization with Normalizing Flows and Aggressive Training
Abstract: This paper presents FlowSUM, a normalizing flows-based variational
encoder-decoder framework for Transformer-based summarization. Our approach
tackles two primary challenges in variational summarization: insufficient
semantic information in latent representations and posterior collapse during
training. To address these challenges, we employ normalizing flows to enable
flexible latent posterior modeling, and we propose a controlled alternate
aggressive training (CAAT) strategy with an improved gate mechanism.
Experimental results show that FlowSUM significantly enhances the quality of
generated summaries and unleashes the potential for knowledge distillation with
minimal impact on inference time. Furthermore, we investigate the issue of
posterior collapse in normalizing flows and analyze how the summary quality is
affected by the training strategy, gate initialization, and the type and number
of normalizing flows used, offering valuable insights for future research.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Conditional Variational Diffusion Models
Abstract: Inverse problems aim to determine parameters from observations, a crucial
task in engineering and science. Lately, generative models, especially
diffusion models, have gained popularity in this area for their ability to
produce realistic solutions and their good mathematical properties. Despite
their success, an important drawback of diffusion models is their sensitivity
to the choice of variance schedule, which controls the dynamics of the
diffusion process. Fine-tuning this schedule for specific applications is
crucial but time-costly and does not guarantee an optimal result. We propose a
novel approach for learning the schedule as part of the training process. Our
method supports probabilistic conditioning on data, provides high-quality
solutions, and is flexible, proving able to adapt to different applications
with minimum overhead. This approach is tested in two unrelated inverse
problems: super-resolution microscopy and quantitative phase imaging, yielding
comparable or superior results to previous methods and fine-tuned diffusion
models. We conclude that fine-tuning the schedule by experimentation should be
avoided because it can be learned during training in a stable way that yields
better results.

Output: Computer Vision


Instruction: What field is the article from?

Input:
Title: Interpretability in Machine Learning: on the Interplay with Explainability, Predictive Performances and Models
Abstract: Interpretability has recently gained attention in the field of machine
learning, for it is crucial when it comes to high-stakes decisions or
troubleshooting. This abstract concept is hard to grasp and has been
associated, over time, with many labels and preconceived ideas. In this
position paper, in order to clarify some misunderstandings regarding
interpretability, we discuss its relationship with significant concepts in
machine learning: explainability, predictive performances, and machine learning
models. For instance, we challenge the idea that interpretability and
explainability are substitutes to one another, or that a fixed degree of
interpretability can be associated with a given machine learning model.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Uncertainty Propagation through Trained Deep Neural Networks Using Factor Graphs
Abstract: Predictive uncertainty estimation remains a challenging problem precluding
the use of deep neural networks as subsystems within safety-critical
applications. Aleatoric uncertainty is a component of predictive uncertainty
that cannot be reduced through model improvements. Uncertainty propagation
seeks to estimate aleatoric uncertainty by propagating input uncertainties to
network predictions. Existing uncertainty propagation techniques use one-way
information flows, propagating uncertainties layer-by-layer or across the
entire neural network while relying either on sampling or analytical techniques
for propagation. Motivated by the complex information flows within deep neural
networks (e.g. skip connections), we developed and evaluated a novel approach
by posing uncertainty propagation as a non-linear optimization problem using
factor graphs. We observed statistically significant improvements in
performance over prior work when using factor graphs across most of our
experiments that included three datasets and two neural network architectures.
Our implementation balances the benefits of sampling and analytical propagation
techniques, which we believe, is a key factor in achieving performance
improvements.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: Extracting Definienda in Mathematical Scholarly Articles with Transformers
Abstract: We consider automatically identifying the defined term within a mathematical
definition from the text of an academic article. Inspired by the development of
transformer-based natural language processing applications, we pose the problem
as (a) a token-level classification task using fine-tuned pre-trained
transformers; and (b) a question-answering task using a generalist large
language model (GPT). We also propose a rule-based approach to build a labeled
dataset from the LATEX source of papers. Experimental results show that it is
possible to reach high levels of precision and recall using either recent (and
expensive) GPT 4 or simpler pre-trained models fine-tuned on our task.

Output: Artificial Intelligence


Instruction: What field is the article from?

Input:
Title: Real-time Network Intrusion Detection via Decision Transformers
Abstract: Many cybersecurity problems that require real-time decision-making based on
temporal observations can be abstracted as a sequence modeling problem, e.g.,
network intrusion detection from a sequence of arriving packets. Existing
approaches like reinforcement learning may not be suitable for such
cybersecurity decision problems, since the Markovian property may not
necessarily hold and the underlying network states are often not observable. In
this paper, we cast the problem of real-time network intrusion detection as
casual sequence modeling and draw upon the power of the transformer
architecture for real-time decision-making. By conditioning a causal decision
transformer on past trajectories, consisting of the rewards, network packets,
and detection decisions, our proposed framework will generate future detection
decisions to achieve the desired return. It enables decision transformers to be
applied to real-time network intrusion detection, as well as a novel tradeoff
between the accuracy and timeliness of detection. The proposed solution is
evaluated on public network intrusion detection datasets and outperforms
several baseline algorithms using reinforcement learning and sequence modeling,
in terms of detection accuracy and timeliness.

Output: Cryptography and Security


Instruction: What field is the article from?

Input:
Title: Topic Segmentation of Semi-Structured and Unstructured Conversational Datasets using Language Models
Abstract: Breaking down a document or a conversation into multiple contiguous segments
based on its semantic structure is an important and challenging problem in NLP,
which can assist many downstream tasks. However, current works on topic
segmentation often focus on segmentation of structured texts. In this paper, we
comprehensively analyze the generalization capabilities of state-of-the-art
topic segmentation models on unstructured texts. We find that: (a) Current
strategies of pre-training on a large corpus of structured text such as
Wiki-727K do not help in transferability to unstructured conversational data.
(b) Training from scratch with only a relatively small-sized dataset of the
target unstructured domain improves the segmentation results by a significant
margin. We stress-test our proposed Topic Segmentation approach by
experimenting with multiple loss functions, in order to mitigate effects of
imbalance in unstructured conversational datasets. Our empirical evaluation
indicates that Focal Loss function is a robust alternative to Cross-Entropy and
re-weighted Cross-Entropy loss function when segmenting unstructured and
semi-structured chats.

Output: Computational Linguistics


Instruction: What field is the article from?

Input:
Title: Multi-Signal Reconstruction Using Masked Autoencoder From EEG During Polysomnography
Abstract: Polysomnography (PSG) is an indispensable diagnostic tool in sleep medicine,
essential for identifying various sleep disorders. By capturing physiological
signals, including EEG, EOG, EMG, and cardiorespiratory metrics, PSG presents a
patient's sleep architecture. However, its dependency on complex equipment and
expertise confines its use to specialized clinical settings. Addressing these
limitations, our study aims to perform PSG by developing a system that requires
only a single EEG measurement. We propose a novel system capable of
reconstructing multi-signal PSG from a single-channel EEG based on a masked
autoencoder. The masked autoencoder was trained and evaluated using the
Sleep-EDF-20 dataset, with mean squared error as the metric for assessing the
similarity between original and reconstructed signals. The model demonstrated
proficiency in reconstructing multi-signal data. Our results present promise
for the development of more accessible and long-term sleep monitoring systems.
This suggests the expansion of PSG's applicability, enabling its use beyond the
confines of clinics.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation
Abstract: Federated learning (FL) is an emerging paradigm for decentralized training of
machine learning models on distributed clients, without revealing the data to
the central server. The learning scheme may be horizontal, vertical or hybrid
(both vertical and horizontal). Most existing research work with deep neural
network (DNN) modelling is focused on horizontal data distributions, while
vertical and hybrid schemes are much less studied. In this paper, we propose a
generalized algorithm FedEmb, for modelling vertical and hybrid DNN-based
learning. The idea of our algorithm is characterised by higher inference
accuracy, stronger privacy-preserving properties, and lower client-server
communication bandwidth demands as compared with existing work. The
experimental results show that FedEmb is an effective method to tackle both
split feature & subject space decentralized problems, shows 0.3% to 4.2%
inference accuracy improvement with limited privacy revealing for datasets
stored in local clients, and reduces 88.9 % time complexity over vertical
baseline method.

Output: Machine Learning


Instruction: What field is the article from?

Input:
Title: PaperQA: Retrieval-Augmented Generative Agent for Scientific Research
Abstract: Large Language Models (LLMs) generalize well across language tasks, but
suffer from hallucinations and uninterpretability, making it difficult to
assess their accuracy without ground-truth. Retrieval-Augmented Generation
(RAG) models have been proposed to reduce hallucinations and provide provenance
for how an answer was generated. Applying such models to the scientific
literature may enable large-scale, systematic processing of scientific
knowledge. We present PaperQA, a RAG agent for answering questions over the
scientific literature. PaperQA is an agent that performs information retrieval
across full-text scientific articles, assesses the relevance of sources and
passages, and uses RAG to provide answers. Viewing this agent as a question
answering model, we find it exceeds performance of existing LLMs and LLM agents
on current science QA benchmarks. To push the field closer to how humans
perform research on scientific literature, we also introduce LitQA, a more
complex benchmark that requires retrieval and synthesis of information from
full-text scientific papers across the literature. Finally, we demonstrate
PaperQA's matches expert human researchers on LitQA.

Output: Computational Linguistics