We frame the detection problem as a binary classification task: given an excerpt of text, label it as either human-written or machine-generated.

we are interested in how variables such as excerpt length and decoding strategy impact performance on this classification task.


In such a scenario, the exclusivity of the relation between the nodes Donald Trump and US will be much higher than the exclusivity of the relation between the nodes John Smith and US, as in the full knowledge graph, the relation is the president of is much less common than the relation is a citizen of.

the stronger the exclusivity of the relation between the two nodes, the higher their semantic relatedness.


It does not mention how the articles should be structured, how complex should be the concepts used, or how difficult the inferences that reader needs to make in order to understand the text should be.

we can expect Simple English Wikipedia to be lexically and syntactically simpler than the original English Wikipedia, but not necessarily conceptually simpler.


For the efficient collection of dialogue data in various domains, we propose a collection platform using gamification.

we target collecting task-oriented dialogues.


The table indicates large proportion of annotation disagreement between the annotators involves the CONvey tag.

confusing the CONvey and RE-Quest tags is prominent.


Generating diverse question-answer pairs is crucial to the performance of downstream RC models.

diversity of answer spans ensures that various parts of the passage are used, and different question types are generated.


We can see that using synthetic data from any of the four domains significantly improved the performance for SQuAD.

when training the RC model with data from all domains + SQuAD training data (last row), there is a large gain in both EM (3.8) and F1 (2.7).


Several studies have incorporated a human in the term collection process (Godbole et al., 2010;Coden et al., 2012).

dictionaries are built in an interactive process where the human gives feedback to the machine and the machine suggests candidates based on the given feedback (Alba et al., 2017, 2018)


HB enjoys both benefits of coverage by LR and quickness by +Fd(p).

a conventional classifier and our method are complementary; LR becomes favorable when the user prioritizes coverage than quickness, and +Fd(p) becomes favorable when vice versa.


We use our recast datasets to explore how well different common classes of NLI models capture temporal reasoning.

we use three types of models: (i) neural bag of words (NBOW; Iyyer et al., 2015) (ii) InferSent (Conneau et al., 2017), and (iii) RoBERTa (Liu et al., 2019).


To this end, we made several assumptions about the distribution, modeled the distribution accordingly, and validated each assumption by comparing the goodness of each model.

we considered two types of word classes-the semantic class of direct objects of a verb and the semantic class in a thesaurus-and tried to build models that properly estimate how likely it is that a word in the vector space is a member of a given word class.


We assumed that the vectors of the words that do not belong to the target word class can be essential clues to distinguish a possible member of a word class from others.

we explored a model based on the offset between positive and negative instances and discriminative learning-based models to investigate the impact of negative instances.


The results in these tables indicate that the models considering the geometry of the distribution or the existence of subgroups in the word class outperform the centroid-based model (CENT) for both the English and Japanese SP datasets.

a simple Gaussian model (GM) performed the best among the models that only depend on positive instances.


Rather than targeting topicdependent models, we target a subclass of arguments.

we focus on arguments that have been classified by Walton et al. (2008) under the argument from consequences scheme.


However, it is not true for many NLP tasks due to their pragmatic nature where the meaning of the same sentence might differ depending on the context or background knowledge.

for the NLI task, Manning (2006) advocate that annotation tasks should be "natural" for untrained annotators, and the role of NLP should be to model the inferences that humans make in practical settings.


It employed 50 independent annotators for a "graded" textual inference task, yielding a total of roughly 19,840 annotations, and validates that disagreements among the annotations are reproducible signals.

in their work, the labeling schema is modified from 3-way categorical NLI to a graded one, whereas our study keeps the original 3-way labeling schema to facilitate a direct comparison between old labels and new labels, and focuses more on giving an in-depth analysis regarding the relation between the level of disagreements among humans and the state-of-the-art model performance.


Recent work revisiting the entity-mention paradigm (Luo et al., 2004;Webster and Curran, 2014), which seeks to maintain explicit representations only of entities, rather than all their constituent mentions, has shown practical benefits for memory while being competitive with state-of-theart models (Xia et al., 2020).

unlike other approaches to coreference resolution which maintain representations of both mentions and their corresponding entity clusters (Rahman and Ng, 2011;Stoyanov and Eisner, 2012;Clark and Manning, 2015;Wiseman et al., 2016;Lee et al., 2018) , the entity-mention paradigm stores representations only of the entity clusters, which are updated incrementally as coreference predictions are made.


Empirical results on LitBank and OntoNotes show that the model is competitive with an unbounded memory version and outperforms a strong rule-based baseline.

we report state of the art results on LitBank.


Therefore, in this phase, the emulator and compiled knowledge are hard-coded; here the focus is learning the parser.

this sub-goal focuses on extending executable semantic parsing from relatively narrow domains to handle more general literal language on-the-fly, similarly to zero-shot semantic parsing (Givoli and Reichart, 2019).


Other research projects have also built such collection-based tasks before as well.

the NLP decathlon (McCann et al., 2018), from which the name of this paper is inspired, collects together a diverse set of NLP tasks -from sentiment detection to parsing.


To show such grounding helps, we report results with and without grounding on those tasks in Table 4, reporting perplexity.

for Wizard of Wikipedia (knowledge) and Image Chat (images) such grounding has a clear effect.


We use the HotpotQA dataset (Yang et al., 2018), a popular benchmark for multi-hop QA task, for the main evaluation of the SRLGRN

two sub-tasks are included in this dataset: Answer prediction and Supporting facts prediction.


Although BERT achieves relatively better performance, ALBERT architecture has significantly fewer parameters (18x) and is faster (about 1.7x running time) than BERT.

aL-BERT reduces memory consumption by cross-layer parameter sharing, increases the speed, and obtains a satisfactory performance.


In the ablation study, we disassemble CSS and PSS into the basic components to analyze the contribution of each component.

we consider the proposed two message passing mechanisms POT and BLU.


Whereas Sup and UnSup contribute comparably with "5K all" annotated lexicon.

at low annotation level, i.e. ”1K unique”, where Sup BLI does not work well, the participation of UnSup extends the valuable additional lexicon.


In this work, we demonstrate that certain attention heads of a visually grounded language model actively ground elements of language to image regions.

some heads can map entities to image regions, performing the task known as entity grounding.


As such, we propose a novel Hierarchical Variational Auto-Encoder (HVAE) model to tackle sentiment drift problem.

we take sentiment distribution changes as the drifts.


Data is processed from a new time period according to the principle of prequential evaluation, and drift adaption is done in an rebuild way.

if drift does not occur when a new period arrives, labels of data are predicted and then appended to the training set to retrain the classifier.


Memory Augmented Neural Networks Our entity memory layer is closely tied to memory-based neural layers (Weston et al., 2014;Sukhbaatar et al., 2015).

it can be seen as a memory network where memory access is supervised through entity linking, and memory slots each correspond to a learned entity representation.


As discussed in Section 3, there are significant differences between EAE and KNOWBERT other than the choice of entity representation.

kNOWBERT has an explicit entity-entity attention mechanism.


In this paper, we propose a multi-task learning (MTL) framework to effectively utilize monolingual data for MNMT.

the model is jointly trained with translation task on multilingual parallel data and two auxiliary tasks: masked language modeling (MLM) and denoising autoencoding (DAE) on the source-side and target-side monolingual data respectively.


We further present two simple yet effective scheduling strategies for the multilingual and multi-task framework.

we introduce a dynamic temperature-based sampling strategy for the multilingual data.


Furthermore, we demonstrate that MTL can effectively improve the translation quality on zero-shot language pairs with no bitext training data.

mTL achieves even better performance than the pivoting approach for multiple low-resource language pairs.


Utilizing monolingual data with MTL significantly improves the zero-shot translation quality of the X→X system, further demonstrating the effectiveness of the proposed approach.

mTL achieves significantly better results than the pivoting approach on the high-resource pair  CsÃ¢â€ â€™De and almost all low-resource pairs.


We follow HotpotQA (Yang et al., 2018) to construct a more challenging dev/test split in our benchmark

we use some statistical features like the "size of the table", "similarity between answer passage and question", "whether question directly mentions the field", etc.


1) Ranking model: As the "retriever cells" contain many noises, we leverage a ranker model to predict the "correct" linked cells for the next stage.

this model takes each cell c along with its neighboring N c (cells in the same row) and feed them all into the cell encoder to obtain their representations {H c }.


During inference, we apply these three models sequentially to get the answer.

we use greedy search at first two steps to remain only the highest probably cell and finally extract the answer using the RC model.


In the reasoning phase, we mainly utilize BERT (Devlin et al., 2019) as our encoder for the cells and passages due to its strong semantic understanding.

we use four BERT variants provided by huggingface library 3 , namely baseuncased, based-cased, large-uncased, and largecased.


Notably, NVDM (Miao et al., 2016) employs variational autoencoder (VAE) (Kingma and Welling, 2013) to model topic inference and document generation.

nVDM consists of an encoder inferring topics from documents and a decoder generating documents from topics, where the latent topics are constrained by a Gaussian prior.


The method allows summaries to be understood in context to prevent a summarizer from distorting the original meaning, of which abstractive summarizers usually fall short.

we present a new method to produce self-contained highlights that are understandable on their own to avoid confusion.


To avoid these, we propose to apply ensemble of the top-3 models for each policy.

we first generate distribution with the top-3 models independently with the same policy, and then take the arithmetic average of the three distributions as the final token distribution at that step.


The embedding size and hidden size in BiLSTM are 100 and 200 respectively.

bERT  mentioned above refer to bERT base .


RAE (Zhang et al., 2018) further introduces the likelihood that two attribute values co-participate in a common n-ary fact, and adds the corresponding relatedness loss multiplied by a weight factor to the embedding loss of m-TransH.

rAE applies a fully-connected neural network to model the above likelihood.


Experimental results manifest the merits and superiority of NeuInfer.

on simple entity inference, NeuInfer outperforms the state-of-the-art method significantly in terms of all the metrics.


We design a global-to-local neural network to encode coarse-grained and fine-grained semantic information of entities.

we learn entity global representations by employing R-GCN (Schlichtkrull et al., 2018) on the created heterogeneous graph, and entity local representations by aggregating multiple mentions of specific entities with multi-head attention (Vaswani et al., 2017).


While semantically invariant text transformations can remarkably alter a model's predictions, the complementary problem of model undersensitivity is equally troublesome: a model's text input can often be drastically changed in meaning while retaining the original prediction.

previous works (Feng et al., 2018;Ribeiro et al., 2018a;Welbl et al., 2020) show that even after deletion of all but a small fraction of input words, models often produce the same output.


This paper proposes a semantic-enhanced Gaussian mixture model (SEG) for unknown intent detection.

we model utterance embeddings with a Gaussian mixture distribution and inject dynamic class semantic information into Gaussian means, which enables learning more classconcentrated embeddings that help to facilitate downstream outlier detection.


To deal with this task, we devise a data-driven global Skill-Aware Multi-Attention generation model, named SAMA.

to model the complex mapping relationships between input and output, we design a hierarchical decoder that we first label the job description with multiple skills, then we generate a complete text guided by the skill labels.


To address these challenges, we focus on the richness and accuracy of skills in generated job requirements and propose a global Skill-Aware Multi-Attention (SAMA) model for JPG task.

we devise a two-pass decoder to generate informative, accurate, and fluent job requirement paragraph.


To obtain the long-term dependency vector representation, we use a bi-directional LSTM (Schuster and Paliwal, 1997) as the text encoder.

the input sequence is transformed into a hidden state sequence H = (h 1 , h 2 , ..., h m ) by concatenating the representations of the forward and backward hidden states the initiated encoder hidden state h 0 is a zero vector, and the last encoder hidden state h m is used for initiating the skill decoder.


We consider addressing the limitation of existing work by modeling knowledge connections between domains explicitly.

a simple baseline to incorporate domain-shared and domain-private features is shared-private framework (Liu et al., 2017; Zhong et al., 2018; Wu et al., 2019b).


Generally, all the proposed components are effective to contribute the final performance.

we can clearly observe the effectiveness of our dynamic fusion mechanism where w/o domain-specific knowledge fusion causes 1.8% drops and the same trend in removing domain-shared knowledge fusion.


In contrast to the sharedprivate model, a dynamic fusion module (see §2.3) is further introduced to explicitly capture the correlation between domains.

a gate is leveraged to automatically find the correlation between a current input and all domain-specific models, so that a weight can be assigned to each domain for extracting knowledge.


In this paper, we propose a novel retrieve-andrewrite initialization method for UMT.

we first retrieve semantically similar sentence pairs from monolingual corpora of two languages with the help of unsupervised cross-lingual sentence embeddings.


For a fair comparison, our models and all baselines use the same neural encoder to encode text spans before the attention or OT operation is applied.

we use RoBERTa (Liu et al., 2019), a state-of-the-art pre-trained encoder, for the StackExchange and MultiRC dataset.


Lastly, we examine how large a bias-specific dataset needs to be in order to conclude that a given classifier is biased.

we consider a co-reference resolution system that is more accurate on sentences containing stereotypical gender roles.


As shown in Table 5, our two systems outperform S2S by a large margin, especially in terms of appropriateness, informativeness, goal success rate and coherence.

s2s tends to generate safe and uninformative responses, failing to complete goals in most of dialogs.


Our model augments the end-toend neural architecture with the ability to incorporate well-established graphs into both the document representation and summary generation processes.

a graph-informed attention mechanism is developed to incorporate graphs into the document encoding process, which enables our model to capture richer cross-document relations.


A natural idea is to combine our graph model with pretrained LMs so as to combine the advantages of them.

the tokenlevel transformer encoding layer of our model can be replaced by a pre-trained LM like BERT.


The results demonstrate that GraphSum and GraphSum+RoBERTa are able to generate higher quality summaries than other models.

the summaries generated by GraphSum and GraphSum+RoBERTa usually contains more salient information, and are more fluent and concise than other models.


To the best of our knowledge, we are the first to exploit the idea of meta-learning for transferring zeroshot knowledge in a cross-lingual setting for natural language understanding, in particular for the tasks of NLI and QA.

we exploit the usage of Model Agnostic Meta-Learning (MAML) which uses gradient descent and achieves a good generalisation for a variety of tasks (Finn et al., 2017).


Few-shot learning methods have initially been introduced within the area of image classification (Vinyals et al., 2016;Ravi and Larochelle, 2017;Finn et al., 2017), but have recently also been applied to NLP tasks such as relation extraction (Han et al., 2018), text classification Rethmeier and Augenstein, 2020) and machine translation (Gu et al., 2018).

in NLP, these few-shot learning approaches include: (i) the transformation of the problem into a different task (e.g., relation extraction is transformed to question answering (Levy et al., 2017;Abdou et al., 2019)); or (ii) meta-learning (Andrychowicz et al., 2016;Finn et al., 2017).


However, a narrative plays a different and more specific role than a general context.

a narrative may cover the whole story (a part of a script), thus a good conversation should also cover all the aspects mentioned in a narrative, which is not required with a general context.


Aspect-based sentiment analysis (ABSA) aims at fine-grained sentiment analysis of online affective texts such as product reviews.

its objective is to determine the sentiment polarities towards one or more aspects appearing in a single sentence.


Grounding annotation schemes in formal ontologies, and interlinking them with each other establishes a high degree of interoperability at a comparably low cost.

this approach differs from full-fledged standardization in that it does not require revisions of the actual annotation, but can be solely performed at the level of the vocabularies themselves.


To ensure faster convergence, we introduce a new constraint to the likelihood function: for each token position i, the corresponding latent label s i must have a non-zero probability in at least one labelling function (the likelihood of this label is otherwise set to zero for that position).

the aggregation model will only predict a particular label if this label is produced by least one labelling function.


Evaluation results on two datasets (CoNLL 2003 and news articles from Reuters and Bloomberg) show that the method can boost NER performance by about 7 percentage points on entity-level F 1 .

the proposed model outperforms the unsupervised domain adaptation approach through contextualised embeddings of Han and Eisenstein (2019).


In spite of its high relevance, KG embedding methods have not been used for multi-hop KGQA – we fill this gap in this paper

we propose EmbedKGQA, a novel system which leverages KG embeddings to perform multi-hop KGQA.


Fig. 4 shows the results (higher value/darker color is better): we find that responses generated from the symbolic planner as input do not perform well when compared to the ground truth.

the proportion of time that the ground truth response is preferred over that generated by the symbolic planner is significant (e.g.


From the figure, it is observed that for each RW methods, the node expansion based representation beats the performance of the tweet representation without any node expansion.

the sentiment polarized node expansion beats the performance of classifiers with and without non-polarized node expansion by an average margin of 9.19% and 10.57%, respectively.


This multistep process can serve as a natural benchmark for measuring whether systems achieve localization through a human-like process of progressively getting closer to the target location by interpreting intermediate steps.

this setup may helps researchers make sure that our systems are arriving at the referred location for the right reasons.


Out of the 4262 distinct sentences, only 188 were recorded more than once.

the chance of two annotators producing the same output string is quite low.


GNNs (Kipf and Welling, 2017) have recently become the paradigm for graph representation learning.

for the entity alignment task, the main merit of GNN-based methods lies in capturing the high-order proximity of entities based on their neighborhood information (Wang et al., 2018).


However, HyperKA still exhibits satisfying performance at very small dimensions in comparison to other methods, such as under the dimensions of 10 and 25.

hyperKA with 25 dimension even outperforms a number of methods in Table 2 with much higher dimensions, e.g., AlignE, GCN-Align and KECG.


The sequence labelling models, however, outperform both the baselines by a huge margin, demonstrating the effectiveness of the model on the task.

the best performance is achieved by combining pre-trained embeddings from multiple language models trained on different data sources.


However, despite impressive results obtained by systems using word embeddings, some issues were largely left unexplored, such as (i) the links between representations delivered through word embeddings vs. lexicographic meaning representations; (ii) the cognitive plausibility of the word embeddings (which is different from testing the agreement with conceptual similarity ratings); and (iii) the ways to acquire word embeddings to deliver common-sense usage of language (more on common-sense knowledge later on).

different from lexicographic resources where the minimal addressable unit of meaning is word sense (the synset), with few notable exceptions (such as NASARI [Camacho-Collados, Pilehvar, and Navigli 2015b] and SENSEEMBED [Iacobacci, Pilehvar, and Navigli 2015]), word embeddings typically describe terms.


In order to account for lexical ambiguity, Reisinger and Mooney (2010) propose representing terms as collections of prototype vectors; the contexts of a term are then partitioned to construct a prototype for the sense in each cluster.

for each word different prototypes are induced, by clustering feature vectors acquired for each sense of the considered word.


The language model proposed by Huang et al. (2012) exploits both local and global context that are acquired through a joint training objective.

word representations are computed while learning to discriminate the next word, given a local context composed of a short sequence of words, and a global context composed of the whole document where the word sequence occurs.


Because we are able to determine not only the distance between each two senses of the input terms, but also the distance between each input term and all of its senses, we use this information to fine tune the computed similarity scores and use ranking as a criterion to grade senses' relevance.

we hypothesize that the relevance of senses for a given term can be helpful for the computation of similarity scores, so we devised a measure that also accounts for the ranking of distances between senses and seed term.


Reported results are Pearson correlation indices, measuring the agreement with human annotated data.

we compare the Pearson scores obtained by the HCTI system using LESSLEX and GloVe vectors.


The investigation on abstract concepts has recently emerged as central in the multidisciplinary debate between grounded views of cognition versus modal (or symbolic) views of cognition (Bolognesi and Steen 2018).

in the first hypothesis cognition might be embodied and grounded in perception and action (Gibbs Jr 2005): accessing concepts would amount to retrieving and instantiating perceptual and motoric experience.


The first approach is to use models that have explicit mechanisms for representing syntactic structure.

our results suggest that the most important aspect of syntactic structure to include is constituency structure, as constituency models appear to implicitly learn dependency structure as well.


Both tasks are particularly challenging as many conditions must be met in a text to ensure it forms a proper WS.

translation is made difficult by the fact that some WS rely on linguistic phenomena that play differently in different languages.


We initialize the neutral topics and topic intensities with a pre-trained model.

we pre-train a Poisson factorization topic model using the algorithm in Gopalan et al. (2015).


Additionally, we use variational inference with reparam-eterization gradients to train all methods.

we perform mean-field variational inference, positing Gaussian variational families on all real variables and lognormal variational families on all positive variables.


KBQA includes simple questions that retrieve answers from single-hop triples ("what is Donald Trump's nationality") (Berant et al., 2013;Yih et al., 2014), multi-hop questions that infer answers over triple chains of at least 2 hops under specific constraints ("who is the president of the European Union 2012") (Yih et al., 2016;, and complex questions that involve set operations ("how many rivers flow through India and China") .

complex question answering (CQA) (Saha et al., 2018) is a sophisticated KBQA task in which a sequence of discrete actions-e.g., set intersection and union, counting, comparison-needs to be executed, and is the subject of this paper.


Instead of directly transferring the knowledge to the target domain, this paper proposes to borrow the benefit of multi-task learning, which allows a single model to be capable of handling multiple tasks/domains by feeding the corresponding data.

we utilize question answering for learning the capability of rationalization instead of a rationalizationspecific language model.


we first compute F1 score for each instance as the average F1 among all predictions in the instance.


In this paper, we propose a joint learning approach to improve neural abstractive multi-document summarization by using single-document summarization dataset.

we use the shared document encoder and summary decoder to process each document in the document set, and apply a decoding controller to aggregates all output probabilities from the summary decoder for multi-document summarization.


We have used the largest multilingual corpora available on a relatively long time, allowing thanks to its size to set a yearly granularity of analysis.

we used Google Books Ngrams 1 in English and French to conduct the analysis.


Our results show that while toxic prompts unsurprisingly yield higher toxicity in generations, nontoxic prompts still can still cause toxic generations at non-trivial rates (Table 2).

all five models have a toxicity probability near or above 0.5 for non-toxic prompts.


We find that certain prompts consistently cause all models to generate toxicity (e.g., the four prompts in Figure  1).

there are 327 prompts that yielded at least one generation with 0.9 TOXICITY from all models, and 1,225 prompts when considering only the out-of-the-box language models (i.e., GPT-1, GPT-2, GPT-3, CTRL, CTRL-WIKI).


To further investigate the phenomenon of neural toxic degeneration, and partially motivated by the surprising effectiveness of domain-adaptive pretraining on non-toxic data, we turn our focus to two corpora used to pretrain several language models.

we quantify the toxicity in OPENAI-WT (GPT-2's training data; Radford et al., 2019) and its open-source replica OWTC (Gokaslan and Cohen, 2019), inspired by previous work in analyzing social biases in large text corpora (Fast et al., 2016).


Other researchers have also shown that language models can "falsely" solve the task.

they might be taking advantage of dataset failures or artifacts on the input sentences in order to guess the answer (Gururangan et al., 2018; Agrawal et al., 2016; Levy et al., 2015).


To evaluate if Transformer-based models leverage the artifacts, we tested the models by removing the premise sentence in the development set.

the models  are unaware of the premises of the dataset.


We also noticed that some adversaries are model-specific, as they affect one model but not the rest.

in the noise tests, we observed that the robustness trend also holds, but noticed some unexpected behavior in relative analysis, as some types of noise affect the models more severely than others, thus revealing specific weak points across all Transformer-based models that did not seem evident at first sight.


The question-answer alignment and question/answer semantics are expected to be preserved in the representations.

the question/answer embeddings must reflect their semantics in the texts of being aligned as pairs.


We adapt the RuleTaker dataset to FEVER by introducing a new NOTENOUGHINFO label for unprovable question-context pairs.

we construct two FEVER-style RuleTaker datasets, namely RuleTaker-CWA and RuleTaker-Skip-Fact (example in Table 2).


In fact, this definition is not well suited for any type of machine learning algorithm, but for those types of algorithms that use labeled data as in our case.

a supervised neural network model is a mathematical function whose weights are refined iteration after iteration.


As an example, we analyze correlations between content structures and event coreference structures in news articles, and conduct experiments to incorporate system predicted sentence content types into an event coreference resolution system.

we analyze the lifespan and spread of event coreference chains over different content types, and design constraints to capture several prominent observations for event coreference resolution.


an intra-type event chain starts from a sentence of any type will die out within sentences of the same content type.


We can see that non-main contents (e.g., content types C2-D3) are more likely to be self-contained from introducing to finishing describing an event.

historical (D1) and anecdotal (D2) contents exhibit an even stronger tendency of having intratype event repetitions compared to other non-main content types.


In addition, we built a discourse-aware event singleton classifier, that resembles the sentence type classifier, to identify singleton event mentions in a document.

the singleton classifier combines document and sentence representations provided by the content type classifier with contextualized event word representations obtained from a separate word-level biLSTM layer with 512 hidden units.


The ILP model constrained by system predicted content structures (the row +Content Structure) outperforms the pairwise classifier baseline system as well as the two most recent systems consistently across all the evaluation metrics over the two benchmark datasets.

our ILP system outperforms the previous state-of-the-art, the heuristics-based ILP system Choubey and Huang, with average F1 gains of 0.67% and 1.32% on KBP 2016 and KBP 2017 corpora respectively.


We examine the language of the SOLO tweets to determine if the concept words loneliness, lonely, and solitude tend to be used in different emotional contexts.

we explore the question of whether Twitter users perceive the concept of solitude as more positive and selfdriven and the concept of loneliness as more negative and externally imposed as suggested by psychology literature.


In this section, we measure the emotional context in which the SOLO query terms, loneliness, lonely, and solitude, occur.

we investigate whether people use these terms in different emotional contexts and whether they are associated with the qualities suggested in the psychology literature.


In this work we take a step towards these goals by considering grounded dialogue involving open-ended discussion of a given image, a setting that is naturally fun for humans (Hu et al., 2014), and study neural conversational models for task.

we explore both generative and retrieval models that handle multimodal dialogue by fusing Transformer architectures (Vaswani et al., 2017) for encoding dialogue history and responses and ResNet architectures  for encoding images.


In this section, we describe our method of computing embeddings for target OOV words by using a weighted sum of pre-trained word embeddings.

we calculate the weights over known words from similarity scores derived on the basis of their surface information (Figure 1).


To be consistent with the seq2seq model for AMR parsing, the pre-trained models in this paper are all built on the Transformer.

for each pretraining task listed in Table 1, we learn a seq2seq model which will be used to initialize seq2seq model for AMR parsing in the fine-tuning phase.


Despite being a widely spoken language, it has been widely acknowledged that it has few publicly available tools and resources, apart from a few notable exceptions (Mahmoud El-Haj et al., 2015).

arabic NLP lacks resources such as corpora, lexicons, machine-readable dictionaries, Datasets in addition to fully automated fundamental NLP tools such as tokenizers, partof-speech taggers, parsers, stemmers and semantic role labelers.


When applying corpus-based techniques, one of the key things to consider is the extent to which size and subject affect the overall performance of the system.

based on the underlying processes involved, the COALS algorithm should be particularly sensitive to changes in domain and semantic space dimension.


The conjugations of verbs and the declensions of nouns are much different and more complex in Greek language than in English.

among different conjugations and declensions of a noun or an adjective, a word has different endings.


This method only considers the absolute distance of token i and j.

it does not distinguish the sign of the distance j - i.


To minimize the training cost, we test method 4 on a pre-trained BERT large model.

we load a pre-trained BERT large model, bert-large-uncased-whole-word-masking, from pytorch transformer 5 as the initial model and fine-tune the existing parameters and the new relative position embedding parameters for 3 epochs, staring with a small learning rate of 5e âˆ’ 5.


As in (Yih et al., 2016), we have found that careful design of the annotation tool leads to significant improvements in efficiency and accuracy.

we re-affirm the conclusion from (Yih et al., 2016) that having each worker do one task (e.g. labeling a single node in the tree) makes annotation easier for workers)


We demonstrate the results of several neural parsing models on our dataset.

we show the results of a re-implementation of (Dong  and Lapata, 2016) adapted to our grammar, and a straightforward fine-tuned BERT model (Devlin et al., 2018).


However, previous works (Giannakopoulos et al., 2014) have suggested that having better precision in such a task gives more perceived value for the user than recall.

users prefer small, clean clusters than larger clusters which may contain more of the relevant articles but also more off-topic articles.


In this paper we work with DNNs for text analysis and, given a text and a word embedding, consider the problem of quantifying the robustness of the DNN with respect to word substitutions.

we define the maximal safe radius (MSR) of a text as the minimum distance (in the embedding space) of the text from the decision boundary, i.e., from the nearest perturbed text that is classified differently from the original.


We employ our framework to perform an empirical analysis of the robustness trends of sentiment analysis and news classification tasks for a range of embeddings on vanilla CNN and LTSM models.

we consider the IMDB dataset (Maas et al., 2011), the Stanford Sentiment Treebank (SST) dataset (Socher et al., 2013), the AG News Corpus Dataset (Zhang et al., 2015) and the NEWS Dataset (Vitale et al., 2012).


In this work, we exploit approximate, scalable, linear constraint relaxation methods (Weng et al., 2018a; Zhang et al., 2018; Wong and Kolter, 2018), which do not assume Lipschitz continuity.

we adapt the CNNCert tool (Boopathy et al., 2019) and its recurrent extension POPQORN (Ko et al., 2019) to compute robustness guarantees for text classification in the NLP domain.


bounds of MSR for CNNs and LSTMs, respectively, with CNN-Cert and POPQORN tools, we implement the MCTS algorithm introduced in Section 2.2 to search for meaningful perturbations (i.e., upper bounds), regardless of the NN architecture employed.

in Section 3.1 we consider robustness against single and multiple word substitutions and investigate implicit biases of LSTM architectures.


we propose to leverage such meaningful k-mers to complement the representation produced by the language model (i.e., z i defined in Eqq. (1)) as “word”-level information.

in the source building, we obtain a k-mer vocabulary using the sensor names and their annotations -every ground-truth segment in a sensor name becomes a k-mer.


Since it is impossible for the model to predict for classes out of the training set, we thus only keep the overlapping classes between a pair of source and target buildings in evaluation.

given a pair of buildings, if a class exists only in either of the two buildings, we will mark it as an "other" class.


We evaluate the performance of SeNsER with regard to chunking and tagging using the precision, recall, and F 1 scores, similar to NER tasks.

for each sensor name, we get a few predicted triplets, i.e., (position begin , position end , category), and only when both the position and category exactly match the ground-truth annotations does it count as a correct extraction.


We also examine the generalizability of our method, i.e., how it would perform when applied to a building with a completely distinct vocabulary and naming convention.

we train a tagger using the sensor names and annotations in building A and B and apply it to building C.


To this end, we propose a new task towards persona-based empathetic conversations and present the first empirical study on the impact of persona on empathetic responding.

we first present a novel large-scale multi-domain dataset for persona-based empathetic conversations.


In this regard, a line of future work, briefly discussed in Section 3., is to further validate the multi-level annotation schemes.

reliability needs to be estimated on the basis of larger and more diverse samples.


Through well-designed probing experiments, we empirically validate that the improvement of SSANs can be attributed in part to mitigating two commonly-cited weaknesses of SANs: word order encoding and structure modeling.

the selective mechanism improves SANs by paying more attention to content words that contribute to the meaning of the sentence.


We used them as NLP benchmarks, which cover classification, sequence labeling and sequence generation categories.

the performances of semantic role labeling and language inference models heavily rely on structural information (Strubell et al., 2018), while machine translation models need to learn word order and syntactic structure (Chen et al., 2018; Hao et al., 2019c).


Figure 5 illustrates an example, in which more queries in SSANs attend most to the inserted word "the" than SANs.

sANs pay more attention to the surrounding words (e.g., distance < 3), while the inserted word "the" only accepts subtle attention.


Our proposed RACL is a unified multi-task learning framework which enables propagating the interactive relations (denoted as the same R 1 ..R 4 as those in Figure 1) for improving the ABSA performance, and it can be stacked to multiple layers to interact subtasks at different semantic levels.

we present the overall architecture of RACL in a single RACL layer contains three modules: AE, OE, and SC, where each module is designed for the corresponding subtask.


A plausible explanation is differences in the libraries.

the libraries used for training neural networks are in active development, and there may be important changes in short time periods.


In this paper, we propose a novel QA model based on text enhanced knowledge graph, which enriches entity representation by text semantic information and complements the relations in KB through structural information of the text.

the model firstly encodes entities in KB combining text information and applies graph convolutional networks (GCN) (Wu et al., 2020) to reason across KB.


In KB+Text's setting, our method also achieves the best performance, proving that our proposed enhancement strategy can effectively enhance incomplete KB by fully introducing the semantic and structural information implied in the text.

our model improves a lot compared with KB-only, more than the work of (Sun et al., 2018), which demonstrates our way that treating documents as hyperedges is more productive than regarding them as heterogeneous nodes.


AFS eases model convergence, and improves the translation quality by ∼1.3-1.6 BLEU, surpassing several strong baselines.

without data augmentation, AFS narrows the performance gap against the cascade approach, and outperforms it on LibriSpeech En-Fr by 0.29 BLEU, reaching 18.56.


Instead, we resort to sparsification techniques which have achieved great success in NLP tasks recently (Correia et al., 2019; Child et al., 2019; Zhang et al., 2020).

we employ L0 DROP (Zhang et al., 2020) for AFS to dynamically retain informative speech features, which is fully differentiable and independent of concrete encoder/decoder architectures.


Also, unlike prior work on improving interpretability (Erion et al., 2019;Plumb et al., 2019), it does not require pre-defined important attributions or pre-collected explanations.

we propose variational word masks (VMASK) that are inserted into a neural text classifier, after the word embedding layer, and trained jointly with the model.


In this work, we introduce a jointly masked sequence-to-sequence model and explore its application on non-autoregressive neural machine translation (NAT).

we first empirically study the functionalities of the encoder and the decoder in NAT models, and find that the encoder takes a more important role than the decoder regarding the translation quality.


Therefore, we propose a jointly masked sequence-to-sequence model which is inspired by the idea of masked language modeling (Devlin et al., 2018).

for the encoder, we follow the masking strategy of BERT (Devlin et al., 2018) and randomly mask a number of tokens of the source sentence.


Firstly, we vary the number of encoder and decoder layers respectively to see which will bring more performance gain.

on a basic model with a 5-layer encoder and a 5-layer decoder, we increase the number of layers to the encoder and decoder separately.


We follow (Ghazvininejad et al., 2019) and introduce an additional prediction process to estimate the length by the source sentence.

we add a special token to the encoder and predict the target length with the output hidden vector of this token.


Next, for each iteration, we apply consecutive masking to the translation candidates as we have done in the training stage.

we select several tokens with the lowest probabilities from the current translation candidates, and mask these tokens as well as their adjacent ones.


We propose two mechanisms to exploit the structure of evidence while leveraging the advances of pre-trained models like BERT, GPT or XLNet.

using XLNet as the backbone, we first utilize the graph structure to redefine the relative distances of words, with the intuition that semantically related words should have short distances.


Considering the aforementioned issues, we implement an approximate solution to trade off between the efficiency of implementation and the informativeness of the graph.

we reorder evidence sentences with a topology sort algorithm with the intuition that closely linked nodes should exist in neighboring sentences.


With a given claim, we represent the retrieved evidence sentences as a graph, and then use the graph structure to guide the reasoning process.

we apply semantic role labeling (SRL) to parse each evidence sentence, and establish links between arguments to construct the graph.


To fully harness the power of fine-tuning in a more principled manner, we propose a new learning framework for robust and efficient fine-tuning on the pre-trained language models through regularized optimization techniques.

our framework consists of two important ingredients for preventing overfitting: (I) To effectively control the extremely high complexity of the model, we propose a Smoothnessinducing Adversarial Regularization technique.


The pre-trained language model is then adapted to downstream tasks and further fine-tuned.

the top layer of the language model can be replaced by a task-specific layer and then continue to train on downstream tasks.


We present an evaluation benchmark, GLUECoS, for code-switched languages, that spans several NLP tasks in English-Hindi and English-Spanish.

our evaluation benchmark includes Language Identification from text, POS tagging, Named Entity Recognition, Sentiment Analysis, Question Answering and a new task for code-switching, Natural Language Inference.


Multilingual BERT is pre-trained on monolingual corpora of 104 languages and has been shown to perform well on zero shot cross-lingual model transfer and code-switched POS tagging (Pires et al., 2019).

we use the bert-basemultilingual-cased model for our experiments.


To predict the labels of the nodes following the practices in (Zhu et al., 2005), we employ a modified Viterbi algorithm in which the nodes in the vertical chain are decoded first.

the constructed graph consists of two types of cliques: one from the horizontal chains and the other from the vertical chain.


A modified Viterbi algorithm is used to find the best labeling sequence.

we first applies the Viterbi algorithm to decode the vertical chain.


The BERT-based parser self-trained with 40% silver Fisher trees and 60% gold Switchboard trees is our best model.

for a batch size of 30, in each mini-batch 12 parse trees come from the silver Fisher data and 18 parse trees from the gold Switchboard.


One effective way to obtain domain-invariant features is adversarial training (Ganin et al., 2016;Li et al., 2017;Zheng et al., 2019).

a domain discriminator is learned by minimizing the classification error of distinguishing the source from the target domains, while a deep classification model learns transferable representations that are indistinguishable by the domain discriminator.


To address above problems, we design a novel pre-training task to make BERT domain-aware and then improve the BERT's fine-tuning procedure by adversarial training.

a novel post-training procedure is implemented that adapts BERT with unlabeled data from different domains to enhance the domain-awareness.


Based on the post-trained BERT, we now could utilize the adversarial training to abandon the distilled domain-specific features to derive the domaininvariant features.

a sentiment classifier and a domain discriminator is designed operating on the hidden state h [CLS] of the special classification embedding [CLS].


We experiment with a strong neural QE approach based on BERT (Devlin et al., 2019).

we focus on the bert-base-cased version of the multilingual BERT.


In general, the annotation should not depend on the table contents and sorting assumptions.

use direct references to the presented row order id as little as possible.


In this paper, we aim to overcome the above problems to automatically generate faithful texts from tables.

we aim to produce the writing that a human without any external knowledge would do given the same table data as input.


Our model adopts the powerful Transformer model (Vaswani et al., 2017) to translate a table to a text sequence.

the Transformer is a Seq2Seq model, consisting of an encoder and a decoder.


Our first idea is inspired by related work in machine translation .

we propose to constrain a table embedding to be close to the corresponding target sentence embedding.


Patt is the attention weights (probability) returned from the encoder-decoder attention module in the Transformer.

when generating the current word yi , the encoder-decoder attention module calculates the probability vector Patt denoting the probabilities of attending to each word in the input table.


To further reduce the number of parameters and improve the computational efficiency, we adopt the factorized embedding parameterization proposed recently (Lan et al., 2019).

we decompose a word embedding matrix of size V × D into the product of two matrices of sizes V × H and H × D, respectively.


We posit that generated questions should not be easily classified into either LM-type or copy-type if the two styles were to be inter-mixed in a balanced way.

the style of the questions should not be highly biased toward either side.


We show that the choice of probability space puts constraints on the distant supervision assumptions that can be captured, and that all three choices interact, leading to large differences in performance.

we provide a framework for understanding different distant supervision assumptions and the corresponding trade-off among the coverage, quality and strength of distant supervision signal.


This paper employs a novel knowledgeaware pointer network.

we expand the scope of the original pointer networks by exploiting the attention distribution of knowledge representation.


To induce a useful latent generation plan and to effectively condition on a sampled plan, we propose a constrained story decoder and constrained inference network.

our constrained decoder begins a story sentence by deterministic copying the corresponding anchor word, and then generates words to the left and then to the right (Figure 3).


Replicability and reproducibility are core ideas of modern scientific methods (Ivie and Thain, 2018), and thus reproducibility of scientific research results is an important topic in many research areas.

over the last decade, terms such as "reproducibility crisis" or "p-hacking" have gained attention from different scientific communities, especially after studies showed failure in reproduction of researches in life and medicine science (Begley and Ellis, 2012), psychology ans behavioral science (Anderson et al., 2015), as well as astrophysics (Collaboration and others, 2015).


In addition to the model architecture presented in paper, the authors describe various components for preprocessing and postprocessing steps taken in the development of the system that effectively raise the system performance.

in the preprocessing steps, they cropped the sentences to only keep the entity pairs and tokens in between, and cleaned the sentences by flattening nested entities and removing all tokens between parentheses and brackets ((, ), [, ]).


Due to the lack of jointly labeled data for these relational phenomena and the restriction on the structures they articulate, we propose a joint constrained learning framework for modeling event-event relations.

the framework enforces logical constraints within and across multiple temporal and subevent relations by converting these constraints into differentiable learning objectives.


To characterize the event pairs in the document, we employ a neural encoder architecture which provides event representations from two groups of features.

the representation here incorporates the contextualized representations of the event triggers along with statistical commonsense knowledge from several knowledge bases.


We also incorporate the following sources of commonsense knowledge to characterize event pairs.

we first extract relevant knowledge from ConceptNet (Speer et al., 2017), which is a large-scale commonsense knowledge graph for commonsense concepts, entities, events and relations.


In this section, we present the experiments on eventevent relation extraction.

we conduct evaluation for TempRel and subevent relation extraction based on two benchmark datasets ( §4.1- §4.4).


Since there is not a large-scale dataset that amply annotates for both TempRel and subevent relations, we evaluate the joint training and prediction of both categories of relations on two separate datasets.

we use MATRES (Ning et al., 2018b) for TempRel extraction and HiEve  for subevent relation extraction.


This is motivated by the idea that while the average number of “old” citations per paper is stable (cf. Sec. 3.2), they might be distributed in an unbalanced way.

there might be a subset of publications that does not cite any "older" work.


Negative training can be derived from Empirical Bayes Risk Minimization (Och, 2003).

the overall objective is to minimize the expected risk that the model exhibits undesirable decoding behavior: where c(x, y) refers to the binary criteria that will be 1 if (x, y) exhibits undesirable behavior, and 0 otherwise.


In this paper we introduce a method of measuring more specific legislator attitudes using an alternative expression of preferences: tweeting.

we present an embedding-based model for predicting the frequency and sentiment of legislator tweets.


Different from classical neural generation models, our generation model is in a two-stage generation manner, motivated by coarse-to-fine decoding (Dong and Lapata, 2018).

given a fluent sentence as the input, in the first stage, a Planner selects the positions of where to insert reparandum; in the second stage, a Generator generates disfluent segments accordingly for the predicted areas.


In this paper, we propose a neural model named TWASP for joint CWS and POS tagging following the character-based sequence labeling paradigm, where a two-way attention mechanism is used to incorporate both context feature and their corresponding syntactic knowledge for each input character.

we use existing language processing toolkits to obtain the auto-analyzed syntactic knowledge for the context, and the proposed attention module can learn and benefit from them although their quality may not be perfect.


First, for all encoders, the two-way attentions provide consistent enhancement to the baselines with different types of knowledge.

although the baseline model is well-performed when BERT (or ZEN) serves as the encoder, the attention module is still able to further improve its performance with the knowledge produced by the toolkits even though the toolkits have worse-than-baseline results for the joint task


This suggests that BERT is aware of the structure of the document it is given.

we observe a decent accuracy in identifying discourse relations between adjacent EDUs, perhaps due to the "next sentence prediction" task in pre-training, as pointed out in (Shi and Demberg, 2019).


Their goal is to check if the attention heads of a given pre-trained model can track syntactic relations better than chance or baselines.

raganato and Tiedemann (2018) analyze a machine translation model's encoder by extracting dependency trees from its self-attention weights, using Chu-Liu/Edmonds algorithm.


Text simplification (Shardlow, 2014) that rewrites a sentence into an easy-to-understand form for language learners (Petersen and Ostendorf, 2007) and children (Belder and Moens, 2010) is attracting attention.

lexical simplification (Paetzold and Specia, 2017a), which paraphrases complex words into simpler ones according to the context while maintaining the syntactic structure of the input sentence, is being actively researched (mainly in English).


Complementing these works, we directly manipulate models' modeling of the context surrounding identifier terms by regularizing explanations of these terms.

we use post-hoc explanation algorithms to interpret and modulate finetuned language models like BERT (Devlin et al., 2018), which achieve state of the art performance on many hate speech detection tasks (MacAvaney et al., 2019;Mandl et al., 2019).


The goal of this paper is to establish a new state of the art for French NER by (i) providing a new, easy-to-use UDaligned version of the named entity annotation layer in the FTB and (ii) using this corpus as a training and evaluation dataset for carrying out NER experiments using state-ofthe-art architectures, thereby improving over the previous state of the art in French NER.

by using both FastText embeddings (Bojanowski et al., 2017) and one of the versions of the CamemBERT French neural contextual language model (Martin et al., 2019) within an LSTM-CRF architecture, we can reach an F1-score of 90.25, a 6.5point improvement over the previously state-of-the-art system SEM (Dupont, 2017).


Graph attention is used for capturing such global semantic information in a graph.

it allows each node to deal with the triples that are composed of the embeddings of the neighbor nodes, embeddings of the corresponding edges, and its own embedding.


Most models, including a GCN (Damonte and Cohen, 2019), GGNN (Beck et al., 2018), and GraphLSTM (Song et al., 2018), use a nonpairwise interaction function to represent the information to be aggregated from the neighborhoods.

they ignore the receiver node (i.e., the node to be updated), operating only on the sender node (i.e., the neighbor node) and the edge attribute (Battaglia et al., 2018).


However, most previous works are not designed for QDA.

they do not aim to generate context-relevant questions for improving downstream model performance.


Figure 2 illustrates the process of question rewriting.

we take the process of rewriting an answerable question to a relevant unanswerable question as an example to present the process.


The reason might be that the autoencoder trained with denoising task will be insensitive to the word embedding revision of CRQDA.

some revisions guided by the MRC gradients might be filtered out as noises by the autoencoder, which is trained with a denoising task.


Experiment results demonstrate its validity and effectiveness where state-of-the-art performance is achieved on many tasks using the ngrams, which are automatically learned from the training data other than external or prior knowledge.

our method outperforms some existing encoders trained on much larger corpora.


Compared to ERNIE 2.0, which used many more pre-training tasks and significantly more non-public training data, ZEN is still competitive on SA, SPM and NLI tasks.

zEN outperforms ERNIE 2.0 (B) on SA (TEST) and SPM (TEST), which indicates that ngram enhanced character-based encoders of zEN can achieve performance comparable to approaches using significantly more resources.


The word suggestions are determined by a language model.

they are the top-n words with highest probability that begin with the prefix of the word that has been typed so far.


To fill this gap, we propose to investigate particular orderings of persuasive strategies that affect a request's persuasiveness and identify situations where these orderings are optimal.

we take a closer look at strategies (Table  1) and their orderings in requests from the subreddit/online lending community r/Borrow 1; and utilize them to examine research questions like: When should requesters follow strategy orderings (e.g., ending loan requests with politeness) that rely on social norms?


All experiments are performed with the Transformer architecture (Vaswani et al., 2017) using the open-source Tensorflow-Lingvo implementation (Shen et al., 2019).

we use the Transformer Big model containing 375M parameters (6 layers, 16 heads, 8192 hidden dimension) (Chen et al., 2018) and a shared source-target Sen-tencePiece model (SPM) 3 (Kudo and Richardson, 2018).


We propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data.

after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets.


This is especially helpful in the case of commonsense KGs (e.g., ConceptNet (Speer et al., 2017)) that consist of non-canonicalized text and hence suffer from severe sparsity (Malaviya et al., 2019).

these methods usually feed concatenated triples (e.g., [HEAD, Relation, TAIL]) into LMs for training or finetuning.


Compared to the second class, we learn from free-form text through MLMs rather than triples, which fosters generalization on other downstream tasks.

given a corpus of raw text and a KG, two KG-guided self-supervision tasks are formulated to inject structured knowledge into MLMs.


It explores implicit relational information underlying raw text by the guidance of a KG, and is shown to mask more informative entities compared to the previous random approaches.

the scheme is designed to avoid or reduce masking two types of entities: trivial and undeducible.


This work is in line with Baidu-ERNIE (Sun et al., 2019a) and SpanBERT  which replace word-level mask (Devlin et al., 2019) with span-level one for knowledge information and longterm dependency.

baidu-ERNIE uses uniformly random masking for phrases and entities, whereas SpanbERT directly masks out token spans sampled under geometric distribution.


All these tests try to capture some fixity of the expressions, that tend to entail idomaticity as productive composition rules can not be applied.

the fixedness of a MWE can be identified by applying a transformation on the given MWE that leads to an unexpected meaning shift or an unacceptable sequence, with respect to a regular setting.


To this end, a practitioner assumes a probability distribution on the observations y, parameterized by θ, the properties of the systems.

y is assumed to have a distribution 5 with unknown parameters θ.


A 95% CI of this quantity provides a range of values that are not rejected under the corresponding null-hypothesis.


In Bayesian inference frameworks, a priori assumptions and beliefs are encoded in the form of a prior distribution P(θ) on parameters of the model

a prior distribution describes the common belief about the parameters of the model.


We argue that what is crucial to the cross-modality attention layer is the ability to selectively enrich the semantic representation of a sentence through the aid of an image.

we need to avoid introducing noises resulted from when the image fails to represent some semantic meaning of words, such as abstract concepts.


However, a standard language modeling objective does not lead to a model that handles the two challenges mentioned above; in addition, other systematic issues limit its ability to handle TCS.

language models do not directly utilize the ordinal relationships among temporal units.


We assume adjacent units have a distance of 1, and we generate y based on a Gaussian distribution with a standard deviation of 0.5.

we assume the two immediate neighbors of a gold label are reasonably possible.


To produce more general representations, we also keep the temporal label and mask the event tokens instead at a certain probability, so that we are able to maximize both P (Tmp-Label|Event) and P (Event|Tmp-Label) in the same learning process, where Tmp-Label refers to the temporal label associated with the event.

we use the reserved "unused" tokens in BERT-base model lexicon to construct a 1-to-1 mapping from every value in every dimension to the new vocabulary.


we apply weight adjustment to the total loss with a weight factor calculated as the observed label's count relative to the number of all instances.


For example, "I brush my teeth [Month]" will be discarded because all candidate answers are approximately uniformly distributed so that one cannot identify a subgroup of labels to be more likely.

we ask one annotator to select from 4 choices regarding each (event, temporal value) tuple.


Beyond unsupervised intrinsic experiments, we also evaluate the capability of the event temporal representation as a product of our model.

we finetune both BERT baseline and our model with the same process to compare the internal representations of the transformers.


Lexical semantic change models build on the assumption that meaning change results in the modification of a word's linguistic distribution.

with the exception of a few methods based on word frequencies and parts of speech (Michel et al., 2011; Kulkarni et al., 2015), lexical semantic change detection has been addressed following two main approaches: form-based and sense-based (for an overview, see Kutuzov et al., 2018; Tang, 2018).


By metaanalyzing a set of existing NLP papers and datasets, we characterize typical power for a variety of settings and conclude that underpowered experiments are common in the NLP literature.

for several tasks in the popular GLUE benchmark, small test sets mean that most attempted comparisons to state of the art models will not be adequately powered.


When one does not have access to the baseline model or an informative prior, one can make use of historical trends.

we can try to estimate what a typical improvement will look like, given the current state of the art (SOTA).


We also observe a slight negative correlation between effect size and sample size.

as sample size gets larger (and, thus, as estimates get more precise), the estimated effect size gets smaller.


In this paper, we present a large dataset to foster computational studies on the natural selection of English words.

the dataset includes sets of synonymous monosemous words and their frequencies in English books across 200 years.


We follow existing approaches to obtain an opinion set Or for every review in our corpus.

we used a pre-trained tagging model (Miao et al., 2020) to extract opinion phrases, their polarity, and aspect categories.


While CLWE enable many multilingual tasks (Klementiev et al., 2012;Guo et al., 2015;Zhang et al., 2016;Ni et al., 2017), most recent work only evaluates CLWE on bilingual lexicon induction (BLI).

a set of test words are translated with a retrieval heuristic (e.g., nearest neighbor search) and compared against gold translations.


This is run with min-occur set to 1, in order to preserve information and provide a fair, higher-AMI clustering that incorporates a maximum amount of corpus knowledge.

we do not eliminate low frequency words from the vocabulary.


We attribute this to the Zipf-Mandelbrot distribution of word frequencies (Montemurro, 2001) and the effect frequency has on each word type's contribution to global mutual information.

the majority of words share little mutual information with most other words; thus, most words may be clustered quickly and will not later be moved -later cluster changes are unlikely to have strong far-reaching effects.


For the last experiment, we compute and report our ablation results over 2 factors in Table 4.

the scores degrade on FB13 and SEARCH17 when not using the positional embeddings.


In the pattern matching of term lists against abstracts, we basically performed matching within generated variations of term list entries.

we applied lemmatization process and regular expression based pattern matching.


Moreover, two graph convolutional networks are leveraged to learn the latent representations depicting how sessions interact with one another in terms of users, and how posts are correlated with each other in terms of words.

we address several challenges in this work: (a) how to perform explainable cyberbullying detection that can boost detection performance, (b) how to highlight explainable comments without the ground truth, (c) how to model the correlation between posted text and user comments, and (d) how to model the interactions between sessions in terms of users, and the interactions between textual posts in terms of words.


Inspired by (Yang et al., 2016),  we adopt a hierarchical neural network to model word-level and sentence-level representations through selfattention mechanisms.

we first learn the comment embedding vector by utilizing the word encoder with self-attention.


That said, we intend to simultaneously learn and derive the attention weights of words on posted text and comments.

first, similar to comment encoding, word embeddings of a posted text are obtained by a pre-trained word2vec model.


To answer EQ3, we examine whether HENIN can accurately detect cyberbullying sessions at early stages.

we aim to understand how a model performs given only a partial proportion of observed comments.


We choose GRU+A as the baselines for comment explainability since it can learn attention weights for comments as a kind of explainability.

we want to see if the top-ranked explainable comments determined by our HENIN are more likely to be related to the major contexts in cyberbullying media sessions.


Memory Utilization Surprisingly, the top-1 memory usage of the PKM-augmented PLM at the 4th layer is about 2%, which is remarkably low, though top-32 memory usage at this layer is almost 100%.

the model does not take advantage of the lower memory layer effectively.


In this work we conduct a systematic study of the effects of backtranslated data from different sources, as well as how to optimally select subsets of this data taking into account the loss in quality and lexical richness when data is translated with different MT systems.

we aim to (i) provide a systematic analysis of backtranslated data from different sources; and (ii) to exploit a reduction in the amount of training data while maintaining high translation quality.


concatenate the data backtranslated with all the systems and select the optimal sentence pairs avoiding the selection of the same target sentence more than once.

each selected target sentence will have only one associated source sentence originating from one specific system.


The vector of a context-insensitive word is assigned a higher norm.

if a word is usually found in specific contexts, it should be regarded as a significant word (Luhn, 1958).


Motivated by these works and our preliminary experimental results, we propose to use the word vector norm as a criterion to determine the difficulty of a sentence.

we first train a simple word embedding model on the training corpus, and then obtain an embedding matrix E w2v .


To clear up any doubts, we further conducted the experiments on the large-scale Zh-En translation without tuning these two hyperparameters, that is, directly using λm = 2.5 and λw = 0.5.

the only difference is the use of a large number of training steps in Zh-En, namely, 150K, for the purpose of better model fitting.


The line before the change is called the source and the line after is the target.

an edit is a pair of the source and the target.


We used NanigoNet 8 , a language detector based on GCNNs (Gated Convolutional Neural Networks) (Dauphin et al., 2017) that supports human languages as well as programming languages.

we ran the language detector against both the source and the target and discarded all the edits where either is determined as written in a non-human language.


Finally, after annotating a small amount of samples for the three languages, we computed some basic statistics about each edit that may help in classifying typo edits from nontypo ones.

we computed three statistics: 1. Ratio of the target perplexity over the source calculated by a language model 2. Normalized edit distance between the source and the target 3. Binary variable indicating whether the edit purely consists of changes in number


Owing to the challenges posed in reasoning over formal representations (Musen and Van Der Lei,1988), and backed by the recent successes of transformers (Vaswani et al., 2017) in NLP, Clark et al. (2020) propose a new version of the problem by replacing the formal representations of rule-bases with natural language (English).

their task requires predicting the truth value of a statement by reasoning over a set of facts and rules, all expressed in natural language.


The connectivity constraint provides only marginal improvement as our model mostly predicts connected proofs without any explicit supervision.

only 57 examples have disconnected proofs without this constraint.


Almost all contemporary search systems are based on datadriven techniques that train computers to improve search and information retrieval.

user activity history is the most crucial data for ranking search results by their popularity and relevance.


Note that voice suffixes in Turkish can be stacked.

a passive suffix and a causative suffix can be found on a single verb, or a verb can take two passive suffixes and become a double passive construction.


Neural network-based language models (LMs) have been shown to learn relevant properties of language without being explicitly trained for them.

recent work suggests that they are able to capture syntactic relations to a large extent (Gulordava et al., 2018; Kuncoro et al., 2018; Wilcox et al., 2018)


SCM requires to model the relationship between cases from the information of different granularity, like fact level, event level and element level.

sCM is a particular form of semantic matching (Xiao et al., 2019), which can benefit the legal information retrieval.


Different from previous news classification tasks whose category set is general topics like entertainment and sports, MATINF-C is a fine-grained classification under a single domain.

the distance between different categories is smaller, which provides a more challenging stage to test the continuously evolving state-of-the-art neural models.


Though recent studies show that multi-task learning is effective, there is still one more question to answer.

when training models on multiple tasks, multiple datasets are used by default.


We take into account only the two ends of the scale, namely only the highly polarized text.

we consider the articles with bias score 1 and 2 as unbiased (negative class), and the ones with bias score 4 and 5 as biased (positive class).


As a first step to examine the quality of the human labels, we measure the inter-annotator agreement (ITA) within each collection.

we calculate the agreement for the expert dataset, the Figure Eight dataset and the MTurk dataset separately.


We model messagelevel representation from its word sequence.

given u's historical message m, we first use a pre-trained word embedding layer to map each word into a vector space, and then employ a Convolutional Neural Network (CNN) (Kim, 2014) encoder to model word occurrence with their neighbors.


We, then, use word-to-word translation to code-switch (CS) the English training set with the target language (T L) for these words.

we replace all utterances of "important" words with it's TL dictionary translation.


Rather than blaming individual shooters, the German press paid more attention to U.S. public opinion manifesting as gun violence protests and the U.S. gun regulations.

compared to the U.S.'s news coverage, foreign media tended to attribute the responsibility to the U.S. government.


Hence, we propose a novel wordaligned attention to exploit explicit word information, which is complementary to various character-based Chinese pre-trained language models.

we devise a pooling mechanism to align the character-level attention to the word level and propose to alleviate the potential issue of segmentation error propagation by multi-source information fusion.


One possible reason for the result is that the cop (between arguments and predicates) sets up a bridge for sib and gp structures.

these observations suggest that the three structure learning may be complementary.


Pre-training can bring about a significant improvement in performance on both in-domain and out-of-domain test sets; however, the in-domain improvement is significantly greater than that of out-of-domain when the two domains are far apart.

the difference between in-domain and out-of-domain in German and English is large, while the two domains in Czech are similar.


We take as our starting point a state-of-theart image captioning system and develop several model variants that exploit information from human gaze patterns recorded during language production.

we propose the first approach to image description generation where visual processing is modelled sequentially.


We hypothesise that a system that encodes gaze data as a proxy for human visual attention will lead to better, more human-like descriptions.

we propose that training such a system with eye-movements sequentially aligned with utterances (see Figure 1) will produce descriptions that reflect the complex coordination across modalities observed in cognitive studies.


We utilise the Dutch Image Description and Eye-Tracking Corpus (DIDEC;van Miltenburg et al., 2018).

we use the data collected as part of the description-view task in DIDEC, where participants utter a spoken description in Dutch for each image they look at.


To empirically motivate our generation models, as a preliminary experiment we investigate the level of coordination between visual attention and linguistic production in the DIDEC dataset.

we test whether scanpath similarity and sentence similarity are correlated and whether taking into account the sequential nature of the two modalities results in higher cross-modal alignment.


We take the original model as our baseline and modify it to integrate visual attention defined by gaze behaviour.

we replace the meanpooled vector v by a gaze vector g computed from masked images representing fixation patterns as explained in Section 3.


In order to generate story-like textual descriptions that complement the factual captions, we additionally train our model to exploit our diverse completesentence annotations.

instead of producing the commonsense knowledge given the videos and captions, we finetune our pre-trained V2C-Transformer model on predicting the human rewritten texts, and generate complete story-like captions.


Since the neural networks can be near-perfect nonlinear function approximators (Liang and Srikant, 2016), we can expect an NAR model to learn the AR translation process quite well, as long as the model has enough capacity.

we first obtain the greedy output of a trained AR model, and use the resulting paired data to train the NAR model.


For instance, a sequence labelling model with a 60% accuracy at sentence level means that only 60 % of positive events are covered by the set corresponding to argmax y p Î¸ (y|x) predictions.

only 60 % of the correct outputs of model p Î¸ will be used for constructing the forecaster.


Although we confirm the results presented in this work, we demonstrate that the method presented there is not sufficiently robust to handle mappings between a bit more distant languages.

we show that some Slavic languages (Czech and Polish) are quite challenging for the method.


Turning now to (R2), we argue that work analyzing "bias" in NLP systems should provide explicit statements of why the system behaviors that are described as "bias" are harmful, in what ways, and to whom, as well as the normative reasoning underlying these statements.

researchers and practitioners should articulate their conceptualizations of "bias."


Participatory action research in education (Kemmis, 2006) and in language documentation and reclamation (Junker, 2018) is also relevant.

work on language reclamation to support decolonization and tribal sovereignty (Leonard, 2012) and work in sociolinguistics focus-ing on developing co-equal research relationships with community members and supporting linguistic justice efforts (e.g., Bucholtz et al., 2014Bucholtz et al., , 2016Bucholtz et al., , 2019 provide examples of more emancipatory relationships with communities.


Different from previous studies which utilize various external information limited to a specific scenario, our model incorporating historical dialogue information into generation is easy to generalize and applied to practical applications.

we introduce a novel historical dialogue selection strategy to find appropriate historical seller responses for the latest customer question.


These experiments explain such learning phases by showing that the training process is inherently compositional due to bottom-up learning.

not only are the shorter sequences learned first, but they form the basis for longer relations learned over them.


These results indicate that predictable patterns play a vital role in shaping the representations of symbols around them by composing in a way that cannot be easily linearized as a sum of the component parts.

as seen in Figure 10, the DI between open symbol and scaffold is substantially higher for the familiar-setting model and increases throughout training.


We use word similarity to extend these seed words and finally got a dictionary with 3,227 fine-grained semantic words.

we use spaCy to compute the similarity between words.


Given the MIMIC database as substrate and the aforementioned policy initiatives to reduce unnecessary hospital readmissions, as well as the goal of providing structure to text, we elected to focus on patients who were frequently readmitted to the ICU (Ryan et al., 2015).

a patient who is admitted to the ICU more than three times in a single year.


In this work, we adapt general-purpose extractive summarization algorithms (Nallapati et al., 2017;Zheng and Lapata, 2019) to identify informative scenes in screenplays and instill in them knowledge about narrative film structure (Hauge, 2017;Cutting, 2016;Freytag, 1896).

we adopt a scheme commonly used by screenwriters as a practical guide for producing successful screenplays.


We first present an unsupervised variant which modifies the computation of scene centrality in the directed version of TEXTRANK (Equation 1).

we use the pre-trained network described in Section 3.3 to obtain TP-specific attention distributions.


We should note that Frermann et al. (2018) discriminate between different cases presented in the same episode in the original CSI dataset.

there are episodes in the dataset, where except for the primary crime investigation case, a second one is presented occupying a significantly smaller part of the episode.


where h is a simple linear layer for classification, and the parameters of both language model and the linear layer will be fine-tuned on the downstream commonsense reasoning task.

in the training process, the objective is to minimize the negative log-likelihood of ground-truth answers a * i as follows.


In this paper, we investigate the Knowledge Distillation (KD) method, which has been shown to be effective for knowledge ensembling (Hinton et al., 2015;Kim and Rush, 2016;Furlanello et al., 2018), for heterogeneous structure integration.

we employ a sequential LSTM as the student for distilling heterogeneous syntactic structures from various teacher tree encoders, such as GCN (Kipf and Welling, 2017) and TreeLSTM (Tai et al., 2015a).


This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562.

it used the Bridges-GPU AI system at the Pittsburgh Supercomputing Center (PSC) through allocations TG-CCR200004.


As for the generation models, we compute several metrics that are commonly used in the domain of Natural Language Generation.

we consider three measures based on n-gram matching: BLEU-2 (Papineni et al., 2002), 8 ROUGE (Lin, 2004), and observed.


Haber et al. (2019) reported a rise in the proportion of content words for all utterance types in later rounds of the PhotoBook games.

we also observe such an increase in our referring utterance chains, and a similar trend is exhibited as well by the output of the ReRef and Copy models: generated subsequent references contain a significantly higher proportion of nouns and adjectives compared to first descriptions.


In comparison to NEO, SCAN tends to assign even more extreme probabilities.

sCAN tends to make more serious errors: in more than 5 millions cases, the predicted probability is 0 (or close to 0) for the gold sense instead of 1.


We investigate how the size of each sense (as opposed to the full target size) contributes to the performance of the model.

we observe the difference between targets where the senses have a similar size and targets where there is a strong imbalance between the senses.


The model architecture of CoMMA is shown in Figure 1.

coMMA differs from standard Transformer (Vaswani et al., 2017) as follows: 1) Some tokens are randomly replaced by a special mask token M with probability p, and the model is trained to predict original unmasked tokens.


It incorporates analyses and predictions from various tools designed for processing texts in Sanskrit, and utilises them to ease the cognitive load of the human annotators.

sHR++ uses sanskrit Heritage Reader (Goyal and Huet, 2016), a lexicon driven shallow parser for enumerating all the phonetically and lexically valid word splits along with their morphological analyses for a given string.


This section elaborates on the mechanism of our proposed affective neural networks.

how we incorporate affective influence values into affective deep neural networks.


It is based on the assumption that people tend to maintain culturally shared perceptions of identities and behaviors in transient impressions during observation and participation of social events (Joseph, 2016).

social perceptions, actions, and emotional experiences are governed by a psychological intention to minimize deflections between fundamental sentiments and transient impressions that are inherited from the dynamic behaviors of such interactions.


Hence, the proposed stock embedding represents the price as well as the semantics of the text, as we acquire it by training on both news articles and stock prices.

our stock embedding is trained through a binary classification problem, namely, whether a stock price goes up or down in comparison with the previous day's price.


This setting enables learning of the correlation among stocks, in addition to avoiding overfitting and the problem of small sample sizes.

the classifier is shared across all stocks, thus achieving a sample size about 50 to 100 times larger.


At the beginning of each year, given some expected gain E, the portfolio was computed by using all news articles and historic prices until the end of the previous year.

for each year, the training set in the experiment consisted of samples strictly earlier than those constituting the test set.


While a reader could not know that the visitor is the same individual as the author when reading the letter above, we treat all mentions as coreferent if a reader can determine that they are identical at any point in the narrative.

we annotate from the perspective of the textual reality, rather than from the reader's state of knowledge at the moment of encountering the mention to be resolved.


Firstly, teacher-student learning can probably help to learn label preferences for some specific words in the target language.

if a word appears in the unlabeled target-language data and the teacher model consistently predicts it to be associated with an identical label with high probabilities, the student model would learn the preferred label w.r.t that word, and predict it in cases where the sentence context may not provide enough information.


Pre-trained Transformers are now ubiquitous in natural language processing, but despite their high end-task performance, little is known empirically about whether they are calibrated.

do these models' posterior probabilities provide an accurate empirical measure of how likely the model is to be correct on a given example?


In order to arrive at our goal of data set equivalents of KORE 50 for YAGO, Wikidata and Crunchbase, some work was necessary prior to the manual annotation of input documents.

we had to prepare the KORE 50 data set for the annotation by extracting all sentences from the data set.


On the other hand, we observe that the community votes of each answer can be a valuable numerical indicator reflecting its veracity.

in PQA forums, each answer can receive upvotes and downvotes from the former buyers.


To investigate the effectiveness of our proposed evidence ranking strategy, we substitute it with two possible alternatives and present the results in Table  5.

we first report the results with QA pair only ("QA only") as a base model.


However, here we expand upon the recently introduced task of unsupervised question answering (Lewis et al., 2019) to examine the extent to which synthetic training data alone can be used to train a QA model.

we focus on the machine reading comprehension setting in which the context is a given paragraph, and the QA model can only access this paragraph to answer a question.


We focus mainly on factoid QA; the question concerns a concise fact.

we emphasize questions whose answers are named entities, the majority type of factoid questions.


We evaluate the proposed model on three benchmark task-oriented dialog datasets: the Cambridge Restaurant (CamRest676) (Wen et al., 2017b), Stanford In-Car Assistant (In-Car) (Eric et al., 2017) and MultiWOZ (Budzianowski et al., 2018), with 676/3031/10438 dialogs respectively.

multiWOZ is one of the most challenging dataset up-to-date given its multi-domain setting, complex ontology and diverse language styles.


In the sentence, the spans highlighted in red are aspect terms and the spans in blue are opinion terms.

the tag A represents that the two words of word-pair (wi , wj ) belong to the same aspect term.


We proposed ESD, a context-aware definition generation model that explicitly models the decomposed semantics of words.

we model the decomposed semantics as discrete latent variables, and training with auxiliary losses to ensure that the model learns informative latent codes for definition modeling.


This motivates us to explore font associations with regular users in a crowd-sourced setting.

we investigate how users relate fonts to different characteristics of the input text.


Instead we aim to train all human's embeddings to be neutral with respect to sensitive attributes.

we wish to make it impossible to predict, for example, a person's gender, from their embedding.


As a second method of measuring the extent to which sensitive information remains in the trained embeddings, we train a feedforward neural network to try and predict the attributes of a human entity from their embedding alone.

the input to the network is the embedding of dimension d, and the output is a softmax distribution over labels (male and female for gender).


However, the results imply that gestures can be employed to predict audience response to some extent as well.

head movements seem to be good indicators.


We now fully specify our approach by describing our model, how we find pairs in the unlabeled pool D train all to query, and how we choose the seed set B 1 .

a key technical challenge is that the set of training pairs D train all is too large to enumerate, as it grows quadratically.


To address the issues, we propose meta graph learning (MGL), a meta learning framework to learn how to cross-lingual transfer for mPLM.

mGL models each CLT process as heterogeneous information propagation over a dynamic graph, which captures latent language correlations and makes the downstream CLT adaptation more interpretable.


Taking the AMR in Figure 1(a) as an example, a model may produce "the girl wants the boy to go", which conveys an opposite meaning to the AMR graph.

this can be very likely if "the girl wants" appears much more frequent than "the boy wants" in the training corpus.


We proposed reconstructing input graphs as autoencoding processes to encourage preserving the input semantic information for graph-to-text generation.

the auxiliary losses for recovering two complementary views (triple relations and linearized graph) of input graphs are introduced, so that our model is trained to retain input structures for better generation.


For each input phone, one can also define a binary measure of focus at level l, which checks that the focus of the frames at that level has not shifted to an input phone other than the one whose frames it corresponds to.

this binary focus measure is 1 if the focus of the phone at a level as defined above is larger than the contribution from the input frames of every other phone.


Finally, we showcase how the structure of OneStopQA annotations can be used for detailed comparisons between human and machine readers.

we demonstrate that human subjects and a state-of-the-art machine reading comprehension model have similar distributions of erroneous answers, suggesting a deeper link between human and machine readers than previously reported.


We observe that the two weaker models, Sliding Window and Stanford AR, perform better on RACE than on OneStopQA.

notable is the large drop in the performance of Stanford AR from 42.8 on RACE to 34.3 on OneStopQA (p << .001, ttest).


We use datapoints from Github repositories, where each bimodal datapoint is an individual function with paired documentation, and each unimodal code is a function without paired documentation.

we use a recent large dataset  provided by Husain et al. (2019), which includes 2.1M bimodal datapoints and 6.4M unimodal codes across six programming languages (Python, Java, JavaScript, PHP, Ruby, and Go).


Although the pre-training objective of Code-BERT does not include generation-based objectives , we would like to investigate to what extent does CodeBERT perform on generation tasks.

we study code-to-NL generation, and report results for the documentation generation task on CodeSearchNet Corpus in six programming languages.


We focused on only two of the five classes of TPs and FPs.

we focused on the two classes that can help us measure the model's precision.


HK and TW literature consists of popular books for which many movie and drama adaptations are made.

for HK literature, the text contains code-mixed characters with Vernacular Cantonese, which is quite unusual in formal publishing practices, and these books are often cited as an example for popularizing Cantonese in the 60s (Snow, 2004).


Here, we provide further details on how we applied our methodology to the dataset of counseling conversations in order to measure the orientation of counselor utterances, as described in Section 5.

we list empirical choices  made in extracting and then processing the training set of 351,935 texter and counselor messages used to measure phrasing orientations.


Here, we provide further details about the subset of data we used to analyze counselors' orientation behavior (Section 5.4).

our aim was to characterize behavior in typical conversations rather than exceptional cases or those that reflected earlier versions of the training curriculum.


This can be explained by the geometric similarity between the embedding spaces of the two languages.

we measure the geometric similarity of the language pairs using the Gromov-Hausdorff (GH) distance (Patra et al., 2019), which is recently proposed to quantitatively estimate isometry between two embedding spaces.


We further analyze our model by dissecting it and measuring the contribution of its different components.

our goal is to assess the contribution of back-translation, reconstruction, nonlinearity in the mapper, and non-linearity in the autoencoder.


Obtaining salient words by applying existing saliency identification approaches (Ribeiro et al., 2018) is, however, unable to produce unified action representations.

system utterances with the same intention might not share similar wordings, and existing attribution approaches can only identify salient words within utterances.


Based on the learned state tracking model, a straightforward idea of obtaining salient words is to apply importance attribution approaches.

these approaches measure the importance of each word by observing the prediction difference caused by replacing it (Ribeiro et al., 2018; Jin et al., 2020).


To address this issue, we adopt a denoising training strategy inspired by unsupervised machine translation (Lample et al., 2018(Lample et al., , 2019, and obtain a DST model that is more robust to the attribute transformation.

we apply a noise function g(x) to the utterances, and modify the DST model training loss as: where the noise function corrupts the input utterance by performing word drops and word order shuffling as specified in Lample et al.


In the cross-domain setting, we adopt a leave-one-out approach to evaluate the generalization ability via a more challenging few-shot learning task.

we use one domain as low-resource target domain (with only 1% of dialogues are available for training) while the others as source domains.


We consider mainly two types of content planning model that works on natural language actions: action decoder and classifier, denoted as Act-DEC and Act-CLS, respectively.

an action decoder generates a text span and feed it to the language generation model, while action classifier conducts classification to select one action from the action set given by the training set.


We create an auxiliary reading comprehension (RC) task from these documents.

if a document contains both problem and solution sections, a synthetic example is created where the problem description section is the query, the solution section is the target answer, and the entire document excluding the query section is the context.


Although ROBERTA's pretraining corpus is derived from multiple sources, it has not yet been established if these sources are diverse enough to generalize to most of the variation in the English language.

we would like to understand what is out of ROBERTA's domain.