Robustness to noise was only achieved by training with the same distribution-at the expense of performance degradation on other types of noise.

our method performed well on natural noise at test time by using a simplified synthetic noise model during training.


The field has largely moved toward probabilistic decoding strategies that randomly sample from the output distribution token-by-token.

when many low-likelihood words cumulatively contain quite a bit of probability mass, choosing one of these words can lead to odd or contradictory phrases and semantic errors.


The verb catch also has a number of possible senses, which can depend on the nature of the object, e.g., a body part object (caught his foot) "selects" the meaning of "becoming entangled".

there is still a difference between this and the more idiomatic lost his footing; caught would have the same interpretation whichever body part is chosen, and this interpretation is probably the default one in a construction context.


We found that "core" construction activities, like drilling, dismantling, lifting or offloading generally achieve high levels of agreement.

other "supporting" ac- Unexpected/unintentional and (potentially) harmful accident or incident occurring during construction activities.


Third, in terms of the AUC metric, the performance of news recommendation methods on unseen users is slightly lower than that on overlap users which are included in training data.

the performance on both kinds of users in terms of MRR and nDCG metrics has no significant difference.


As the results in Table 1 show, the current implementation that uses DBpedia Spotlight instead of KanDis for entity linking step, on average achieves a 5% lower accuracy on the unsupervised binary task than the original implementation (Hulpus , et al., 2019).

it achieves a 4% better accuracy on the binary classification between original articles and their first level of simplification (0-1).


Smooth continuation or just passing the time by chatting is the "goal" of this type of dialogue.

the task-oriented dialogue has a specific goal for dialogue participants to achieve together through dialogue.


In many tasks that are submitted to the crowdsourcing system, each worker works alone; therefore, they can do the task at any time at any place they want.

when we collect human-human dialogues, a pair of workers must be online at the same time.


The aspect of dialogue collection was gamified in these games.

they cannot use their games for other tasks.


is no significant difference in dialogue time and utterance numbers between the gamified and plain versions.

the utterance contents are different between them.


If the interpretation is faithful, then the curve will drop rapidly, resulting in a small area under a curve (AUC).

replacing the token with zero or removing it from a sentence can cause the OOD problem again.


For predicting a class entailment, the token with a similar meaning were assigned high attribution score, such as "swim" (b3).

tokens that makes two sentences contradicting were highlighted for predicting contradiction, such as "cafe" vs. "bar" (b4).


Stance detection has been studied on various types of formal texts such as congressional debates (Thomas et al., 2006) and company-internal discussions (Murakami and Raymond, 2010).

like most recent related work on the topic, we are particularly interested in informal texts from online social media.


Word sense disambiguation is required for these cases, which is known to be very challenging for verbs.

a potential solution could be to annotate VerbNet frames with effects, but this is outside the scope of this work.


The assumption might be true in human educational settings where prescriptivism is preferred over descriptivism because the goal is to test humans with well-defined knowledge or norms (Trask, 1999).

it is not true for many NLP tasks due to their pragmatic nature where the meaning of the same sentence might differ depending on the context or background knowledge.


To be specific, the estimated collective human performance gives JSD and KL scores far below 0.1 on all three sets.

the best JSD achieved by the models is larger than 0.2 and the best KL achieved by the models barely goes below 0.5 across the table.


A shortcoming of these methods is that they consider all occurrences of a term in the corpus when calculating its representation, including many contexts that are irrelevant to the concept at hand due to polysemy, noise in the corpus or noninformative contexts.

the pattern-based approach consid-ers specific indicative patterns that signal the desired concept, looking for them in a large corpus, and extracting the terms that appear in them.


If the input contains two sets of information in quotes, the program will assume it is receiving both Ham-NoSys symbols and their glosses.

if the input only contains one set of information in quotes, it will assume it is just receiving HamNoSys symbols


The previous challenge involved literal language, amenable to symbolic execution.

non-literal language is pervasive in everyday speech (Lakoff and Johnson, 1980).


The performance is impressive, with some tasks yielding lower perplexity than BERT pre-training + single task finetuning.

it still lags significantly behind fine-tuning applied after pushshift.io Reddit pretraining.


This means that multi-tasking across many tasks helps transfer learning.

the gap between zeroshot performance and multi-task or fine-tuning performance means there is still a significant challenge in improving these results.


DFGN (Xiao et al., 2019) and SAE (Tu et al., 2019) construct entity graph through named entity recognition (NER).

to the above mentioned models, our SRLGRN builds a heterogeneous graph that contains a document-level graph of various sentences and replaces the entity-based graphs with argumentpredicate based sub-graphs using SRL.


Several deep SRL models achieve highly accurate results in finding argument spans (Zhou and Xu, 2015;Tan et al., 2018;Marcheggiani et al., 2017;He et al., 2017).

those models are evaluated based on given gold predicates.


SOCC was recollected to study different aspects of on-line comments such as the connections between articles and comments; the connections of comments to each other; the types of topics discussed in comments; the nice (constructive) or mean (toxic) ways in which commenters respond to each other; and how language is used to convey very specific types of evaluation.

the main focus of the annotation is oriented toward the study of the constructiveness and evaluation in the comments.


In the anamnesis reports, 1,079 sentences (35.20%) were found to contain negations out of 3,065 sentences.

1,219 sentences (22.80%) out of 5,347 sentences were annotated with negations in the radiology reports.


Negation cue detection and scope identification are the tasks for which there are more corpora.

it is noteworthy that in some of the corpora (BioInfer, Genia Event, Product Review, EMC Dutch, IxaMed-GS, and German negation and speculation corpus) negation cues have not been annotated, despite the fact that the cue is the element that denotes the presence of negation in a sentence and the one to which the rest of the elements (scope, event, and focus) are connected.


UHU-HUVR and IULA Spanish Clinical Record corpora could also be merged for the identification of lexical cues.

we cannot merge corpora in their actual form because, as we have analyzed before, the annotation formats and guidelines are different.


As shown in Table 1, a significant fraction of the questions posted in the two forums have answers that were accepted by the person who asked the question (accepted answers).

the majority of these Accepted Answers rely on the question or on fuller forum discourse history and are not good stand-alone candidates for a MRC dataset.


Especially, identifying when public drifts occur is of great importance to different stakeholders, such as government agencies, companies and news agencies, where they can take proactive actions to avoid damages and pay close attentions to new topics/sentiments etc (Hu et al., 2017).

the dynamic nature of drifts/changes makes it a challenging problem, although concept drift analysis can be applied to focus on detecting variation of data distributions over time.


This indicates that the entity specific model architecture is more efficient in capturing the sort of information required for this knowledge probing task than the general encoderdecoder architecture used by T5.

when T5 is enhanced with an extra pre-training steps focusing on likely answer spans from Wikipedia (T5-11B + SSM) its performance leapfrogs that of EAE.


In addition to BERT's pre-training objective, ERNIE also masks entities and trains the model to predict them.

with both KNOWBERT and EAE, ERNIE takes entity linking as an input rather than learning to do it inside the model.


Back Translation: With the same monolingual corpus, MTL achieves better performance on some language pairs (e.g. Fr→En, Gu→En), while getting outperformed on some others, especially on the En→X direction.

back translation is computationally expensive as it involves the additional procedure of training 10 bilingual models (20 for the X→X system) and generating translations for each monolingual sentence.


Existing question answering datasets focus on dealing with homogeneous information, based either only on text or KB/Table information alone.

as human knowledge is distributed over heterogeneous forms, using homogeneous information alone might lead to severe coverage problems.


This gap suggests the necessity to aggregate heterogeneous information in HybridQA.

the hybrid model's score is still far behind human performance.


From Table 5, we can clearly observe that using either Table-Only or Passage-Only model achieves a poor accuracy below 20%.

the proposed HYBRIDER can achieve up to 50% EM increase by leveraging both structured and unstructured data during reasoning.


These models simulate a 'fake' KB-incomplete scenario by masking out triples from KB.

the questions in HYBRIDQA are inherently hybrid in the sense that it requires both information forms to reason, which makes our testbed more realistic.


For this, human interpreters usually start translation before the source sentence ends.

this makes the translation process much more challenging than the full-sentence translation, because to balance the translation quality and latency, interpreters need to make decisions on when to continue translation and when to stop temporarily to wait for more source side information, which are difficult, especially for syntactically divergent language pairs, such as German and English.


At each step, there is a wait-k policy synchronizing the adaptive policy, meaning that they have the same lag at that step.

specifically, at any step t, if the lag of the adaptive policy is k , then we apply the NMT model with the wait-k policy and force it to predict existing target tokens until step t, when the model will make a new prediction as the output of step t. the above method only shows how to simulate the adaptive policy to make a prediction at one step if we would like to write at that step, but it does not tell us at which steps we should write.


If we have a set of models trained independently with different wait-k policies, then we can apply ensemble of those models (Dietterich, 2000; Hansen and Salamon, 1990) to improve the translation quality, which is also used to improve the translation quality of full-sentence translation (Stahlberg and Byrne, 2017).

there may be two issues to apply ensemble of all models: (1) the runtime for each prediction could be longer, resulting in higher latency; and (2) the translation accuracy may be worse, for the best model for one policy may give bad performance when doing inference with another policy.


Ranzato et al. (2016) propose MIXER algorithm, which is the first application of REINFORCE algorithm (Williams, 1992) in training sequence to sequence model.

an additional model, which is used to predict expected reward, is required in MIXER.


Knowledge inference on knowledge graph has attracted extensive attention, which aims to find out connotative valid facts in knowledge graph and is very helpful for improving the performance of many downstream applications.

researchers have mainly poured attention to knowledge inference on binary facts.


Natural language interfaces (NLIs) have been the "holy grail" of natural language understating and human-computer interaction for decades (Woods et al., 1972;Codd, 1974;Hendrix et al., 1978;Zettlemoyer and Collins, 2005).

early attempts in building NLIs to databases did not achieve the expected success due to limitations in language understanding capability, among other reasons (Androutsopoulos et al., 1995;Jones and Galliers, 1995).


Concurrent work on model CHECKLIST evaluation (Ribeiro et al., 2020) includes an invariance test which also examines model undersensitivity.

to CHECKLIST, our work focuses with more detail on the analysis of the invariance phenomenon, the automatic generation of probing samples, an investigation of concrete methods to overcome undesirably invariant model behaviour, and shows that adherence to invariance tests leads to more robust model generalisation.


On the one hand, this exercise provided a good opportunity to assess the added value of the ontology extraction method adopted and to give it an objective evaluation.

as the final Portuguese ontology evolved from a projection supported by a pre-existing ontology for English, this extraction exercise provided also a ground to gain insight on the similarity of two lexical semantics-based ontologies, for the same semantic domains, but with terms from two different languages, Portuguese and English.


The prediction accuracy (microaveraged recall) on seen intents may be reduced compared to ReCapsNet and other baselines, since some utterances of seen intents are classified to unseen intents.

the F1 score on seen intents increases significantly, indicating that it has much higher precision than that of the baselines.


Mutual strengthening of opinions: a major cause of disagreement seems to be the piling up of opinions that make each other stronger and therefore better to recognize.

each single expression of such a concatenation can be easily treated differently by the annotators.


Meng et al. (2017) applied a Seq2seq model with a copy mechanism to a keyword extraction task.

models without explicit modeling the sentence planning have a great limitation in generating complex argument structures depending on hierarchy.


Our method is parallel to SLKS because we also utilize the previously selected knowledge.

we explicitly compute the difference between knowledge selected at different turns, while SLKS only encodes the already selected knowledge in an implicit way.


As a result, PostKS++ generates a response which is not coherent to the previous context at the 3 rd turn.

our two models select both diverse and appropriate knowledge sentences at all the turns, thereby generating informative responses and making the dialog coherent and natural.


This makes it difficult to scalable for a new domain with limited labeled data.

there has been relatively little research on how to effectively use data from all domains to improve the performance of each domain and also unseen domains.


As shown in Figure 2(b), The second strand of work (Wen et al., 2018; Qin et al., 2019b) trains model separately for each domain, which can better capture domain-specific features

those methods ignore shared knowledge between different domains (e.g. the location word exists in both schedule domain and navigation domain)


The method explicitly differentiates shared and private knowledge.

this framework still has two issues: (1) given a new domain with extremely little data, the private module can fail to effectively extract the corresponding domain knowledge.


Their works consider only domain-specific features.

our framework explicitly leverages domain-shared features across domains.


On the one hand, we train the domain classifier to minimize the domain classification loss.

we update the parameters of the network underlying the domain classifier to maximize the domain classification loss, which works adversarially towards the domain classifier.


As shown in the table, the accuracy of attention methods decreases significantly when we remove attention weights other than those deemed active by the threshold λ.

our model retains high accuracy even with just the active alignments since sparsity is naturally modeled in our contrained optimal transport framework.


For instance, the compositional meaning of the verb-object pair break the ice could be paraphrased as "clearing the ice from the sea so that ships could pass".

the same expression also has an established sense, "to overcome initial stiffness or reserve in a social setting" (Ammer, 2013).


Unlike measuring embedding bias, measuring classification bias is difficult: most NLP datasets are not annotated with protected attributes, precluding the use of standard fairness measures such as equal opportunity (Hardt et al., 2016).

manually annotating a large dataset with a protected attribute is slow and expensive.


Graph representations of documents such as similarity graph based on lexical similarities (Erkan and Radev, 2004) and discourse graph based on discourse relations (Christensen et al., 2013), have been widely used in traditional graph-based extractive MDS models.

they are not well studied by most abstractive approaches, especially the end-to-end neural approaches.


It can be observed that the proposed learning approach provides positive impacts across most of the target languages.

including Swahili (sw) as an auxiliary language in X-MAML is not beneficial for the performance on the other target languages.


Recently, deep neural networks have been used to tackle the problem (Bamman et al., 2019), and related problems such as generating coherent and cohesive text (Cho et al., 2019) and identifying relations in generated stories (Roemmele, 2019) have also been addressed.

these studies only focused on how to understand a narrative itself (e.g., how to extract information from a narrative).


This is a step towards guided response generation and bears some similarities with our study.

a narrative is more general than keywords, and it provides a description of the dialogue session rather than imposing keywords to the next response.


3) When we remove context-narrative matching, the performance drops too, indicating that context-narrative matching may provide implicit and complementary information for controlling the alignment of response and narrative.

4) when we remove the contextresponse matching, the performance also drops, however, at a much smaller scale, especially on P weak , than when narrative-response matching is removed.


For a similar purpose, Huang and Carley (2019) used graph attention networks (GAT) to explicitly establish the dependency relationships between words.

these approaches generally ignore the dependency relations which might identify the connections between aspects and opinion words.


Finally,  Fries et al. (2017) presented a weak supervision approach to NER in the biomedical domain.

unlike the model proposed in this paper, their approach relies on an ad-hoc mechanism for generating candidate spans to classify.


These methods rely on multiple classifiers run simultaneously and whose outputs are combined at prediction time.

our approach (as in other weak supervision frameworks) only requires labelling functions to be aggregated once, as an intermediary step to create training data for the final model.


Such approaches are useful in case the KG is incomplete.

this leads to another level of complexity in the QA system, and text corpora might not always be available.


Hence models that require question-specific sub-graph construction (GraftNet, PullNet) are unable to recall the answer entity in their generated sub-graph and therefore performs poorly.

their performance improves only after including additional text corpora.


Along with conversational intents, dialogue acts are also used for natural language understanding (NLU) in task-oriented systems (Li et al., 2019; Peskov et al., 2019).

to these prior approaches, our work uses more in-depth meaning representations for open-domain dialogue systems based on lexical conceptual structures (explained in Section 3.1).


In this work, we limit the types to GIVE, GAIN, LOSE, and PERFORM.

we do not restrict the ask action and target at all.


We can see similar trends of performance for RW based sequences in case of sentiment polarized node expansion also.

sentiment polarized node expansion strategically mitigates the problem of under-specified tweets by extending the tweet-network view to include less-noisy informative nodes so that the generated walks are more diverse and discriminating.


This combination (T+Filtered) outperforms the tweet only prediction by 3.9% -depicting nodes selected for expansion are important for inference.

as T+Biased without node expansion, T+Unbiased and T+Biased with sentiment polarized node expansion beat this T+Filtered by a margin of 1.8%, 2.7% & 6.4% accuracy respectively.


For SemEval-2016, a fascinating thing to observe is -Unbiased and Biased RW-based sequences almost give a comparable performance in terms of accuracy.

the Biased RW view consistently outperformed the Unbiased view in F-macro measure in both datasets for each of the cases of node expansion.


The most related work to Refer360° is Touchdown (Chen et al., 2018) which introduces two tasks: a vision and language navigation task and a spatial description resolution (SDR) task (i.e. a referring expression recognition task for a simulated outdoor environment).

with Touchdown, in our setup instructors, followers, and learning systems observe a partial FoV of the scene, but they can change the FoV continuously to explore the scene.


Following the Efficient Annotation of Scalar Labels framework (EASL; Sakaguchi and Durme, 2018), we present annotators 5 sentence-pairs, each with a slider bar enabling direct assessment for each pair and ask annotators to calibrate their score for a sentence-pair based on the scores they provided to the other four pairs.

to the uniform scale employed in the original EASL protocol, we modify the interface to allow finer-grained values near 0.0 and 1.0, following psychological findings that humans are especially sensitive to values near the ends of the probability spectrum (Tversky and Kahneman, 1981).


One method would be to train a categorical NLI model on SNLI and when fine-tuning on u-SNLI, replace the last layer of the network from a categorical prediction with a sigmoid function.

a typical categorical loss function would not take into account the ordering between the different categorical labels.


Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks.

the large memory footprint of word embeddings makes it challenging to deploy NLP models to memory-constrained devices (e.g., selfdriving cars, mobile devices).


This break-through performance facilitates the demand to deploy such models to embedded systems (e.g., self-driving cars, mobile devices).

the neural models typically require a large storage or memory footprint, which is a significant concern when deploying neural models to memory-constrained devices (Hinton et al., 2015).


When we only pretrain the compressing model, the large portion of words, around 80%, is assigned to the largest code-book (i.e., 128).

when we fine-tune the pre-trained models to the task, the ratio of the large one is significantly decreased.


Other studies for ontology embeddings  consider relative positions between spheres as the hierarchical relationships of corresponding concepts.

they are still limited to linear embeddings, hence may easily fall short of preserving the deep hierarchical structures of KGs.


This observation is generally in line with our expectations because a small dimension limits the expressiveness of KG embeddings.

hyperKA still exhibits satisfying performance at very small dimensions in comparison to other methods, such as under the dimensions of 10 and 25.


These elements contain potentially crucial information for problem resolution.

they cannot be correctly parsed by tools designed for natural language.


The author uses code tags in HTML sources of posts for supervised training of a character level sequence labelling model.

the code tags in the posts usually include all forms of non-natural language text like code snippets, command outputs, error messages or stack traces, and file paths (See Fig 2).


They also propose a dependency parser-based approach for extracting these attributes.

while this approach pays attention to the semantics of the problem, the syntactical idiosyncrasies are ignored.


To the best of our knowledge, no previous system for learning word embeddings has explicitly focused on the acquisition of this sort of knowledge; by contrast, ConceptNet is at the base of other projects concerned with the development of lexical resources (Mensa, Radicioni, and Lieto 2018) and their usage along with formal ontologies Rho 2015, 2017).

conceptNet is principally a lexical resource, and as such it disregards the conceptual anchoring issue: If we consider the term bat, the bat node in conceptNet mixes all possible senses for the given term, such as the nocturnal mammal, the implement used for hitting the ball, the acronym for "brown adipose tissue," an entity such as the radar-guided glide bomb used by the US Navy in World War II, and so forth.


For instance, starting from the term sunset eng we encounter the sense bn:08410678n (representing the city of Sunset, Texas).

this sense is provided with the following lexicalizations: out of these three terms only sunset eng actually appears in CNN, giving us a final singleton t + = {sunset eng }.


In spite of the simplicity of the system using LESSLEX embeddings, our results overcome those reported in literature, where by far more complex architectures were used.

such scores are higher than the agreement among human raters, which can be thought of as an upper bound to systems' performance.


A popular way to tackle NLP problems with overlap between input and output is to equip seq2seq models with a copying mechanism (Jia and Liang, 2016;Zhao et al., 2019;Chen and Bansal, 2018;Gulcehre et al., 2016;See et al., 2017;Gu et al., 2016), usually borrowing ideas from pointer networks  to point to single tokens in the source sequence.

we use pointer networks to identify entire spans that are to be copied which results in a much more compact representation and faster decoding.


Fact-checking -a task in which the veracity of an input claim is verified against a corpus of documents that support or refute the claim -has been studied to combat the proliferation of misinformation in political news, social media, and on the web (Thorne et al., 2018;Hanselowski et al., 2019).

verifying scientific claims poses new challenges to both dataset construction and effective modeling.


The conventional approach to CQA is to train one model to fit the entire training set, and then use it for answering all complex questions at the test time.

such a one-size-fits-all strategy is sub-optimal as the test questions may have diversity due to their inherently different characteristics (Huang et al., 2018).


There are different ways of explaining how machines make the decision (Ribeiro et al., 2016; Alvarez-Melis and Jaakkola, 2017;  Lee et al., 2019; Liu et al., 2019a), and one of these methods is to extract rationales  (Lei et al., 2016; DeYoung et al., 2019).

most prior work focused on extracting rationales in a supervised manner , but not all datasets contain such annotated rationales for model learning, making the rationalization task difficult and impractical.


The larger variance of baselines is due to rationalizing-specific training (Lei et al., 2016), which may cause instability when extracting rationales.

our model utilizes the generality from multi-task learning, which is expected to extract rationales in a more stable manner (lower variance), indicating that the proposed method generalizes to different aspects better than baselines.


Naturally, they provide critical clues to select the candidate news C 2 and C 3 which reveal relevant information.

they are less informative to identify the candidate news C 1 , which is about the competition of National Football League (NFL).


At the lexical level, it includes semantic tag annotations, which we rely on in our work.

due to the novelty of this task, the available annotations are limited in quantity and consist of a mix of gold and silver standard data.


A simple method is to concatenate multiple documents into a long flat text and treat it as a long sequence to sequence task.

it blurs the boundaries between documents and loses the hierarchy within the document cluster.


On the one hand there are works for cognate detection harnessing computational methods that propose the first step in a (semi-) automatic analysis of cognates using the vast amount of digitally available data, when manual annotation requires a lot of man-hours (Jager et al., 2017; List et al., 2018).

there are semantic analyses of cognates, that manually investigate cognates to look for links between two different languages (List et al., 2018;Aske, 2015).


A wealth of work has shown that toxicity and social biases in training data are acquired by large pretrained sentence encoders (e.g., gender bias in BERT; May et al., 2019;Zhao et al., 2019;Basta et al., 2019;Kurita et al., 2019).

fewer studies have investigated toxicity in autoregressive language models, whose generations also suffer from incoherence, blandness, and repetitiveness (Holtzman et al., 2020;Welleck et al., 2019).


Many researchers have studied previous (non-Transformer) models to understand their actual behavior under different scenarios, showing that these models are taking advantage of clues or failures of datasets and that slight perturbations on the input data can severely reduce their performance.

recent models have not been systematically tested with adversarial-examples in order to show their robustness under severe stress conditions.


On the one hand, Natural Language Inference (NLI), also known as recognizing textual entailment (RTE) which consists of finding semantic relations between a premise sentence and an associated hypothesis, by classifying if they are entailed, in contradiction or in neutral relationship.

we apply stress tests on a questionanswering (QA) task, also known as machine reading comprehension (MRC) which consists of predicting the answer to a question given a paragraph.


In the NLI task, we verified that the distraction test significantly reduces the performance of all models, especially in the negation test.

tests on noise examples show that models are somewhat robust, possibly because they were pre-trained in a huge corpus that may have had natural noise.


Most language-based methods for human attribute prediction assume all documents generated by a person are equally informative.


Several recent works (Liu et al., 2020; Soleimani et al., 2020; Zhao et al., 2020) leverage representations from large pre-trained language models (LMs) like BERT (Devlin et al., 2019), and RoBERTa  to achieve state-of-the-art results on FEVER.

it is unclear how factual knowledge encompassed in these LMs influences the verification process.


Data-driven approaches using neural networks have achieved promising performances in natural language generation (NLG).

neural generators are prone to make mistakes, e.g., neglecting an input slot value and generating a redundant slot value.


Recently, end-to-end neural network-based approaches have shown significant improvements over traditional pipeline-based models in English coreference resolution.

such advancements came at a cost of computational complexity and recent works have not focused on tackling this problem.


This makes it hard to locate the heads in the mentions because the head location is not deterministic.

having deterministic head locations is a very desirable linguistic trait for solving the aforementioned computational complexity issue.


In both these examples, the embedded clause (that global warming is serious) is presented as an opinion belonging to a source entity (scientists).

differences in the choice of predicate (agree vs. claim) and in how the source is described lead to very different interpretations.


Intuitively, it would be incompatible with real world events to describe Exxon as vocally denouncing fossil fuels or Al Gore as vocally criticizing climate science, but it is possible to describe such entities as silently holding contradictory beliefs (and in doing so, highlight their hypocrisy).

we also see exceptions (demonstrate in LL, reveal in RL), suggesting that more complex interactions are involved.


Finally, we experiment with using tweets from known GW activists/skeptics as well as GW article headlines taken from extreme liberal/conservative news sources as additional training data, inferring labels based on the stance of the Twitter user or news source.

we find that adding these examples yields a lower performance compared to using only the human-annotated data.


A wide range of computational models has been applied for extracting different forms of discourse structures.

across several tasks, neural network methods (Ji and Eisenstein, 2015; Becker et al., 2017) are found the most effective, with relatively superior performance obtained by modeling discourse-level context (Dai and Huang, 2018a,b).


On the one hand, loneliness—a negative and unwanted state of being alone—has been shown to be correlated with increased cognitive decline, dementia, depression, suicide ideation, self harm, and even death (GerstEmerson and Jayawardhana, 2015; Hawkley and Capitanio, 2015; Luo et al., 2012; Endo et al., 2017)

solitude-a positive and self-driven state of being alone-has been shown to improve autonomy, creativity, and well-being (Long et al., 2003;Knafo, 2012;Coplan and Bowker, 2017;Coplan et al., 2019a).


Proponents of self-determination theory (Deci and Ryan, 2010) postulate that time alone that is intrinsically motivated (i.e., choosing to spend time alone) is better for one's well-being than time alone that arises for external reasons (e.g., one who is alone due to the nature of their work) (Chua and Koestner, 2008;Nguyen et al., 2018).

the experience of being alone may also differ as a result of when this state arises.


These are words referring to peaceful and spiritual activities of being with oneself, recharging, and enjoying the present moment.

the words more strongly associated with loneliness than with solitude refer to negative personal experiences of being sad, scared, bored, hurt, and broken-hearted.


Similar percentage of male users is inferred in the solitude sub-corpora (52%).

in the lonely and loneliness sub-corpora the majority of the inferred users are female (56% and 52%, respectively).


There were 12% more tweets with the word lonely written by female users than tweets written by males, even though in the General Tweet Corpus (used as control) there were 10% more tweets written by male users.

the emotional content in the SOLO tweets written by male and female users was strikingly similar.


Table 8 shows an inference that relies on inferring the gender of an entity based on a possessive adjectives.

in French, gender inflection of possessive adjectives depends on the gender of the possessee instead of the gender of the possessor.


The dataset covers already more than 22 years of publications, including mode than 15 million entries, and it will be available for academics researches.

despite the full range of customization of neural networks, it is possible to achieve even better results.


Their elements are always pairs of conjunctions of atomic predications.

the TBox includes the semantic relations codified in PrOnto, e.g., the is-a relations between the ontological classes associated with the predicates, as well as definitions of other needed predicates, such as (3) and (4) above for handling negation and disjunction.


For the pre-training task of machine translation, we have chosen English-to-German (EN-DE) with 3.9M sentence pairs.

it is still unclear whether it is critical to choose the right language pair.


Since standard BPE produces single segmentation, to realize this regularization the author had to propose a new subword segmentation, different from BPE.

the introduced approach is rather complicated: it requires training a separate segmentation unigram language model, using EM and Viterbi algorithms, and forbids using conventional BPE.


Subword regularization was shown to achieve significant improvements over the method using a single subword sequence.

the proposed method is rather complicated and forbids using conventional BPE.


Thus there is a potential danger that models trained with BPEdropout may tend to use more fine-grained segmentation in inference and hence to slow inference down.

in practice this is not the case: distributions of lengths of generated translations for models trained with BPE and with BPEdropout are close (Figure 4 (b)).


So far, the only way to overcome this BPE imperfection, its deterministic nature, was to create another subword segmentation algorithm (Kudo, 2018).

we show that BPE itself incorporates the ability to produce multiple segmentations of the same word.


Many solutions for calculating semantic similarity, both topological and statistical, have already been developed for English.

few are designed that they can be adapted to under-resourced languages like Arabic language, because they often use advanced natural language processing techniques which are specific to a language.


In 54.14%(1155 answers), the difference is at most one point.

in 11.01%(235 answers), the difference is more than one point.


Ideally, using word embedding as the learning target via minimizing CD can effectively train the decoder to model the semantic relation existing in the embedding space.

such an approach suffers from the hubness problem (Faruqui et al., 2016) of word embedding in practice (as we later discuss in Sec. 4.5).


Recently, neural sequential models (Lample et al., 2016; Akbik et al., 2018; Vaswani et al., 2017) have shown strong performance for various sequence labeling task.

these deep neural models are label hungrythey require large amounts of annotated sequences to achieve strong performance.


We aim to generate augmented labeled sequences for the queried samples in each iteration, thereby introducing more data diversity and improve model generalization.

data augmentation for active sequence labeling is challenging, because we need to generate sentences and token-level labels jointly.


Several other studies have independently proposed different setups for probing varied aspects of explainability techniques (DeYoung et al., 2020; Sundararajan et al., 2017).

existing studies evaluating explainability methods are discordant and do not compare to properties from previous studies.


Thus, when a model recognizes a highly indicative pattern of the predicted class k, the tokens involved in the pattern would have highly positive saliency scores for this class and highly negative saliency scores for the remaining classes.

when the model is not highly confident, we can assume that it is unable to recognize a strong indication of any class, and the tokens accordingly do not have high saliency scores for any class.


In Figure 8 (a) our approach finds that the word many is salient and perturbing it slightly can make the NN change the class of the review to negative.

lIME does not identify many as significant.


Our SeNsER can effectively learn the common features of different buildings such as temperature and equipment operating status.

the building names vary a lot in different buildings and share no similar features; for example, recall the examples in Table 1, the building name phrases are EBU3b, ap&m, and SDH.


The new Audio Span Features alone demonstrate really poor performance and are missing almost all the disfluency events (Detection Recall and F1 lower than 0.001).

the Audio Span Features along the acoustic and prosodic representation show the best performance on the frame-based system, especially in the identification task (Identification F1 0.118 and Error Rate below 1).


Clearly, explicit incitement, of the type expressed by an utterance of "kill all [members of a minority group]" or "let's make sure that all [members of a minority group] do not feel welcome," is not very difficult to discern.

since "most contemporary societies do disapprove of" such invocations, "overt prejudicial bias has been transformed into subtle and increasingly covert expressions" (Leets, 2003, p.146).


The recent years have seen advances in deep learning methods for text matching (Mueller and Thyagarajan, 2016;Gong et al., 2017;Chen et al., 2017;Lan and Xu, 2018).

almost all of these models are initially proposed for English text matching.


Although character-based models can overcome the problem of data sparsity to some degree (Li et al., 2019), the main drawback of these models is that explicit word information is not fully exploited, which can be potentially useful for semantic matching.

word-based models often suffer some potential issues caused by word segmentation.


This allows the interactive relations among different subtasks to be modeled explicitly for the joint training methods.

none of existing studies along this line has fully exploited the power of such relations.


Most results in Table 2 are within a reasonable range of the results reported in the original article.

some large discrepancies occur in some of the experiments.


Regardless of the source of the problem, a mismatch between the results from the software and the publications is not desirable, but there is little we can solve with exact replications.

code and data availability is also crucial for reproduction.


Previous research has investigated such persuasive effects for argumentative content.

this paper studies how important the style of news editorials is to achieve persuasion.


We find that the model trained on backtranslated data is often conservative choosing to leave many input sentences almost unchanged.

as shown by examples in Table 6, the model learned useful transformations by compressing long sentences and utilizing vocabulary adapted to the language of press releases (e.g. �the researchers� in the last example).


Considering that the structures of the monolingual embedding spaces are closely related to the choice of the context window, it is natural to expect that the context window has a considerable impact on the performance of mapping-based bilingual word embeddings.

most existing work has not provided empirical results on the effect of the context window on cross-lingual embeddings, as their focus is on how to learn a mapping between the two embedding spaces.


Neural network models have achieved remarkable performance on text classification due to their capacity of representation learning on natural language texts (Zhang et al., 2015;Yang et al., 2016;Joulin et al., 2017;Devlin et al., 2018).

the lack of understanding of their prediction behaviors has become a critical issue for reliability and trustworthiness and hindered their applications in the real world (Lipton, 2016;Ribeiro et al., 2016;Jacovi and Goldberg, 2020).


Particularly in each iteration, the first term in Equation 9 is approximated with a single sample from q(R|x (i) ) (Kingma and Welling, 2014).

sampling from a Bernoulli distribution (like from any other discrete distributions) causes difficulty in backpropagation.


Moreover, structured data can also test a model's ability for reasoning and numerical inference (Wiseman et al., 2017) and for building representations of structured objects (Liu et al., 2018), providing an interesting complement to tasks that test these aspects in the NLU setting (Pasupat and Liang, 2015;Chen et al., 2019;Dua et al., 2019).

constructing a data-to-text dataset can be challenging on two axes: task design and annotation process.


The simplest approach is simply to use the entire table as the source, adding special tokens to mark which cells have been highlighted

many tables can be very large and this strategy performs poorly.


This hallucination phenomenon has been widely observed in other existing data-to-text datasets (Lebret et al., 2016;Wiseman et al., 2017).

the noisy references in these datasets make it difficult to disentangle model incapability from data noise.


The hidden Markov model (HMM) is a fundamental tool for sequence modeling that cleanly separates the hidden state from the emission structure.

this separation makes it difficult to fit HMMs to large datasets in modern NLP, and they have fallen out of use due to very poor performance compared to fully observed models.


Previous works usually apply beamsearch-based methods or stochastic searching methods to lexically-constrained generation.

when the search space is too large, beam-search-based methods always fail to find the constrained optimal solution.


In cases where the datasets have been a part of shared tasks, we report the highest scores obtained in each task as the State Of The Art (SOTA) for the dataset.

note that we report this to situate our results in context of the same, and these cannot be directly compared, since each task's SOTA is obtained by varied training architecture, suited to perform well in one particular task alone.


The difference between these two tasks is that zero pronoun resolution focuses on resolving anaphoric pronouns to their antecedents assuming the position of the dropped pronoun is already known.

in dropped pronoun recovery, we consider both anaphoric and non-anaphoric pronouns, and attempt to recover the type of dropped pronoun but not its referent.


Existing work has focused on modeling referent semantics of the dropped pronoun from context, and globally optimizing the prediction sequences by exploring label dependencies.

there is also something need to do about how to recover the referent as a proper pronoun based on the referent semantics.


Similarity defined as association, i.e., the mental activation of a term when another is presented (Chiarello et al., 1990; Lemaire and Denhiere, 2006), e.g., knife-murder, has been estimated in terms of frequency of co-occurrence of words in language (and the physical world) (Turney, 2001; Turney and Pantel, 2010; McRae et al., 2012; Bruni et al., 2012).

to associative relatedness, a concept of semantic similarity defined in terms of shared superordinate category (Lupker, 1984; Resnik, 1995) (taxonomical similarity (Turney and Pantel, 2010)) or shared semantic features (Tversky, 1977; Frenck-Mestre and Bueno, 1999; Turney, 2006) has been proposed.


We observe that our Phase 1 set-up tends to encourage placing antonymous words in the same broad groups, based on their relatedness (e.g., antonymous pairs stay and leave, and lose and gain end up clustered together).

in Phase 2 antonyms are predominantly kept apart: out of 67 antonymy pairs shared with SimVerb (i.e., pairs labeled ANTONYMS in SimVerb), only 2 are placed closer in the arena (inhale - exhale and sink - swim).


Note that these are not bigram, trigram, and quadgram patterns in "true" sense as one of the two activations is dropped during max-pooling process with a pool size of 2 after every CNN layer.

on forwarded high activations, the notion of bigram, trigram, and quadgram holds true.


These results substantiate the findings on coarse-grained classification task which suggest that instead of training embeddings, using existing pre-trained embeddings by fine-tuning them on the task in hand is a more perceptive choice.

a carefully tailored model on top of these embeddings is advantageous.


Pre-trained language models have achieved huge improvement on many NLP tasks.

these methods are usually designed for written text, so they do not consider the properties of spoken language.


Our annotators collectively assigned it a low score of 24%.

hTER would miss-classify it as a good translation since there is only one token that requires post-editing.


Unsurprisingly, supervised models are more accurate than weakly supervised ones.

training supervised models is still challenging: both x and y are structured, so models typically generate y in multiple steps, but the training data cannot reveal which parts of x generate which parts of y and how they are combined.


In other words, use direct references to the presented row order id as little as possible.

use id if the question explicitly asks about the presentation order, e.g., "the first on the list" or "the first listed".


The outcomes of the study show that higher-dimensional sentence representations improve translation quality and also the performance in classification tasks.

shorter sentence representations increase the accuracy in non-trainable similarity tasks.


The proposed QG method added to the BERT-large QA model enjoys a performance boost of about 10 to 20 points, depending on the datasets, with an addition of only about 100 pre-labeled instances.

it is sufficient to add only 1K pre-labeled instances to reach the performance level of the supervised models, where its performance begins to level off.


Knowledge-driven conversation approaches have achieved remarkable research attention recently.

generating an informative response with multiple relevant knowledge without losing fluency and coherence is still one of the main challenges.


We assume an alignment between the anchor words and the story sentencesthe i th anchor word corresponds to the i th sentence in the story.

stories do not naturally occur with a tagged set of such anchor words or keywords.


The training data for this T2is identical to T1.1with both entity mentions and relations manually annotated (D1.1).

the test data only contains entity mentions without relations between them.


There were a few places in the RHZ that were indicating 19 as "maximum sentences length", which can be interpreted as the count includes tokens from entities.

we decided to count excluding the entity tokens as described in the section 2.3 of the RHZ paper.


Due to scarcer annotated data, the pure data-driven baseline method (TACOLM) falls behind the statistical learning one (i.e. StructLR) with comprehensively designed features.

our model successfully complements the insufficient supervision signals, partly by incorporating linguistic and commonsense knowledge.


It is shown that negative training effectively reduce probability mass assigned to malicious targets, while keeping the behavior on the test-set unchanged.

almost every word in the malicious target sentences gets lower probability, especially when FWA is not used.


Collecting a set of explanations E requires additional effort-it took the authors about 1 minute or less to construct each explanation, though we note that it only needs to be done once per dataset (not per example).

collecting a small number of explanations can significantly and disproportionately reduce the number of labeled examples required.


The results are dataset-specific: random explanations help on Spouse but not on Disease.

in both cases, random explanations do significantly worse than the original explanations (Table 4).


The ability to incorporate prior knowledge of the "right" inductive biases into model representations dangles the prospect of building models that are more robust.

more work will need to be done to make this approach more broadly applicable.


Numbers of data augmentation methods have been proposed to alleviate the dependence on labeled data.

current augmentation approaches such as random insertion or repetition fail to resemble training corpus well and usually resulted in unnatural and limited types of disfluencies.


Other neural baselines were able to achieve reasonable BLEU and Sent Acc.

their results could not serve as augmented data to pretrain sequence tagging models, since there was no indication where were the disfluent segments in output sentences.


Previous studies for joint CWS and POS tagging mainly follow the character-based tagging paradigm with introducing contextual information such as n-gram features or sentential representations from recurrent neural models.

for many cases, the joint tagging needs not only modeling from context features but also knowledge attached to them (e.g., syntactic relations among words); limited efforts have been made by existing research to meet such needs.


Some previous studies (Huang et al., 2007;Jiang et al., 2009;Wang et al., 2011; verified the idea for this task by learning from autoprocessed corpora.

their studies treat auto-processed corpora as gold reference and thus are unable to distinguishingly use it according to its quality (the resulted knowledge is not accurate in most cases).


If a model is syntactically uninformed, we would expect "media" and "on" to have comparable impacts on the prediction of "transitions", and vice versa.

we observe a far greater impact (darker color) between "media" and "transitions" than that between "on" and "transitions".


The above findings prove that BERT has learned its own syntax as a by-product of self-supervised training, not by directly copying any human design.

giving the superior performance of BERT on downstream tasks, it is natural to ask if BERT is learning an empirically useful structure of language.


To ensure a fair comparison with them, we also introduced this right-branching bias.

our results show that our method is also robust without this bias (e.g., only 0.9 F1 drops on PTB23).


In particular, we observe a decent accuracy in identifying discourse relations between adjacent EDUs, perhaps due to the "next sentence prediction" task in pre-training, as pointed out in (Shi and Demberg, 2019).

our probes fall behind the left-chain baseline, which benefits from its strong structural prior 4 (principal clause mostly in front of its subordinate clause).


Their results show that the BERT model captures syntax-sensitive agreement patterns well in general.

subject-verb agreement cannot provide more nuanced tests of other complex structures (e.g., dependency structure, constituency structure), which are the interest of our work.


Recent efforts (Lu et al., 2018;Schwartz et al., 2019) propose to leverage the semantics of class name to enhance class representation.

different from us, these methods focus on image classification where effects of name semantic are implicit and label dependency is not required.


Few-shot learning in natural language processing has been explored for classification tasks, including text classification Geng et al., 2019;Yu et al., 2018), entity relation classification Gao et al., 2019;Ye and Ling, 2019), and dialog act prediction (Vlasov et al., 2018).

fewshot learning for slot tagging is less investigated.


The GitHub README describes the code as being based on the study we aim to reproduce.

it implements a different training scheme.


Adding the CamemBERT embeddings always increases the performance of the model LSTM based models.

as opposed to adding ELMo, the difference with/without CamemBERT is equally considerable for both the LSTM-seq2seq and LSTM-CRF.


These methods use a graph state long short-term memory (LSTM) network, gated graph neural network (GGNN), or graph convolution network (GCN) to encode AMR graphs directly, and they can explicitly utilize the information provided by the graph structure.

these graph encoders still cannot significantly outperform sequence encoders.


They first use a generative model to generate the question with the paragraph and answer as model input, and then use a pre-trained MRC model to filter the synthetic question data.

they are unable to generate context-relevant unanswerable questions.


This can cause the augmented question to miss the original key information and not to able to infer the original answer.

it can be observed that the generated answerable questions of CRQDA still maintain the key information for the original answer inference.


Normally, the pre-training procedures are designed to learn on tokens corresponding to small units of texts (e.g., word pieces for English, characters for Chinese) for efficiency and simplicity.

some important information carried by larger text units may be lost for certain languages when we use a standard encoder, such as BERT.


Sun et al. (2019a) proposed to perform both entity-level and phraselevel masking to learn knowledge and information from the pre-training corpus.

their approaches are limited in the following senses.


Clearly, encoding premise targets into Seq2Seq boosts its effectiveness, indicating the importance of modeling premise targets.

both Seq2Seq variants perform poorly compared to our approaches.


Generally, over-performing triplets had little effect on the success rates due to reduced strategy attention.

under-performing triplets were relatively highly attended to.


These methods include bi-directional language modeling (Peters et al., 2018), masked language models (Devlin et al., 2019), word order permutation (Yang et al., 2019), more robust training (Liu et al., 2019) and more efficient architectures (Lan et al., 2019).

little focus has been put on learning discourse coherence as part of the pretraining objective.


The success of multilingual NMT on low-resource languages relies heavily on transfer learning from high-resource languages for which copious amounts of parallel data is easily accessible.

existing multilingual NMT approaches often do not effectively utilize the abundance of monolingual data, especially in lowresource languages.


While massively multilingual models have obtained impressive quality improvements for low-resource languages as well as zero-shot scenarios (Aharoni et al., 2019;Arivazhagan et al., 2019a), it has not yet been shown how these massively multilingual models could be extended to unseen languages, beyond the pipelined approaches (Currey and Heafield, 2019;Lakew et al., 2019).

self-supervised learning approaches have excelled at down-stream cross-lingual transfer (Devlin et al., 2019;Raffel et al., 2019;, but their success for unsupervised NMT (Conneau and Lample, 2019;Song et al., 2019) currently lacks robustness when languages are distant or monolingual data domains are mismatched (Neubig and Hu, 2018;VuliÃ„â€¡ et al., 2019).


For instance, TAC will assign the same merit score when gold interval [10,20] is compared with predicted interval [5,15], versus when gold [100,200] is compared with prediction [95,195].

a human would judge the latter more favorably, because a 5minute delay in a 10-minute trip would usually be considered more serious than in a 100-minute journey.


TIMEPLEX predicts 1967 for this query (earning an aeIOU credit of 33.33).

tNtComplex predicts 2013 (earning almost no credit) -this also highlights that it does not capture commonsense that a 4 the list of such relation pairs is given in the Appendix C person can marry only after they are born.


These benchmarks help to provide a uniform evaluation of new modeling developments.

recent work shows a problem with this standard evaluation paradigm based on i.i.d. test sets: datasets often have systematic gaps (such as those due to various kinds of annotator bias) that (unintentionally) allow simple decision rules to perform well on test data (Chen et al., 2016; Gururangan et al., 2018; Geva et al., 2019).


Adversarial examples are almost the methodological opposite of contrast sets: they change the input such that a model's decision changes but the gold label does not (Jia and Liang, 2017; Wallace et al., 2019a).

contrast sets are model-agnostic, constructed by experts to characterize whether a model's decision boundary locally aligns to the true decision boundary around some point.


For most datasets, the average time to perturb each example was 1-3 minutes, which translates to approximately 17-50 hours of work to create 1,000 examples.

some datasets, particularly those with complex output structures, took substantially longer: each example for dependency parsing took an average of 15 minutes (see Appendix B for more details).


In addition, we use knowledge graphs to obtain distractors for the masked entities, and propose a novel distractor-suppressed ranking objective that is optimized jointly with masked language model.

to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training, to inject language models with structured knowledge via learning from raw text.


Recently, Ye et al. (2019) propose to mask one entity mention in a sentence, and then formulate a multiple-choice QA task (Talmor et al., 2019) for representation learning, by treating the masked sentence as the question, and the masked entity plus its negative samples as answer candidates.

this model does not quite match the MLM since the model is pre-trained by multiple-choice QA and only one entity can be masked in a sentence.


Note, methods using IR (e.g., RoBERTa+KE) must retrieve from Wikipedia during finetuning and inference, which increases the computational overhead significantly.

methods based on additional self-supervised pre-training are more efficient, but often achieve sub-optimal performance since they lack explicitly retrieved contexts.


This helps filter human-generated distractor answers since they unlikely appear in Concept-Net.

our method never uses Concept-Net during finetuning and only observes a small subgraph of ConceptNet (about 30% Ã¢Ë†Â¼ 40% linked concepts without relations) during pre-training.


Both KG-guided and random entity masking can mask informative chunks and long-term dependency needs to be modeled in order to infer the masked tokens.

random span or word masking is likely to mask tokens that can be easily inferred from local context a much simpler task.


For example, Peters et al. (2019) retrieve entities' embeddings according to the similarity between Transformer's hidden states and pre-trained graph embeddings, then treat the retrieved embeddings as extra inputs to the next layer.

Bosselut et al. (2019) directly finetune a pre-trained LM on partially-masked triples from a KG, aiming at commonsense KBC tasks.


As for entity linking, there are many mature entity linking systems for ontological or factoid KGs, such as S-MART (Yang and Chang, 2015), DBpeida Lookup, and DeepType (Raiman and Raiman, 2018).

for a commonsense KG whose content consists of non-canonicalized or free-form texts, there is no such a system to complete its entity linking.


Recently, novel neural architectures have been explored for modeling action items in emails (Lin et al., 2018) and identifying intents in email conversations .

there has been less focus on task-specific email summarization (Corston-Oliver et al., 2004).


This is namely the case of the Reading Comprehension task (Richardson et al., 2013).

reading Comprehension datasets are costly and difficult to collect and are essentially native English datasets.


The results show us that a relatively low number of samples are needed to reach acceptable results on the reading comprehension task.

to outperform the Human Score, i.e. 91.2% and 75.9 %, a larger number of samples is required.


More (2016) combines CRF and structured perceptron with a curated normalization scheme to predict values, and Xu et al. (2019) regard attributes as queries and adopt BIO tags for any attributes, making it applicable for the large-scaled attribute system.

our experimental results show that the model of Xu et al. (2019) may be insufficient to identify which attribute a value corresponds to.


With respect to harnessing cheap supervision, Williams (2012); Gusev et al. (2011) propose to mine web data using a collection of hand-designed query patterns

to our approach, they are based on counting instead of machine learning and cannot handle the contextualization of events.


We use the original baseline system and interchange transformer weights to compare between BERT and ours.

because our model replaces temporal expressions with special tokens, it is at disadvantage to be directly evaluated on the original dataset with temporal expressions in natural language.


As expected, we find that our model achieves better  performance on the three dimensions that are focused in this work (i.e., duration, frequency, and typical time) as well as stationarity.

the improvements are not very substantial, indicating the difficulty of this task and motivates future works.


Policy gradients-based reinforcement learning has proven to be a promising approach for directly optimizing non-differentiable evaluation metrics for language generation tasks.

optimizing for a specific metric reward leads to improvements in mostly that metric only, suggesting that the model is gaming the formulation of that metric in a particular way without often achieving real qualitative improvements.


This line of work relies mainly on distributional semantic models, which produce one abstract representation for every word form.

aggregating all senses of a word into a single representation is particularly problematic for semantic change as word meaning hardly ever shifts directly from one sense to another, but rather typically goes through polysemous stages (Hopper et al., 1991).


Essentially, that work examines how a word gains new senses, and how some senses of a word may become deprecated.

here we examine how different words compete to represent the same meaning, and how the degree of success of words in that competition changes over time.


Furthermore, many languages with complex morphology are spoken by a limited number of people or are listed as endangered languages  (Mager et al., 2018), which reduces the possible annotator pool even more.

However, morphological segmentation is important for downstream tasks like machine translation (Conforti et al., 2018; Vania and Lopez, 2017), dependency parsing (Seeker and C¸ etinoglu ˘ , 2015; Vania et al., 2018), or semantic role labeling (Sahin and Steedman, 2018).


The scores of all systems vary across languages.

iL consistently is among the two best systems in terms of accuracy in all settings.


Looking on the learning curves for each model for increasing training set sizes, we can see that both proposed systems show monotonically increasing performance: they take advantage of more data well, but still achieve decent performance in the low-resource setting, even outperforming all non-neural systems in some settings.

the non-neural models joint and semiCRF have in many cases a good start, but only benefit to a limited extends from additional data.


If we only consider the exact segmentation point prediction, s2s performs better for all languages.

the differences between the observed error rates are relatively small between s2s and PGNet models.


The translation quality only slightly decreases when drastically decreasing the vocabulary.

there is a gap between the character- Determ   level and subword-level model of 1-2 BLEU points.


BLI accuracy is easy to compute and captures the desired property of CLWE that translation pairs should be close.

bLI accuracy does not always correlate with accuracy on downstream tasks such as cross-lingual document classification and dependency parsing (Ammar et al., 2016; Fujinuma et al., 2019; Glavas et al., 2019).


These Process entities can be further selected by combining another query, for example, a compound name.

without annotated documents, we cannot focus on a specific category of terms as shown in Table 8, where we find few terms categorized in Process.


The matscholar has annotations of seven entities, such as material names, synthesis methods, and applications.

it targets the general material science domain while we focus on superconducting domain.


They annotated material names and transition temperatures on five full papers, and obtained pair sets of these entities.

our experimental results showed that we need the amount of around 1,000 annotated abstracts to obtain mostly stable results.


Product key memory (PKM) proposed by Lample et al. (2019) enables to improve prediction accuracy by increasing model capacity efficiently with insignificant computational overhead.

their empirical application is only limited to causal language modeling.


Larger model capacity has brought improvement in accuracy by enabling better modeling of data.

increasing model capacity causes a significant increase in computational cost at both training and inference time despite better accuracy.


Due to the remarkable prediction accuracy, a transformer becomes standard architecture in natural language processing.

memory architecture can also be used to design a function that maps a continuous representation to another representation as a layer in neural networks.


On the assumption that updating memory parameters sparsely using a limited number of data and training steps might be vulnerable to the catastrophic drift, we try to fix memory parameters during fine-tuning as in Table 5.


They either have focused on scenarios where the policy is able to explore and collect more data (Degris et al., 2012; Riedmiller, 2005) such as learning online from an outdated replay buffer (e.g. (Munos et al., 2016)), or have performed off-policy policy evaluation (Farajtabar et al., 2018; Jiang and Li, 2016;Precup, 2000; Thomas and Brunskill, 2016).

we learn a policy entirely offline, from a fixed batch of data, with no ability to explore.


We empirically verify the consistency between the predicted answer and the generated proof by showing that the full accuracy matches the proof accuracy.

in scenarios where questions have open-ended answers, generating answer from a 'proof' in a consistent manner needs more exploration.


Research publications describe both academic and industrial research results.

it could be that industrial research is not entirely disclosed, since not all industrial research results are made public.


While in general the number of publications is higher for Asia than for Europe, this proportion changes, when publications of top conferences are compared, putting Europe in first place followed by North America.

it should be noted that the trend in the last two years is an increasing number of research from Asia in comparison to Europe and North America.


Our analysis of publications related to information retrieval for 2000-2017 also demonstrates more interest in this topic before year 2010.

the number of publications after 2010 is rather stable (about 6K a year).


In most of the rolesets, two to four numbered roles exist.

in some verb groups, such as verbs of motion, there can be six numbered roles in the roleset.


As discussed previously, derived forms that can be produced through a rule have not been included, as long as they share the same meaning with their bases.

forms that have taken on different meanings, i.e. diverging from the bases from which they were derived, have been included


General time and place information can be specified for any verb, therefore we chose not to include these as arguments.

more specific occurrences of these have been included, such as "interval of time", or "place that relates to the structure of the event".


For instance, in a sentence such as “I ate at a restaurant.”, the place information is simply an additional detail and it is unrelated to the internal structure of the event.

in “I went to the library.”, the place information is an important component of the event, since “to go” is a verb of motion that entails a change in location.


Comparing the two corpora, one can observe that Turkish is very fond of phrasal structures and makes extensive use of idioms instead of simple verbs, as mentioned above.

what is truly remarkable is the embedding of arguments inside the phrasal verb.


The AVG model performs slightly better than the BiLSTM on both generation tasks.

these metrics are a very crude measure of physical reasoning performance and are not intuitive.


Online social media platforms are popular outlets for individuals to exchange viewpoints and discuss topics they are interested in.

the huge volume of online conversations produced daily hinders people's capability of finding the information they are interested in.


For recommendation, there are extensive efforts on post-level recommendation (Chen et al., 2012; Yan et al., 2012) and conversation-level (Chen et al., 2011; Zeng et al., 2018, 2019b).

with them which assume static user interests, we capture how user interests change over time and take advantage of the recent advancement of dynamic product recommendation (Wu et al., 2017;Beutel et al., 2018).


This is probably because earlier history enables learning of long-term dynamics and technology change usually happens in a time span that is longer than 1-2 months.

topics on Fun and Learn may change more rapidly, making the earlier history more noisy and less helpful for modeling users' current interests.


In this light, more and more governments have realized the importance of public diplomacy, making great efforts to promote their countries' values and perspectives to foreign publics (Entman, 2008; Golan and Himelboim, 2016).

these efforts are not always successful.


"Economic Consequences" as the first and "Public Opinion" as the second.

in a multiclass setup in which the model is configured to produce a single label, it learns to disregard the second frame "Public Opinion" while strongly attending the words "fargo" and "credit" related to for the theme of "Economic Consequences".


However, in a multiclass setup in which the model is configured to produce a single label, it learns to disregard the second frame "Public Opinion" while strongly attending the words "fargo" and "credit" related to for the theme of "Economic Consequences".

a multi-label model correctly attends all words that are related to both frames i.e. "fargo", "credit", "nuns" and "opposition" and predicts “Economic Consequences" and “Public Opinion" correctly.


These general purpose language identification models work well for longer texts which have similar editing style.

their performance drops significantly on shorter texts such as search queries and shorter tweets (Cavnar and Trenkle, 1994;Sibun and Reynar, 1996;Baldwin and Lui, 2010), as do many NLP components (Baldwin et al., 2013).


Since without word boundaries information, it is intuitive to model characters in Chinese tasks directly.

in most cases, the semantic of a single Chinese character is ambiguous.


Clearly, the automatically labeled datasets in distant supervision contain amounts of sentences with wrong relation labels.

previous works only focus on wrongly labeled instances in training sets but neglect those in test sets.


The model is trained using gold alignments.

our approach is fully unsupervised; it does not require gold alignments generated by human annotators during training.


For each model, one can extract separate word alignments and symmetrize these using heuristics like grow-diagonal (Och and Ney, 2000b;Koehn et al., 2005).

this approach uses the hard word alignments of both directions as an input, and does not consider any other information of the forward and backward model.


obtained among models selected via SSD (GAZE-SEQ: 56.16) is even higher than that obtained by the best CIDEr-selected one (GAZE-AGG: 55.74).

this is likely due to CIDEr being sensitive to lexical differences between the test set and the validation set used for model selection, which could lead to slightly different patterns.


We thus conjecture that this ability is due to the presence of the gaze-dedicated LSTM, which allows for a more abstract processing of the visual input.

the presence of gaze data does not fully solve the issue of words being repeated within the same caption, as illustrated by the rightmost example in Fig.


A model that can provide such detailed interpretations facilitates answering inferential questions, such as "Will the player get angry later?".

existing visual understanding systems are unable to perform such tasks that require speculative reasoning.


Video Question Answering: Since caption generation can only describe observable events, recent work seeks to move closer to comprehension, by learning to answer complex questions about videos.

the datasets used for Video QA (Yang et al., 2003;Xu et al., 2016;Zhu et al., 2017) focus only on directly evident visual concepts and construct the questions mostly about "where" and "what" aspects.


Splitting the stigma definition into simpler concepts was adopted in the current research as well.

the above mentioned approaches do not distinguish between different degrees of the concept, whereas the present work introduced several layers of the concept that were reduced to two in the process: blame and out-of-group generalization.


It is important that NLP applications such as named entity recognition and question answering produce calibrated confidence scores for their predictions, especially if the applications are to be deployed in a safetycritical domain such as healthcare.

the output space of such structured prediction models is often too large to adapt binary or multi-class calibration methods directly.


Their biggest limitation is that they rely heavily on cross-domain mapping of information such as functional similarities, e.g., (Vandevenne et al., 2012; Vandevenne et al., 2016; Shu and Cheong, 2010; Cheong and Shu, 2014;Rugaber et al., 2016; Zhao et al., 2018).

automatically identifying crossdomain relational similarities, requires substantial reasoning beyond what is currently computationally feasible.


The training set contains 30% more images with "woman cooking" than "man cooking".

when evaluating the top predictions of a trained model, the disparity between males and females is amplified to around 70%.


If the bias is not amplified, the dots should be scattered around the reference line.

most dots are on the top-right or bottom-left, showing the bias is amplified.


Recent studies have noticed the problem and focused on generating appropriate seller responses by integrating external information, e.g., product attributes and titles, into single-turn dialogue generation (Zhao et al., 2019; Chen et al., 2019; Gao et al., 2019).

they are difficult to generalize in reality because of limited materials on hand and different scenarios.


Meng et al. (2019) proposed RefNet, which used background descriptions about the target dialogue and used a copy mechanism to copy tokens or semantic units.

all these models are difficult to generalize in reality because of using different materials, which are not always accessible.


Wu et al. (2016) proposed a hierarchical encoderdecoder framework to model all the context utterances which can better grasp the overall information of the dialogues.

these models are difficult to generalize, and their results are unsatisfied since responses maybe vary a lot for the same question towards different occasions and speakers.


Concurrently with this work, Chen et al. (2020) also developed a method of studying the interaction between words using Shapley-based techniques like CD.

their method was based on an assumption of underlying hierarchical structure and therefore unsuitable for the experiments we are about to conduct.


For example, in the sentence, "Socrates asked the student trick questions", "trick questions" has a clear definition and strong connotations that are less evident in each word individually.

knowing that "trick" and "student" co-occur is not sufficient to clarify the meaning and connotations of either word or compose a shared meaning.


The application of a curriculum is based on the often unspoken assumption that the representation of a complex pattern can be reached more easily from a simpler pattern.

we find that effectively representing shorter scaffolds actually makes a language model less effective at generalizing a long-range rule, as found by Zhang et al. (2018).


Open-ended generation -the task of generating text that forms a natural continuation from the input text -requires the model to hallucinate text; hence the focus has been to ensure that the model learns to generate text that is more human-like (i.e., less repetitive or dull with more content-related words) (Holtzman et al., 2020;Welleck et al., 2020;See et al., 2019).

tasks such as document summarization (Nenkova and McKeown, 2011;See et al., 2017;Paulus et al., 2018) and data-to-text generation (Lebret et al., 2016;Wiseman et al., 2017) which are not open-ended, require models to be factual and/or faithful to the source text.


Finally, others have used reinforcement learning to improve informativeness and reduce contradictory information in abstractive summaries, e.g., Pasunuru and Bansal (2018) used a textual entailment-based reward and Arumae and Liu (2019), a question-answering based reward.

these approaches don't evaluate if these rewards improve faithfulness of summaries.


At first glance, these models are quite similar to ours.

we should make it clear that they are totally different.


It is acceptable that when using more complicated language models, the effect of visual knowledge will be weakened.

there are indeed some methods to improve the current results, which will be investigated in our future work.


In some cases, the word jili-da 'be frightened' was annotated with the original frame Experiencer focus when used in a sentence without objects or emotion.

if there were objects and emotion in a sentence, the crowd workers generally selected the frame Emotion directed.


The frame Intentionally act is a comprehensive high-level frame that covers a wide range of acts, and there are many LUs and derived frames associated with it.

during our testing, the crowd workers tended to choose more specific frames that better represented the meaning of the words in sentences.


Also, the word bodo-ha-'(a press) report' was annotated with the frame Reporting to emphasize the role of the speaker.

the crowd workers chose the frame Statement for the verbs yeonsul-hada 'speech', jujanghada 'claim', and suneon-hada 'declare' because those verbs are not influenced by the role of the speaker.


The annotators suggested the frame Emotions of mental activity or the frame Emotion active for a sentence describing the emotion of an experience.

if a sentence was focused on an event or a situation that provoked emotions in an experience, the crowd workers tend to choose the frame Stimulus focus.


Generally, these works are based on the assumption that the translated target-language words have the same frames as the original-language words.

because of the differences in culture and linguistics (Baker et al., 2018), frames are shown in different manners in bilingual corpora (Torrent et al., 2018)


Leveraging large amounts of unlabeled data using Transformer-like architectures, like BERT, has gained popularity in recent times owing to their effectiveness in learning general representations that can then be further fine-tuned for downstream tasks to much success.

training these models can be costly both from an economic and environmental standpoint.


These pre-trained models, particularly the ones based on the Transformer architecture (Vaswani et al., 2017) 1 have achieved state-of-the-art results in a variety of NLP tasks, but come at a great cost financially and environmentally (Strubell et al., 2019;Schwartz et al., 2019).

cross-View Training (cVT) ) is a semi-supervised approach that uses unlabeled data in a task-specific manner, rather than trying to learn general representations that can be used for many downstream tasks.


For computer vision, the information compressed/reduced in image features can be partially retrieved from neighboring pixels since they share similar and uniform characteristics spatially.

for NLP, the syntax and semantics information of Transformer in language/text domain are more sensitive than that of computer vision.


A different model was proposed by Frermann and Lapata (2016), focusing instead on capturing the subtle meaning changes within a sense over time.

evaluating such models is difficult, as the lack of large scale time-stamped data prevents direct quantitative evaluation.


For NEO, the error is below 0.1 (near perfect predictions) for more than 30% of the instances while it is above 0.9 (totally incorrect predictions) for slightly less than 20% of the instances.

sCAN scores correctly more than 40% of the instances while the incorrect predictions are more than 30%.


On one hand, the MAE is calculated as the average error across the instances which are labeled only with this particular true sense.

in the classification setting, all the instances of a target are taken into account for a specific sense.


Such domains might also require the use of more sophisticated aspect term extraction methods.

it is not the case that our model necessarily overlooks implicit aspects.


Kirov and Cotterell (2018) argue that the answer is yes: modern Encoder-Decoder (ED) architectures learn human-like behavior when inflecting English verbs, such as extending the regular past tense form /-(e)d/ to novel words.

their work does not address the criticism raised by Marcus et al. (1995): that neural models may learn to extend not the regular, but the most frequent class — and thus fail on tasks like German number inflection, where infrequent suffixes like /-s/ can still be productively generalized.


Recently, several evaluation test sets have been proposed to measure the word sense disambiguation (WSD) capability of machine translation systems.

to date, these evaluation test sets do not include any training data that would provide a fair setup measuring the sense distributions present within the training data itself.


Furthermore, English as source language produces higher numbers of ambiguous words compared to English as target language.

the extended training sets tend to be larger in the language pairs with English as source language.


Unlike in the fully unsupervised setting, the HSMM model with ordering (HSMM+Ord, row U4) learns to distinguish steps from background when constrained to predict each step region once in a video, with predicted background timesteps (70.6%) close to the ground-truth (72%).

performance of this model is still very low on the task metrics -comparable to or underperforming the ordered uniform baseline with background (row B3) on all metrics.


Narration constraints substantially improve all label accuracy (comparing U1 and U5).

the model overpredicts background, likely because it doesn't enforce each step type to occur in a given video.


The language has over 30 million extant manuscripts potent for digitisation, one hundred times those in Greek and Latin combined (Goyal et al., 2012).

a major source of concern for the processing of texts in a low-resource language like Sanskrit is the lack of availability of labelled data.


Based on the predictions from these systems 5 , the relevant candidate words are highlighted so as to assist the annotator by inviting her attention to those words.

the final decision still remains with the annotator, who may or may not choose the suggestion.


By the end of the experiment, each annotator ends up annotating all the 30 sentences.

an annotator annotates a particular sentence for only one of the 3 annotations tasks, i.e. segmentation annotation without suggestion, segmentation annotation with suggestions and morphosyntactic annotation


The stock-specific information in Hu et al. (2018) was encoded in the neural network, however, making it focused on the price prediction task

we represent such stock-specific information by the stock embedding, i.e., a vector, which is easy to interpret geometrically and extract for other applications.


The upper figure (a) shows the conventional setting without sharing, in which J classifiers are generated, one for each stock.

the lower figure (b) shows one classifier generated for all stocks.


Xie and Sun (2019) proposed a sequence-to-tree (Seq2Tree) method for generating an expression tree in pre-order based on the parent node and the left sibling tree of each node.

global information is still not being considered in the generated expression tree.


The corpus contains 106 texts, only one of which is marked as Evenki in the literary language.

in addition, we will take texts from the Newspaper corpus to be Evenki in the literary language.


In constrast, Analyser Free can produce analysis for wordforms with different ordering of affixes.

the mean ambiguity also increases, as wordforms receive more possible analyses.


Previous works on cross-lingual NER are mostly based on label projection with pairwise texts or direct model transfer.

such methods either are not applicable if the labeled data in the source languages is unavailable, or do not leverage information contained in unlabeled data in the target language.


Differently, model-transfer based methods (Wu and Dredzea, 2019; Wu et al., 2020) focus on training a shared NER model on the labeled data in the source language with languageindependent features, such as cross-lingual word representations (Devlin et al., 2019), and then directly testing the model on the target language.

there are limitations in both labelprojection based methods and model-transfer based methods.


Past work in natural language processing has studied calibration in the nonneural (Nguyen and O'Connor, 2015) and neural (Kumar and Sarawagi, 2019) settings across several tasks.

past work has not analyzed large-scale pre-trained models, and we additionally analyze out-of-domain settings, whereas past work largely focuses on in-domain calibration (Nguyen and O'Connor, 2015;Guo et al., 2017).


For example, the KORE 50 (Hoffart et al., 2012) and DBpedia Spotlight (Mendes et al., 2011) data sets are publicly available, converted into NIF and contain news topics from different domains.

they both rely on DBpedia -"the central point of the Linked Open Data movement," according to  (Roder et al., 2014) and thus are not capable of delivering results on other KGs.


To this end, product-related community question answering (PQA) platforms have emerged in many E-commerce sites such as Amazon and Taobao, allowing users to pose their concerns as questions and receive answers from fellow users to obtain useful product information.

similar to other community question answering (CQA) platforms, the user-provided answers on PQA platforms vary significantly on their qualities (Zhang et al., 2020b), and more seriously, their veracity due to the lack of systematic quality control (Mihaylova et al., 2018).


As pointed out in Mihaylova et al. (2019), verifying the verdict of answers in CQA requires using rich world knowledge.

gathering relevant information as evidence can be difficult due to the open-domain nature of those questions.


The claim and evidence representations can be learned with neural networks such as recurrent neural networks (RNNs) (Rashkin et al., 2017) or convolutional neural networks (CNNs) (Wang, 2017).

none of these work conducts fact checking problem in QA settings with associated well-formatted evidence sentences.


There are 27 confounders in the test set, and my vector addition model places all of them in the top 4 ranks for the confounding term.

the Pixie Autoencoder and BERT do not fall for the confounders, with a mean rank of 171 and 266, respectively.


(Kim, 2014;Li et al., 2015;Zhou et al., 2016;Yang et al., 2016;Chen et al., 2017;Li et al., 2015) focused on classifying the given text as positive, negative, neutral at document level and sentence level.

the accuracy of sentiment classification on multiple opinion aspects/targets level is unsatisfactory.


Existing belief trackers mainly depend on supervised learning with human annotations of belief states for every user utterance.

collecting these turn-level annotations is labor-intensive and time-consuming, and often requires domain knowledge to identify slots correctly.


On the whole, GTS-CNN and GTS-BiLSTM achieve the best results respectively with 2 and 3 inference times on two datasets, and GTS-CNN performs better than GTS-BiLSTM in different inference times.

gTS-BERT reaches a crest only with 1 time of inference because BERT has contained rich context semantics.


To avoid the error propagation of pipeline methods, some studies use joint learning based on traditional machine learning algorithms and hand-crafted features, including Imperatively Defined Factor graph (IDF) (Klinger and Cimiano, 2013a), joint inference based on IDF (Klinger and Cimiano, 2013b), and Integer Linear Programming (ILP) (Yang and Cardie, 2013).

these methods heavily depend on the quality of handcrafted features and sometimes perform worse than pipeline methods (Klinger and Cimiano, 2013b).


Definition generation, which aims to automatically generate dictionary definitions for words, has recently been proposed to assist the construction of dictionaries and help people understand unfamiliar texts.

previous works hardly consider explicitly modeling the "components" of definitions, leading to under-specific generation results.


It is a common practice for human to consult a dictionary when encountering unfamiliar words (Fraser, 1999).

it is often the case that we cannot find satisfying definitions for words that are rarely used or newly created.


Inspired by this, Yang et al. (2019) incorporate sememes (Bloomfield, 1949; Dong and Dong, 2003), i.e. minimum units of semantic meanings of human languages, in the task of generating definition in Chinese.

it is just as, if not more, time-consuming and expensive to label the components of words than to write definitions manually.


This in itself is uncontroversial -we do not mind if the model is able to predict a person's gender.

as gender information is now encoded in the embedding of person1, the model is also able to use this information when scoring other triples, such as (person1, profession, banker).


Overall, the random forest model gave the highest prediction, in classifying news articles as either reporting an epidemic event or not.

for French documents, the model could predict only one class using the default classification threshold of 0.5.


This is particularly promising since speech pauses can be automatically extracted in tools such as PRAAT.

the results imply that gestures can be employed to predict audience response to some extent as well.


Since CDM completely blocks the signals of the contexts being identified unimportant, it may falsely disregard useful signals.

cDW emphasizes flexibility and allows further signals to contribute small weights corresponding to its relatedness with the aspect terms in the dependency-based tree.


Since LCF-BERT uses Semantic Relative Distance, the sentiment term "without a doubt" has been paid the most focus due to its close distance to the aspect term "cuisine" based on word counts metrics.

the signal of a key sentiment word "delicious" is mistakenly down-weighted because it is far away from the aspect term "cuisine".


On QQP, the best model gets 80.2% F1 on heuristically balanced test examples.

on all-pairs test data, the best model can only reach 3.5% precision at a modest 20% recall.


On WikiQA, our best c-MAP of 84.6% is higher than the best previously reported c-MAP without using additional question-answering data, 83.6% (Garg et al., 2020).

on all-pairs test data, the best model gets 6.5% precision at 20% recall.


True positives and false negatives are computationally easy to compute, as they only require evaluating S(x) on all the positive inputs x in D test all .

without any structural assumptions on S, it is computationally infeasible to exactly compute the number of false positives, as that would require evaluating S on every negative example in D test all , which is too large to enumerate.


Static retrieval collects fewer positives over time, as it exhausts the set of positives that are easy to identify.

uncertainty sampling collects many more positives, especially after the first round of training, because it improves its embeddings over time.


The recent emergence of multilingual pretraining language model (mPLM) has enabled breakthroughs on various downstream crosslingual transfer (CLT) tasks.

mPLMbased methods usually involve two problems: (1) simply fine-tuning may not adapt generalpurpose multilingual representations to be task-aware on low-resource languages; (2) ignore how cross-lingual adaptation happens for downstream tasks.


Specifically, MGL models each CLT process as heterogeneous information propagation over a dynamic graph, which captures latent language correlations and makes the downstream CLT adaptation more interpretable.

solely learning the dynamic graph structures may be insufficient since the graph-based metric space usually favors highresource languages over low-resource ones.


Recently, some efforts have been initiated on meta learning for low-resource NLP tasks that straightforwardly views each dataset with its objective as a task (Dou et al., 2019).

this strategy can only make meta-leaner learn knowledge from each language separately.


As we can see, increasing the number of GCN layers from 1 to 2 (default) shows significant improvements.

when further increasing the number of layers to 3 and 4, the performances will be degraded.


Besides, we propose methods to breakdown the whole (graph, sentence) pair into smaller pieces of (edge, word) pairs with alignments, before training our model to reconstruct each edge given the corresponding word.

neither of the previous efforts tried to leverage this valuable information.


One may argue that we could directly predict the original graph so that no structural information would be lost.

each type of graphs can have their own parsing algorithm due to their unique properties (such as directed vs undirected, rooted vs unrooted, etc).


Existing dialogue state tracking (DST) models require plenty of labeled data.

collecting high-quality labels is costly, especially when the number of domains increases.


JamSpell and Pyenchant spelling checkers are inadequate for correcting text errors at character-level, these performed equally poorly, introducing significantly more errors than they fixed.

the seq2seq-based neural model was our best performing WEC system, it proved to be very effective, correcting significantly more errors than it introduced.