When using only mBERT with S2T alignments we have high recall but a very mediocre precision; when using INTER alignments we see big gains in precision at the expense of lower recall, as expected.

this highlights the main advantages of our method: by relying on a big multilingual language model i) we obtain high-quality word alignments featuring high precision and recall, and ii) we do not need to re-train for other language pairs nor different domains.


They also examine the effect of decreasing the size of the dataset.

we trained two pipeline models (pipe-ind and pipeseq) which only differ in their training procedure (see Section 3.2.2).


Recent work showed that these models can be improved if transcriptions are available at training time.

even though the model works on ground-truth text (no translation involved), we still see a 4% drop in R@10 between the reduced english condition and Japanese.


The architecture (summarized in Figure 2c) is basically composed of an ASR module which maps speech to text, followed by the text-image system we just described (our NLU component).

it is not clear how an end-to-end approach compares to a traditional pipeline-based approach when one has access to transcriptions.


Formally, assume that we have an input sequence of w0, w1, . . . , wn−1, and we generate their corrupted positions p0, p1, . . . , pn−1 with our local bag strategy.

it may suffer from the same problem as unidirectional LMs: at one time, contexts from only one direction can be utilized instead of from both directions.


If stored using the same precision, the optimizer memory should be three times of model memory.

to validate our claim that BlockBERt with n Ã¢â€¡Â¥ n blocks can reduce the O(N 2 ) memory usage by a factor of n, we perform the same memory profiling as described in sections 2.1 and 2.2.


Training BERT is a memory-intensive process.

when n = 2, we denote 10:2 to be the configuration which assigns 10 heads to permutation (1, 2) and 2 to permutation (2, 1); when n = 3, we denote 8:2:2 to be the configuration which assigns 8, 2, 2 heads to permutation (1, 2, 3), (2, 3, 1), and (3, 1, 2), respectively.


Based on the phoneme embeddings of a word, we apply the attention mechanism (Bahdanau et al., 2015) to simultaneously identify important phonemes and derive the pronunciation embedding T P i .

to resolve the issues of sparseness and heterographics, the word embedding techniques (Mikolov et al., 2013; Pennington et al., 2014) provide flexible representations to model puns (Hurtado et al., 2017; Indurthi and Oota, 2017; Cai et al., 2018).


To detect the pun, Pedersen (2017) supposes that if there is one pun in the sentence, when adopting different Word Sense Disambiguation (WSD) methods, the sense assigned to the sentence will be different.

static word embeddings are insufficient to represent words.


We run experiments on one Tesla V100 16GB GPU and each epoch takes several minutes.

human annotation of aspectlevel sentiment is laborious and expensive, therefore, some researches focus on weakly supervised DMSC.


This can be manually annotated in at least 7,5 days, but thanks to the automatic annotation accuracy, it was manually corrected into 3 days.

at the present stage of research, TarC consists of 25.000 tokens, however our work is in progress and for future research we plan to enforce the semi-automatic transcription, which has already shown encouraging results (accuracy = 70%).


CODA TUN proposes to keep the MSA rule of maintaining a space between the first negation mark and the verb, in order to uniform CODA TUN to the first CODA (Habash et al., 2012).

first of all, the resource will be useful to give an overview of the TA.


Since there is a one-to-one mapping between Text Plot Specifications (TPSpecs) and plot images, we only need to generate TPSpecs.

the system is in charge of all plotting details, which are not accessible to users.


First proposed by Shen et al. (2017b), it has achieved great progress in recent years (Xu et al., 2018; Lample et al., 2019; Zhang et al., 2018b; Fu et al., 2018; Jin et al., 2019; Yang et al., 2018; Jin et al., 2020).

we present a novel headline generation model, TitleStylist, to produce enticing titles with target styles including humorous, romantic, and click-baity.


Many recent works extended ABS by utilizing additional features (Chopra et al., 2016;Takase et al., 2016;Nallapati et al., 2016;Shen et al., 2016Shen et al., , 2017aTan et al., 2017;Guo et al., 2017).

without samples from P (A, T ), this is a challenging or even impossible task if E S and E T , or G S and G T are completely independent of each other.