A Case Study of Complex Graph Analysis in Distributed Memory: Implementation and Optimization

... Focusing on one of the largest publicly-available hyperlink graphs (the 2012 Web Data Commons graph1, which was in- turn extracted from the open Common Crawl web corpus2), we develop parallel ... 1http://webdatacommons.org/hyperlinkgraph/ 2http://commoncrawl.org ...


A Convolutional Encoder Model for Neural Machine Translation

J Gehring, M Auli, D Grangier, YN Dauphin - arXiv preprint arXiv:1611.02344, 2016

... WMT'15 English-German. We use all available parallel training data, namely Europarl v7, Common Crawl and News Commentary v10 and apply the standard Moses tokenization to obtain 3.9M sentence pairs (Koehn et al., 2007). We report results on newstest2015. ...


A Deep Fusion Model for Domain Adaptation in Phrase-based MT

... test-13 993 18K 17K test-13 1169 26K 28K Table 1: Statistics of the English-German and Arabic-English training corpora in terms of Sentences and Tokens (represented in millions). ep = Europarl, cc = Common Crawl, un = United Nations ...


A Large DataBase of Hypernymy Relations Extracted from the Web

J Seitner, C Bizer, K Eckert, S Faralli, R Meusel… - … of the 10th edition of the …, 2016

... 3http://webdatacommons.org/framework/ 4http://commoncrawl.org ... The corpus is provided by the Common Crawl Foundation on AWS S3 as free download.6 The extraction of ... isadb/) and can be used to repeat the tuple extraction for different or newer Common Crawl releases. ...


A Maturity Model for Public Administration as Open Translation Data Providers

N Bel, ML Forcada, A Gómez-Pérez - arXiv preprint arXiv:1607.01990, 2016

... There are techniques to mitigate the need of large quantities of parallel text, but most often at the expense of resulting translation quality. As a reference of the magnitude we can take as a standard corpus the Common Crawl corpus (Smith et al. ...


A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference

B Paria, KM Annervaz, A Dukkipati, A Chatterjee… - arXiv preprint arXiv: …, 2016

... We used batch normalization [Ioffe and Szegedy, 2015] while training. The various model parameters used are mentioned in Table I. We experimented with both GloVe vectors trained1 on Common Crawl dataset as well as Word2Vec vector trained2 on Google news dataset. ...


A practical guide to big data research in psychology.

EE Chen, SP Wojcik - Psychological Methods, 2016

... as well as general collections, such as Amazon Web Services' Public Data Sets repository (AWS, nd, http://aws.amazon.com/public-data-sets/) which includes the 1000 Genomes Project, with full genomic sequences for 1,700 individuals, and the Common Crawl Corpus, with ...


AI Saleh, MF Al Rahmawy, AE Abulwafa - World Wide Web, 2016

Page 1. A semantic based Web page classification strategy using multi-layered domain ontology Ahmed I. Saleh1 & Mohammed F. Al Rahmawy2 & Arwa E. Abulwafa1 Received: 3 February 2016 /Revised: 13 August 2016 /Accepted ...


A Story of Discrimination and Unfairness

... power has led to high quality language models such as word2vec [7] and GloVe [8]. These language models, which consist of up to half a million unique words, are trained on billions of documents from sources such as Wikipedia, CommonCrawl, GoogleNews, and Twitter. ...


A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs

S Longpre, S Pradhan, C Xiong, R Socher - arXiv preprint arXiv:1611.05104, 2016

... All models in this paper used publicly available 300 dimensional word vectors, pre-trained using Glove on 840 million tokens of Common Crawl Data (Pennington et al., 2014), and both the word vectors and the subsequent weight matrices were trained using Adam with a ...


A Web Application to Search a Large Repository of Taxonomic Relations from the Web

S Faralli, C Bizer, K Eckert, R Meusel, SP Ponzetto

... 1 https://commoncrawl.org 2 http://webdatacommons.org/framework/ 3 https://www.mongodb. com ... of the two noun phrases involved in the isa relations into pre-modifiers, head and post-modifiers [6], as well as the frequency of occurrence of the relation in the Common Crawl...


Abu-MaTran at WMT 2016 Translation Task: Deep Learning, Morphological Segmentation and Tuning on Character Sequences

VM Sánchez-Cartagena, A Toral - Proceedings of the First Conference on Machine …, 2016

... 362 Page 2. Corpus Sentences (k) Words (M) Europarl v8 2 121 39.5 Common Crawl 113 995 2 416.7 News Crawl 2014–15 6 741 83.1 Table 1: Finnish monolingual data, after preprocessing, used to train the LMs of our SMT submission. ...


Action Classification via Concepts and Attributes

A Rosenfeld, S Ullman - arXiv preprint arXiv:1605.07824, 2016

... To assign GloVe [19] vectors to object names or attributes, we use the pre-trained model on the Common-Crawl (42B) corpus, which contains a vocabulary of 1.9M words. We break up phrases into their words and assign to them their mean GloVe vector. ...


Active Content-Based Crowdsourcing Task Selection

... Pennington et. al. [28] showed distributed text representations to capture more semantic information when the models are trained on Wikipedia text, as opposed to other large corpora such as the Common Crawl. This is attributed ...


Adverse Drug Reaction Classification With Deep Neural Networks

... 4http://commoncrawl.org/ 5Source code is available at https://github.com/trunghlt/ AdverseDrugReaction 879 Page 4. max pooling feedforward layer convolutional layer (a) Convolutional Neural Network (CNN) (b) Recurrent Convolutional Neural Network (RCNN) ...


All Your Data Are Belong to us. European Perspectives on Privacy Issues in 'Free'Online Machine Translation Services

P Kamocki, J O'Regan, M Stauch - Privacy and Identity Management. Time for a …, 2016

... http://www.cnet.com/news/google-translate-now-serves-200-million-people-daily/. Accessed 23 Oct 2014. Smith, JR, Saint-Amand, H., Plamada, M., Koehn, P., Callison-Burch, C., Lopez, A.: Dirt cheap web-scale parallel text from the Common Crawl...


An Analysis of Real-World XML Queries

P Hlísta, I Holubová - OTM Confederated International Conferences" On the …, 2016

... crawler. Or, there is another option – Common Crawl [1], an open repository of web crawled data that is universally accessible and analyzable, containing petabytes of data collected over the last 7 years. ... 3.1 Common Crawl. We ...


An Attentive Neural Architecture for Fine-grained Entity Type Classification

S Shimaoka, P Stenetorp, K Inui, S Riedel - arXiv preprint arXiv:1604.05525, 2016

... appearing in the training set. Specifically, we used the freely available 300 dimensional cased word embeddings trained on 840 billion to- kens from the Common Crawl supplied by Pennington et al. (2014). As embeddings ...


Analysing Structured Scholarly Data Embedded in Web Pages

P Sahoo, U Gadiraju, R Yu, S Saha, S Dietze

... the following section. 2.2 Methodology and Dataset For our investigation, we use the Web Data Commons (WDC) dataset, being the largest available corpus of markup, extracted from the Common Crawl. Of the crawled web ...


ArabicWeb16: A New Crawl for Today's Arabic Web

R Suwaileh, M Kutlu, N Fathima, T Elsayed, M Lease

... English content dominates the crawl [12]. While Common Crawl could be mined to identify and ex- tract a useful Arabic subset akin to ArClueWeb09, this would address only recency, not coverage. To address the above concerns ...


M Malinowski, M Rohrbach, M Fritz - arXiv preprint arXiv:1605.02697, 2016

Page 1. Noname manuscript Ask Your Neurons: A Deep Learning Approach to Visual Question Answering Mateusz Malinowski · Marcus Rohrbach · Mario Fritz Abstract We address a question answering task on realworld images that is set up as a Visual Turing Test. ...


Automated Generation of Multilingual Clusters for the Evaluation of Distributed Representations

P Blair, Y Merhav, J Barry - arXiv preprint arXiv:1611.01547, 2016

... (2013a), the 840-billion token Common Crawl corpus-trained GloVe model released by Pennington et al. (2014), and the English, Spanish, German, Japanese, and Chinese MultiCCA vectors5 from Ammar et al. ... Outliers OOV GloVe Common Crawl 75.53 38.57 5 6.33 5.70 ...


Automated Haiku Generation based on Word Vector Models

... and Page 28. 16 Chapter 3. Design Common Crawl data. Those data also come with various vector dimension size from 50-D to 300-D. Those pre-trained word vectors are used directly for this project as they take considerably ...


Automatic Construction of Morphologically Motivated Translation Models for Highly Inflected, Low-Resource Languages

J Hewitt, M Post, D Yarowsky - AMTA 2016, Vol., 2016

... sentences of Europarl (Koehn, 2005), SETIMES3 (Tyers and Alperen, 2010), extracted from OPUS (Tiedemann, 2009), or Common Crawl (Bojar et al ... Turkish, we train models on 29000 sentences of biblical data with 1000 and 20000 sentences of CommonCrawl and SETIMES ...


B1A3D2 LUC@ WMT 2016: a Bilingual1 Document2 Alignment3 Platform Based on Lucene

... 2013. Dirt cheap web-scale parallel text from the common crawl. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1374–1383. Jakob Uszkoreit, Jay M. Ponte, Ashok C. Popat, and Moshe Dubiner. 2010. ...


Big Data Facilitation and Management

Page 1. Faculty of Science and Technology Department of Computer Science Big Data Facilitation and Management A requirements analysis and initial evaluation of a big biological data processing service — Jarl Fagerli INF ...


Bootstrap, Review, Decode: Using Out-of-Domain Textual Data to Improve Image Captioning

W Chen, A Lucchi, T Hofmann - arXiv preprint arXiv:1611.05321, 2016

... We report the performance of our model and competing methods in terms of six standard metrics used for image captioning as described in [4]. During the bootstrap learning phase, we use both the 20082010 News-CommonCrawl and Europarl corpus 2 as out- of-domain ...


bot. zen@ EVALITA 2016-A minimally-deep learning PoS-tagger (trained for Italian Tweets)

... The data was only distributed to the task participants. 4.1.4 C4Corpus (w2v) c4corpus8 is a full documents Italian Web corpus that has been extracted from CommonCrawl, the largest publicly available general Web crawl to date. ...


Building mutually beneficial relationships between question retrieval and answer ranking to improve performance of community question answering

M Lan, G Wu, C Xiao, Y Wu, J Wu - Neural Networks (IJCNN), 2016 International Joint …, 2016

... The first is the 300-dimensional version of word2vec [23] vectors, which is trained on part of Google News dataset (about 100 billion words). The second is 300-dimensional Glove vectors [24] which is trained on 840 billion tokens of Common Crawl data. ...


C4Corpus: Multilingual Web-size corpus with free license

... documents. Our project is entitled C4Corpus, an abbreviation of Creative Commons from Common Crawl Corpus and is hosted under the DKPro umbrella4 at https:// github.com/dkpro/dkpro-c4corpus under ASL 2.0 license. ...


Capturing Pragmatic Knowledge in Article Usage Prediction using LSTMs

... GloVe: The embedding is initialized by the global vectors Pennington et al. (2014) that are trained on the Common Crawl corpus (840 billion tokens). Both word2vec and GloVe word embeddings consist of 300 dimensions. ...


Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution

S Ruder, P Ghaffari, JG Breslin - arXiv preprint arXiv:1609.06686, 2016

... σ: Standard deviation of document number. d: Median document size (tokens). All word embedding channels are initialized with 300-dimensional GloVe vectors (Pennington et al., 2014) trained on 840B tokens of the Common Crawl corpus11. ...


Citation Classification for Behavioral Analysis of a Scientific Field

D Jurgens, S Kumar, R Hoover, D McFarland… - arXiv preprint arXiv: …, 2016

... The classifier is implemented using SciKit (Pedregosa et al., 2011) and syntactic processing was done using CoreNLP (Manning et al., 2014). Selectional preferences used pretrained 300-dimensional vectors from the 840B token Common Crawl (Pennington et al., 2014). ...


CNRC at SemEval-2016 Task 1: Experiments in crosslingual semantic textual similarity

C Lo, C Goutte, M Simard - Proceedings of SemEval, 2016

... The system was 3We use the glm function in R. 669 Page 3. trained using standard resources – Europarl, Common Crawl (CC) and News & Commentary (NC) – totaling approximately 110M words in each language. Phrase ...


Commonsense Knowledge Base Completion

... We use the GloVe (Pennington et al., 2014) embeddings trained on 840 billion tokens of Common Crawl web text and the PARAGRAM-SimLex embeddings of Wieting et al. (2015), which were tuned to have strong performance on the SimLex-999 task (Hill et al., 2015). ...


Comparing Topic Coverage in Breadth-First and Depth-First Crawls Using Anchor Texts

AP de Vries - Research and Advanced Technology for Digital …, 2016

T Samar, MC Traub, J van Ossenbruggen, AP de Vries - International Conference on …, 2016

... nl domain, with the goal to crawl websites as completes as possible. The second crawl was collected by the Common Crawl foundation using a breadth-first strategy on the entire Web, this strategy focuses on discovering as many links as possible. ...


COMPARISON OF DISTRIBUTIONAL SEMANTIC MODELS FOR RECOGNIZING TEXTUAL ENTAILMENT.

Y WIBISONO, DWIH WIDYANTORO… - Journal of Theoretical & …, 2016

... To our knowledge, this paper is the first study of various DSM on RTE. We found that DSM improves entailment accuracy, with the best DSM is GloVe trained with 42 billion tokens taken from Common Crawl corpus. ... Glove_42B Common Crawl 42 billion tokens ...


ConceptNet 5.5: An Open Multilingual Graph of General Knowledge

R Speer, J Chin, C Havasi - arXiv preprint arXiv:1612.03975, 2016

... 2013), and the GloVe 1.2 embeddings trained on 840 billion words of the Common Crawl (Pennington, Socher, and Manning 2014). These matrices are downloadable, and we will be using them both as a point of comparison and as inputs to an ensemble. ...


Content Selection through Paraphrase Detection: Capturing different Semantic Realisations of the Same Idea

... either sentences or pred-arg structures, GLoVe pre-trained WE vectors (Pennington et al., 2014) were used, specifically the ones derived from Wikipedia 2014+ Gi- gaword 5 corpora, containing around 6 billion to- kens; and the ones derived from a Common Crawl, with 840 ...


Corporate Smart Content Evaluation

R Schäfermeier, AA Todor, A La Fleur, A Hasan… - 2016

Page 1. Fraunhofer FOKUS FRAUNHOFER INSTITUTE FOR OPEN COMMUNICATION SYSTEMS FOKUS STUDY – CORPORATE SMART CONTENT EVALUATION Page 2. Page 3. STUDY – CORPORATE SMART CONTENT EVALUATION ...


Crawl and crowd to bring machine translation to under-resourced languages

A Toral, M Esplá-Gomis, F Klubička, N Ljubešić… - Language Resources and …

... Wikipedia. The CommonCrawl project 5 should be mentioned here as it allows researchers to traverse a frequently updated crawl of the whole web in search of specific data, and therefore bypass the data collection process. ...


Cross Site Product Page Classification with Supervised Machine Learning

... An other data set used often is Common Crawl [1], which is a possible source that contain product specification pages. The data of Common Crawl is not complete with HTML-source code and it was collected in 2013, which creates many dead links. ...


CSA++: Fast Pattern Search for Large Alphabets

S Gog, A Moffat, M Petri - arXiv preprint arXiv:1605.05404, 2016

... The latter were extracted from a sentence-parsed prefix of the German and Spanish sections of the CommonCrawl5. The four 200 ... translation process described by Shareghi et al., corresponding to 40,000 sentences randomly selected from the German part of Common Crawl...


CUNI-LMU Submissions in WMT2016: Chimera Constrained and Beaten

A Tamchyna, R Sudarikov, O Bojar, A Fraser - Proceedings of the First Conference on …, 2016

... tag). Our input is factored and contains the form, lemma, morphological tag, 1http://commoncrawl.org/ 387 Page 4. lemma ... 2015. The second LM only uses 4-grams but additionally contains the full Common Crawl corpus. We ...


D6. 3: Improved Corpus-based Approaches

CP Escartin, LS Torres, CO UoW, AZ UMA, S Pal - 2016

... Based on this system and the data retrieved from Common Crawl, several websites were identified as possible candidates for crawling. ... 8http://commoncrawl.org/ 9For a description of this tool, see Section 3.1.2 in this Deliverable. 6 Page 9. ...


Data Selection for IT Texts using Paragraph Vector

MS Duma, W Menzel - Proceedings of the First Conference on Machine …, 2016

... models/doc2vec.html 3http://commoncrawl.org/ 4https://github.com/melix/jlangdetect 5-gram LMs using the SRILM toolkit (Stolcke, 2002) with Kneser-Ney discounting (Kneser and Ney, 1995) on the target side of the Commoncrawl and IT corpora. ...


David W. Embley, Mukkai S. Krishnamoorthy, George Nagy &

... tabulated data on the web even before Big Data became a byword [1]. Assuming “that an average table contains on average 50 facts it is possible to extract more than 600 billion facts taking into account only the 12 billion sample tables found in the Common Crawl” [2]. Tables ...


Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translatin

J Zhou, Y Cao, X Wang, P Li, W Xu - arXiv preprint arXiv:1606.04199, 2016

... 4.1 Data sets For both tasks, we use the full WMT'14 parallel corpus as our training data. The detailed data sets are listed below: • English-to-French: Europarl v7, Common Crawl, UN, News Commentary, Gigaword • English-to-German: Europarl v7, Common</


Deeper Machine Translation and Evaluation for German

E Avramidis, V Macketanz, A Burchardt, J Helcl… - 2nd Deep Machine …, 2016

... corpus entries words Chromium browser 6.3 K 55.1 K Drupal 4.7 K 57.4 K Libreoffice help 46.8 K 1.1 M Libreoffice UI 35.6 K 143.7 K Ubuntu Saucy 182.9 K 1.6 M Europarl (mono) 2.2 M 54.0 M News (mono) 89M 1.7 B Commoncrawl (parallel) 2.4 M 53.6 M Europarl (parallel ...


DeepNNNER: Applying BLSTM-CNNs and Extended Lexicons to Named Entity Recognition in Tweets


Detecting Opinion Polarities using Kernel Methods

R Kaljahi, J Foster - PEOPLES 2016, 2016

... 0.7, 0.8, 0.9}. The pre-trained word vectors used are the publicly available ones trained using GloVe (Pennington et al., 2014) trained on 42B-token corpus of Common Crawl (1.9 M vocabulary) with 300 dimensions. 4 4.1 Word ...


DEVELOPMENT AND APPLICATION OF A STAGE-GATE PROCESS TO REDUCE THE UNERLYING RISKS OF IT SERVICE PROJECTS

E JEONG, SR JEONG, MS RAO, VV KUMAR… - Journal of Theoretical and …, 2016

... RTE. To our knowledge, this paper is the first study of various DSM on RTE. We found that DSM improves entailment accuracy, with the best DSM is GloVe trained with 42 billion tokens taken from Common Crawl corpus. We ...


DFKI's system for WMT16 IT-domain task, including analysis of systematic errors

E Avramidis, A Burchardt, V Macketanz, A Srivastava - Proceedings of the First …, 2016

... Europarl (mono) 2.2M 54.0M News (mono) 89M 1.7B Commoncrawl (parallel) 2.4M 53.6M Europarl (parallel) 1.9M 50.1M MultiUN (parallel) 167.6K 5.8M News Crawl (parallel) 201.3K 5.1M Table 1: Size of corpora used for SMT. ...


Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network

... Word2vec embeddings were learned from Google News (Mikolov et al., 2013), and separately, from Wikipedia1. The Glove embeddings were pretrained on the 840 billion token Common Crawl corpus. ... Word2vec GoogleNews 300 71.32 Glove CommonCrawl 75 69.28 ...


Discontinuous Verb Phrases in Parsing and Machine Translation of English and German

S Loaiciga Sanchez, K Gulordava - 2016
S Loáiciga, K Gulordava

... all 2https://translate.google.com/ 2841 Page 5. monolingual data (Heafield et al., 2013). Approximately 4.5 million sentences were used for training, combining Europarlv7, CommonCrawl and News data. Optimization weights ...


Discovering Disease-associated Drugs Using Web Crawl Data

... System Online) database is used for the biomedical literature. 81.7GB sized text data of Common Crawl which Amazon Web Services hosts is used for the web crawl data. Gene symbol, Disease name and Drug name which ...


Discriminating between similar languages and arabic dialect identification: A report on the third dsl shared task

S Malmasi, M Zampieri, N Ljubešic, P Nakov, A Ali… - Proceedings of the 3rd …, 2016

... PITEOG used their own custom web-based corpus, with no further details provided. • SUKI created an additional dataset using web pages in the Common Crawl corpus. 5 Results for Subtask 2: Arabic Dialect Identification ...


Distributed Graph Storage And Querying System

... generation and consumption of such interrelated data. The number of Facebook users grew from 500 million in 2010 to 1.5 billion in 2014 [1]. The Common Crawl web corpora [2] covered for 2012 contains 3.5 billion web pages and 128 billion hyperlinks between these ...


Distributed Platforms and Cloud Services: Enabling Machine Learning for Big Data

D Pop, G Iuhasz, D Petcu - Data Science and Big Data Computing, 2016

... Data sets are growing faster, being common now to reach numbers of 100 TB or more. The Sloan Digital Sky Survey occupies 5 TB of storage, the Common Crawl Web corpus is 81 TB in size, and the 1000 Genomes Project requires 200 TB of space, just to name a few. ...


DRAFT: Interoperation Among Web Archiving Technologies

DSH Rosenthal, N Taylor, J Bailey - 2016

... WET (WARC Encapsulated Text, or Parsed Text), specified by Common Crawl and Internet Archive respectively, to represent only the text extracted from Web resources. 3 Page 4. ... [12] Common CrawlCommon Crawl. https:// commoncrawl.org/.


Dual Learning for Machine Translation

Y Xia, D He, T Qin, L Wang, N Yu, TY Liu, WY Ma - arXiv preprint arXiv:1611.00179, 2016

... In detail, we used the same bilingual corpora from WMT'14 as used in [1, 6], which contains 12M sentence pairs extracting from five datasets: Europarl v7, Common Crawl corpus, UN corpus, News Commentary, and 109French-English corpus. ...


Dynamic Coattention Networks For Question Answering

C Xiong, V Zhong, R Socher - arXiv preprint arXiv:1611.01604, 2016

... We use as GloVe word vectors pretrained on the 840B Common Crawl corpus (Pennington et al., 2014). We limit the vocabulary to words that are present in the Common Crawl corpus and set embeddings for out-of-vocabulary words to zero. ...


Edinburgh's Statistical Machine Translation Systems for WMT16

P Williams, R Sennrich, M Nadejde, M Huck, B Haddow… - Proceedings of the First …, 2016

... Our final system used two different countbased 5-gram language models (one trained on all data, including the WMT16 Romanian CommonCrawl corpus, without pruning, and one trained on news2015 monolingual only), a neural language model trained on news2015 ...


Efficient construction of metadata-enhanced web corpora

... 3.3 Extraction I designed a text extraction targeting specifically WordPress pages, which is transferable to a whole range of self-hosted websites using WordPress, al- lowing to reach various blogger profiles thanks 8http://commoncrawl. org 9https://github. ...


Efficient Data Selection for Bilingual Terminology Extraction from Comparable Corpora

... We used the FrenchEnglish aligned version at OPUS provided by JRC (Tiedemann, 2012). Common crawl corpus (CC) is a petabytes of data collected over 7 years of web crawling set of raw web page data and text extracts5. ...


Electronic Commerce Meets the Semantic Web

J Jovanovic, E Bagheri - IT Professional, 2016

... The latest Common Crawl corpus (Winter 2014; http://commoncrawl.org) consists of 2.01 billion HTML pages collected from more than 15.68 million pay-level domains (PLDs). An analysis of this corpus shows that 30 percent ...


Embedded Sensors System Applied to Wearable Motion Analysis in Sports

A Valade, A Costes, A Bouillod, M Mangin, P Acco… - 2016

... the general aspect is good: the mean error is 2.5 degrees and the standard deviation is of 6 degrees. In a second time, we tested the system behaviour on a common crawl swimming movement to ensure the functionality on complex actions. 5.2.2 Tests on Sportsmen ...


Enabling Network Security Through Active DNS Datasets

A Kountouras, P Kintis, C Lever, Y Chen, Y Nadji… - Research in Attacks, …, 2016

Y Nadji, D Dagon, M Antonakakis, R Joffe - … in Attacks, Intrusions, and Defenses: 19th …, 2016

... These include but are not limited to Public Blacklists, the Alexa list, the Common Crawl project, and various Top Level Domain (TLD) zone files. More specifically, we are using the zone files that are published daily by the administrators of the zones for com, net, biz and org. ...


Engineering a Distributed Full-Text Index

J Fischer, F Kurpicz, P Sanders - arXiv preprint arXiv:1610.03332, 2016

... For our experiments we use the common crawl corpus as input.1 It provides world wide web crawl data and contains raw content, text only and metadata of the crawled websites from about 1.23 billion web pages. In total the corpus has a size of 541 TB (as of 27.07.2016). ...


Engineering top-k document retrieval systems based on succinct data structures

... relevant documents to a given query”. For large sets of documents, eg web crawls like gov2, clueweb, or CommonCrawl, time-efficient solutions rely on precomputed index data structures. For collections of natural language ...


Enriching Product Ads with Metadata from HTML Annotations

P Ristoski, P Mika - The Semantic Web. Latest Advances and New …, 2016

... Offers - WDC Microdata Dataset: The latest extraction of WebDataCommons includes over 5 billion entities marked up by one of the three main HTML markup languages (ie, Microdata, Microformats and RDFa) and has been retrieved from the CommonCrawl 2014 corpus 5 ...


J Daiber, M Stanojevic, W Aziz, K Sima'an

Page 1. Examining the Relationship between Preordering and Word Order Freedom in Machine Translation Joachim Daiber Miloš Stanojevic Wilker Aziz Khalil Sima'an Institute for Logic, Language and Computation (ILLC) University of Amsterdam {initial.last}@uva.nl Abstract ...


Explorations in Identifying and Summarizing Subjective Content in Text

... puted word embeddings using pre-trained word vectors from GloVe [14] (300-dimensional vectors trained on the 840B token CommonCrawl dataset) to encode our input tokens for both the opinion identification and the opinion summarization task. ...


Exploring Corpora

C Barrière - Natural Language Understanding in a Semantic Web …, 2016

Page 1. Chapter 5 Exploring Corpora In previous chapters, we have worked with very small corpora. Some were even constructed manually to specifically illustrate a particular point, and they have generally contained ten to twenty sentences each. ...


Exploring the Application Potential of Relational Web Tables

... crawls. This situation has changed in 2012 with the University of Mannheim [7] and in 2014 with the Dresden University of Technology [3] starting to extract web table corpora from the CommonCrawl, a large public web corpus. ...


Fast Connected Components Computation in Large Graphs by Vertex Pruning

A Lulli, E Carlini, P Dazzi, C Lucchese, L Ricci

... Consider, for instance, the Web graph (Common Crawl provides 3.5 billion pages with 128 billion hyperlinks[1]), the Linked Open Data datasets (the LOD2 project indexes for 5.7 billion triples/edges [2]) or the Facebook and Twitter social networks (respectively 1.35 billion and ...


Fast Gated Neural Domain Adaptation: Language Model as a Case Study

... The glove 6b model is trained on Wikipedia data and the English Gigaword Fifth Edition corpus;7 the glove 42b model is trained on the Common Crawl data; and the glove 840b model is trained on the the Common Crawl and additional web data. ...


E Shareghi, M Petri, G Haffari, T Cohn - arXiv preprint arXiv:1608.04465, 2016

Page 1. Fast, Small and Exact: Infinite-order Language Modelling with Compressed Suffix Trees Ehsan Shareghi,♭ Matthias Petri,♮ Gholamreza Haffari♭ and Trevor Cohn♮ ♭ Faculty of Information Technology, Monash University ...


FBK HLT-MT Participation in the 1st Translation Memory Cleaning Shared Task

... The English-German corpus was formed using KDE4, GNOME, OpenOffice, PHP, Ubuntu, Tatoeba (Tiedemann, 2012), Europarl v.7 (Koehn, 2005), CommonCrawl (WMT 2013) (Bojar et al., 2013), News Commentary v.11 (WMT 2015) (Bojar et al., 2015), MultiUN (Eisele and ...


Findings of the 2016 Conference on Machine Translation (WMT16)

O Bojar, R Chatterjee, C Federmann, Y Graham…

... Some training corpora were identical from last year (Europarl3, United Nations, French-English 109 corpus, Common Crawl, Russian-English parallel data provided by Yandex, Wikipedia Headlines provided by CMU) and ... Monolingual data sets from CommonCrawl (Buck et al ...


Findings of the WMT 2016 Bilingual Document Alignment Shared Task

C Buck, P Koehn - Proceedings of the First Conference on Machine …, 2016

... Espl`a-Gomis, 2009). 8NLP4TM 2016: Shared task http://rgcl.wlv.ac.uk/nlp4tm2016/ shared-task/ 9http://commoncrawl.org/ 10https://sourceforge.net/p/bitextor/wiki/Home/ 555 Page 3. 3 Training and Test Data We made available ...


Finki at SemEval-2016 Task 4: Deep Learning Architecture for Twitter Sentiment Analysis

D Stojanovski, G Strezoski, G Madjarov, I Dimitrovski - Proceedings of SemEval, 2016

... Our system finki, employs both convolutional and gated recurrent neural networks to obtain a more diverse tweet representation. The network is trained on top of GloVe word embeddings pre-trained on the Common Crawl dataset. ... 154 Page 2. Crawl dataset. ...


Gathering Alternative Surface Forms for DBpedia Entities

... Surface forms have been extracted in a number of works from Wikipedia labels, redirects, disambiguations and anchor texts of internal Wikipedia links, which we complement with anchor texts of external Wikipedia links from the Common Crawl web corpus. ...


Y Wu, M Schuster, Z Chen, QV Le, M Norouzi… - arXiv preprint arXiv: …, 2016

Page 1. Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi yonghui,schuster,zhifengc,qvl,mnorouzi@google.com ...


Grammatical Error Correction: Machine Translation and Classifiers

... Many teams also used native English datasets. The most common ones are the Web1T corpus (Brants and Franz, 2006), the CommonCrawl dataset, which is similar to Web1T, and the English Wikipedia. Several teams used off-the-shelf spellcheckers. ...


Guided Alignment Training for Topic-Aware Neural Machine Translation

W Chen, E Matusov, S Khadivi, JT Peter - arXiv preprint arXiv:1607.01628, 2016

... We first trained a baseline NMT model on English-French WMT data (common-crawl, Europarl v7, and news commentary corpora) for two epochs to get the best result on a development set, and then continued training the same model on the in-domain training set for a few ...


Guided Neural Machine Translation

Page 1. Guided Neural Machine Translation Felix Stahlberg Department of Engineering University of Cambridge This first year report is submitted as part of the degree of Doctor of Philosophy Queens' College August 2016 Page 2. Page 3. Declaration ...


HeLI, a Word-Based Backoff Method for Language Identification

T Jauhiainen, K Lindén, H Jauhiainen - Proceedings of the VarDial Workshop, 2016

... We collected from the Common Crawl 3 corpus all the web pages from the respective domains as in Table 2. When language models were created directly from the pages, the accuracy on the DSL development corpus was 49.86%, which was much ... 3http://commoncrawl.org/ ...


Hybrid Morphological Segmentation for Phrase-Based Machine Translation

SA Grönroos, S Virpioja, M Kurimo - Proceedings of the First Conference on Machine …, 2016

... Due to TheanoLM limitations, only the Europarl and News data (but not Common Crawl) were used for training. ... As monolingual data, we used the Finnish side of Europarl-v8, news.2014.fi.shuffled.v2, news.2015.fi.shuffled and Common Crawl...


IIT Bombay's English-Indonesian submission at WAT: Integrating Neural Language Models with SMT

... Since Commoncrawl provides raw data by web scraping, the Indonesian data obtained was cleaned for noisy sentences and then tokenized and truecased for training the language model. ... statmt. org/europarl/ 6 http://commoncrawl. org/ 70 Page 85. ...


Impact of Data Placement on Resilience in Large-Scale Object Storage Systems

P Carns, K Harms, J Jenkins, M Mubarak, R Ross…

... Figure 2 compares the distribution of file sizes across three different example data populations.1 The first two (the contents of the 1000 Genomes [19] catalog of gene sequencing data and the Common Crawl Corpus [20] catalog of web crawler data) are available as Amazon ...


Improving Translation Selection with Supersenses

H Tang, D Xiong, OL de Lacalle, E Agirre

... We set the Gaussian prior to 1 to avoid overfitting. 4There are 12 subcorpora: commoncrawl, europarl, kde4, news2007, news2008, news2009, news2010, news2011, news2012, newscommentary, openoffice, un 5http://homepages.inf.ed.ac.uk/lzhang10/maxenttoolkit.html ...


INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis

S Ruder12, P Ghaffari, JG Breslin - Proceedings of SemEval, 2016

... respective task. English word embeddings are initialized with 300-dimensional GloVe vectors (Pennington et al., 2014) trained on 840B tokens of the Common Crawl corpus for the unconstrained submission. Word embeddings ...


Is an Image Worth More than a Thousand Words? On the Fine-Grain Semantic Differences between Visual and Linguistic Representations

... 3.2 Word Embeddings We employ 300-dimensional GloVe vectors (Pennington et al., 2014) pre-trained in the largest available corpus (840B tokens and a 2.2M words vocabulary from Common Crawl corpus) from the author's website1. ...


JU-USAAR: A Domain Adaptive MT System

K Pahari, A Kuila, S Pal, SK Naskar, S Bandyopadhyay… - Proceedings of the First …, 2016

... In this task, the information technology (IT) do- main English–German parallel corpus released in the WMT-2016 IT-domain shared task serves as the in-domain data and the Europarl, News and Common Crawl English–German parallel corpus released in the Translation Task ...


Language Models with GloVe Word Embeddings

V Makarenkov, B Shapira, L Rokach - arXiv preprint arXiv:1610.03759, 2016

... Despite the huge size of the Common Crawl corpus, some words may not exist with the embeddings, so we set these words to random vectors, and use the same embeddings consistently if we encounter the same unseen word again in the text. ...


Language Semantic Embeddings in Deep Visual Representation

... As for the final system (shown in Figure 1.1), word representation is learned based on pre-trained GloVe model with Common Crawl data [5]. 2. Examine how to learn visual representation from weakly supervised dataset (Pixabay is the image data source) with ConvNets. ...


Large-scale evaluation of splicing localization algorithms for web images

M Zampoglou, S Papadopoulos, Y Kompatsiaris - Multimedia Tools and Applications, 2016

... In reality, especially for the Web-based forensics case, despite the recent proliferation of PNG files, JPEG remains the norm: it is indicative that among the contents of the Common Crawl corpus,2 87 % of identifiable image suffixes correspond to JPEG (.jpg, .jpeg). ...


Q Ho, J Yin, EP Xing - Journal of Machine Learning Research, 2016

Page 1. Journal of Machine Learning Research 17 (2016) 1-41 Submitted 4/15; Published 4/16 Latent Space Inference of Internet-Scale Networks Qirong Ho∗ hoqirong@gmail.com Institute for Infocomm Research A*STAR Singapore 138632 Junming Yin∗ ...


Learning to recognise named entities in tweets by exploiting weakly labelled data

KJ Espinosa, R Batista-Navarro, S Ananiadou - WNUT 2016, 2016

... Pre-trained Word Embeddings Description Text Type Twitter 2B tweets, 27B tokens, 1.2 M vo- cabulary words, uncased, 100 di- mensions Tweets Common Crawl 840B tokens, 2.2 M vocabulary words, cased, 300 dimensions Web Pages Wikipedia 2014+ Gigaword 5 6B tokens ...


Learning to refine text based recommendations

... Word Vectors: For the ingredient/product prediction task, we used the GloVe pre-trained vectors (Common Crawl, 42 billion tokens, 300dimensional) (Pennington et al., 2014). The word vectors for the AskUbuntu vectors are pre-trained using the AskUbuntu and Wikipedia ...


Page 1. Learning to translate from graded and negative relevance information Laura Jehl Computational Linguistics Heidelberg University 69120 Heidelberg, Germany jehl@cl.uni-heidelberg.de Stefan Riezler Computational ...


Lexical Coherence Graph Modeling Using Word Embeddings

M Mesgar, M Strube - Proceedings of NAACL-HLT, 2016

... 1971). We use a pretrained model of GloVe for word embeddings. This model is trained on Common Crawl with 840B tokens, 2.2M vocabulary. We represent each word by a vector with length 300 (Pennington et al., 2014). For ...


LIMSI@ WMT'16: Machine translation of news

A Allauzen, L Aufrant, F Burlot, E Knyazeva… - Proc. of the ACL 2016 First …, 2016

... Having noticed many sentence alignment errors and out-of-domain parts in the Russian common-crawl parallel corpus, we have used a bilingual sentence aligner3 and proceeded to a domain adaptation filtering using the same procedure as for monolingual data (see ...


Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

M Junczys-Dowmunt, R Grundkiewicz - arXiv preprint arXiv:1605.04800, 2016

... 4. The German monolingual common crawl corpus — a very large resource of raw German text from the Common Crawl project — admissible for the WMT-16 news translation and IT translation tasks. 3.2 Preand post-processing ...


Lurking Malice in the Cloud: Understanding and Detecting Cloud Repository as a Malicious Service

X Liao, S Alrwais, K Yuan, L Xing, XF Wang, S Hao… - Proceedings of the 2016 …, 2016

... Running the scanner over all the data collected by the Common Crawl [?], which indexed five billion web pages, for those associated with all major cloud storage providers (including Amazon S3, Cloudfront, Google Drive, etc.), we found around 1 million sites utilizing 6,885 ...


Machine Translation Quality and Post-Editor Productivity

M Sanchez-Torron, P Koehn - AMTA 2016, Vol., 2016

... corresponding Spanish human reference translations. We trained nine MT systems with training data from the European Parliament proceedings, News Commentary, Common Crawl, and United Nations. The systems are phrase ...


Machine Translation Through Learning From a Communication Game

D He, Y Xia, T Qin, L Wang, N Yu, T Liu, WY Ma - Advances In Neural Information …, 2016

... In detail, we used the same bilingual corpora from WMT'14 as used in [1, 5], which contains 12M sentence pairs extracting from five datasets: Europarl v7, Common Crawl corpus, UN corpus, News Commentary, and 109French-English corpus. ...


Measuring semantic similarity of words using concept networks

G Recski, E Iklódi, K Pajkossy, A Kornai

... We extend this set of models with GloVe vectors4 (Pennington et al., 2014), trained on 840 billion tokens of Common Crawl data5, and the two word embeddings mentioned in Section 1 that have recently been evaluated on the SimLex dataset: the 500-dimension SP model6 ...


Models and Inference for Prefix-Constrained Machine Translation

J Wuebker, S Green, J DeNero, S Hasan, MT Luong

... The English-French bilingual training data consists of 4.9M sentence pairs from the Common Crawl and Europarl corpora from WMT 2015 (Bo- jar et al., 2015). The LM was estimated from the target side of the bitext. For English-German we run large-scale experiments. ...


Multi-cultural Wikipedia mining of geopolitics interactions leveraging reduced Google matrix analysis

KM Frahm, SE Zant, K Jaffrès-Runser… - arXiv preprint arXiv: …, 2016

... At present directed networks of real systems can be very large (about 4.2 million articles for the English Wikipedia edition in 2013 [13] or 3.5 billion web pages for a publicly ac- cessible web crawl that was gathered by the Common Crawl Foundation in 2012 [18]). ...


Multi-Perspective Context Matching for Machine Comprehension

Z Wang, H Mi, W Hamza, R Florian - arXiv preprint arXiv:1612.04211, 2016

... jpurkar et al., 2016). To initialize the word embeddings in the word representation layer, we use the 300-dimensional GloVe word vectors pre-trained from the 840B Common Crawl corpus (Pennington et al., 2014). For the out ...


N-gram language models for massively parallel devices

... benchmark task computes perplexity on data ex- tracted from the Common Crawl dataset used for the 2013 Workshop on Machine Translation, which ... statmt.org/moses/RELEASE-3.0/models/ fr- en/lm/europarl.lm.1 7http://www.statmt.org/wmt13/training-parallelcommoncrawl.tgz ...


Neural Architectures for Fine-grained Entity Type Classification

S Shimaoka, P Stenetorp, K Inui, S Riedel - arXiv preprint arXiv:1606.01341, 2016

... Rocktäschel et al., 2015). For this purpose, we used the freely available 300-dimensional cased word embeddings trained on 840 billion tokens from the Common Crawl supplied by Pennington et al. (2014). For words not present ...


Neural Interactive Translation Prediction

R Knowles, P Koehn - AMTA 2016, Vol., 2016

... The data consists of a 115 million word parallel corpus (Europarl, News Commentary, CommonCrawl), 3http://www. statmt. ... and about 75 billion words of additional English monolingual data (LDC Gigaword, monolingual news, monolingual CommonCrawl). ...


Neural Machine Translation with Pivot Languages

Y Cheng, Y Liu, Q Yang, M Sun, W Xu - arXiv preprint arXiv:1611.04928, 2016

... We use the statistical significance test with paired bootstrap resampling [Koehn, 2004]. Table 1 shows the Spanish-English and English-French corpora from WMT which include Common Crawl, News Commentary, Europarl v7 and UN. ...


Neural Machine Translation with Recurrent Attention Modeling

Z Yang, Z Hu, Y Deng, C Dyer, A Smola - arXiv preprint arXiv:1607.05108, 2016

... 3 Experiments & Results 3.1 Data sets We experiment with two data sets: WMT EnglishGerman and NIST Chinese-English. • English-German The German-English data set contains Europarl, Common Crawl and News Commentary corpus. ...


Neural Network-based Word Alignment through Score Aggregation

J Legrand, M Auli, R Collobert - arXiv preprint arXiv:1606.09560, 2016

... For LSE, we set r = 1 in (4). We initialize the word embeddings with a simple PCA computed over the matrix of word co- occurrence counts (Lebret and Collobert, 2014). The co-occurrence counts were computed over the common crawl corpus provided by WMT16. ...


Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision

C Liang, J Berant, Q Le, KD Forbus, N Lao - arXiv preprint arXiv:1611.00020, 2016

... All the weight matrices are initialized with a uniform distribution in [− √ 3 d , √ 3 d] where d is the input dimension. For pretrained word embeddings, we used the 300 dimension GloVe word embeddings trained on 840B common crawl corpus [? ]. ...


NewsQA: A Machine Comprehension Dataset

A Trischler, T Wang, X Yuan, J Harris, A Sordoni… - arXiv preprint arXiv: …, 2016

... Both mLSTM and BARB are implemented with the Keras framework (Chollet, 2015) using the Theano (Bergstra et al., 2010) backend. Word embeddings are initialized using GloVe vectors (Pennington et al., 2014) pre-trained on the 840-billion Common Crawl corpus. ...


K Heafield, C Geigle, S Massung, L Schwartz - Urbana

Page 1. Normalized Log-Linear Interpolation of Backoff Language Models is Efficient Kenneth Heafield University of Edinburgh 10 Crichton Street Edinburgh EH8 9AB United Kingdom kheafiel@inf.ed.ac.uk Chase Geigle Sean ...


NRC Russian-English Machine Translation System for WMT 2016

C Lo, C Cherry, G Foster, D Stewart, R Islam… - Proceedings of the First …, 2016

... They include the CommonCrawl corpus, the NewsCommentary v11 corpus, the Yandex corpus and the Wikipedia headlines corpus. ... Due to resource limits, we have not used the newly re- leased 3 billion sentence CommonCrawl monolingual English corpus. ...


of Deliverable: Multimedia Linking and Mining

K Andreadou, S Papadopoulos, M Zampoglou… - 2016

Page 1. D3.2 – Multimedia Linking and Mining Version: v1.4 – Final, Date: 02/03/2016 PROJECT TITLE: REVEAL CONTRACT NO. FP7-610928 PROJECT COORDINATOR: INTRASOFT INTERNATIONAL SA WWW.REVEALPROJECT.EU PAGE 1 OF 139 REVEAL FP7-610928 ...


On Approximately Searching for Similar Word Embeddings

... GV 300-dimensional embeddings (Pennington et al., 2014a) learned by the global vectors for word representation (GloVe) model (Pennington et al., 2014b) using Common Crawl corpora, which contain about 2 million words and 42 billion tokens. ...


On Bias-free Crawling and Representative Web Corpora

... Language Resources and Evaluation. Online first: DOI 10.1007/s10579-016-9359-2. Roland Schäfer. 2016b. CommonCOW: Massively huge web corpora from CommonCrawl data and a method to distribute them freely under restrictive EU copyright laws. ...


On the Ubiquity of Web Tracking: Insights from a Billion-Page Web Crawl

S Schelter, J Kunegis - arXiv preprint arXiv:1607.07403, 2016

... We extract third-party embeddings from more than 3.5 billion web pages of the CommonCrawl 2012 corpus, and aggregate those to a dataset containing more than 140 million third-party embeddings in over 41 million domains. ...


Online tracking: A 1-million-site measurement and analysis

... AdFisher builds on similar technologies as OpenWPM (Selenium, xvfb), but is not intended for tracking measurements. Common Crawl4 uses an Apache Nutch based crawler. The Common Crawl dataset is the largest publicly available web crawl5, with billions of page visits. ...


Optimizing Interactive Development of Data-Intensive Applications

M Interlandi, SD Tetali, MA Gulzar, J Noor, T Condie… - Proceedings of the Seventh …, 2016

... 1. 311 service requests dataset. https://data.cityofnewyork.us/Social-Services/311-ServiceRequests-from-2010-to-Present/erm2-nwe9. 2. Common crawl dataset. http://commoncrawl.org. 3. Hadoop. http://hadoop.apache.org. 4. Spark. http://spark.apache.org. 5. WikiReverse. ...


Paragraph Vector for Data Selection in Statistical Machine Translation

... As general domain data we chose the Commoncrawl corpus1 as it is a relatively large corpus and contains crawled data from a variety of domains as well as texts having different discourse types 1http://commoncrawl.org/ (including spoken discourse). ...


A Eisenman, L Cherkasova, G Magalhaes, Q Cai…

Page 1. Parallel Graph Processing on Modern Multi-Core Servers: New Findings and Remaining Challenges Assaf Eisenman1,2, Ludmila Cherkasova2, Guilherme Magalhaes3, Qiong Cai2, Sachin Katti1 1Stanford University ...


ParFDA for Instance Selection for Statistical Machine Translation

E Biçici - Proceedings of the First Conference on Machine …, 2016

... Compared with last year, this year we do not use Common Crawl parallel corpus except for en-ru. We use Common Crawl monolingual corpus fi, ro, and tr datasets and we extended the LM corpora with previous years' corpora. ...


GM Slota, S Rajamanickam, K Devine, K Madduri - arXiv preprint arXiv:1610.07220, 2016

Page 1. Partitioning Trillion-edge Graphs in Minutes George M. Slota Computer Science Department Rensselaer Polytechnic Institute Troy, NY slotag@rpi.edu Sivasankaran Rajamanickam & Karen Devine Scalable Algorithms ...


Page 1. Performance Optimization Techniques and Tools for Distributed Graph Processing VASILIKI KALAVRI School of Information and Communication Technology KTH Royal Institute of Technology Stockholm, Sweden 2016 ...


Phishing Classification using Lexical and Statistical Frequencies of URLs

... We used a sample of 1.2 million phishing URLs extracted from Phishtank and 1.2 million ham URLs from the CommonCrawl corpus to train the model. Classification based on URLs facilitates a defense against all phishing attacks due to the feature they all share, a URL. ...


Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction

M Junczys-Dowmunt, R Grundkiewicz - arXiv preprint arXiv:1605.06353, 2016

... Their method relies on a character-level encoder-decoder recurrent neural network with an attention mechanism. They use data from the public Lang-8 corpus and combine their model with an n-gram language model trained on web-scale Common Crawl data. ...


Phrase-Based SMT for Finnish with More Data, Better Models and Alternative Alignment and Translation Tools

J Tiedemann, F Cap, J Kanerva, F Ginter, S Stymne… - Proceedings of the First …, 2016

... The English language model based on the provided Common Crawl data is limited to trigrams. ... The data is obtained from a large-scale Internet crawl, seeded from all Finnish pages in CommonCrawl.3 However, actual CommonCrawl data is only a small fraction of the total ...


PJAIT Systems for the WMT 2016

K Wołk, K Marasek - Proceedings of the First Conference on Machine …, 2016

... “BASE” in the tables represents the baseline SMT system. “EXT” indicates results for the baseline system, using the baseline settings but extended with additional permissible data (limited to parallel Europarl v7, Common Crawl, ...


Porting an Open Information Extraction System from English to German

T Falke, G Stanovsky, I Gurevych, I Dagan

... For this purpose, we created a new dataset consisting of 300 German sentences, randomly sampled from three sources of different genres: news articles from TIGER (Brants et al., 2004), German web pages from CommonCrawl (Habernal et al., 2016) and featured Wikipedia ...


Practical Variable Length Gap Pattern Matching

J Bader, S Gog, M Petri - Experimental Algorithms, 2016

... implemented on top of SDSL [7] data structures. We use three datasets from different application domains: The CC data set is a \(371 \,\mathrm{GiB}\) prefix of a recent \(145 \,\mathrm{TiB}\) web crawl from commoncrawl.org. ...


Pre-Translation for Neural Machine Translation

J Niehues, E Cho, TL Ha, A Waibel - arXiv preprint arXiv:1610.05243, 2016

... The systems were trained on all parallel data available for the WMT 20161. The news commentary corpus, the European parliament proceedings and the common crawl corpus sum up to 3.7M sentences and around 90M words. ...


Predicting Motivations of Actions by Leveraging Text

C Vondrick, D Oktay, H Pirsiavash, A Torralba - … of the IEEE Conference on Computer …, 2016

... In ECCV. 2012. [5] C. Buck, K. Heafield, and B. van Ooyen. N-gram counts and language models from the common crawl. LREC, 2014. [6] X. Chen, A. Shrivastava, and A. Gupta. Neil: Extracting visual knowledge from web data. In ICCV, 2013. 3004 Page 9. ...


Privacy issues in online machine translation services–European perspective

... 1/11/2014. Retrieved from http://itre.cis.upenn.edu/~myl/languagelog/archives/005 492.html Smith, JR et al. (2013). Dirt cheap web-scale parallel text from the Common Crawl. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. ...


Query Answering to IQ Test Questions Using Word Embedding

M Frąckowiak, J Dutkiewicz, C Jędrzejek, M Retinger… - Multimedia and Network …, 2017

... The pre-trained model based on Google News [16]. Embedding vector size 300, the negative sampling count as 3. 8. Glove Small. Pre-trained model based on Glove approach [20] using common crawl data, accessible on [15]. Embedding vector size 300. 9. Glove Large. ...


Query Expansion with Locally-Trained Word Embeddings

F Diaz, B Mitra, N Craswell - arXiv preprint arXiv:1605.07891, 2016

... the entire corpus. Instead of training a global embedding on the large web collection, we use a GloVe embedding trained on Common Crawl data.4 We train local embeddings using one of three retrieval sources. First, we consider ...


Real-Time Presentation Tracking Using Semantic Keyword Spotting

... gathered from a large corpus. We use a pre-trained vector representation with 1.9 million uncased words and vectors with 300 elements. It was trained using 42 billion tokens of web data from Common Crawl. We will use both the ...


Recurrent versus Recursive Approaches Towards Compositionality in Semantic Vector Spaces

... 0.01, and 1They were in fact trained on 840 billion tokens of Common Crawl data, as in http://nlp.stanford.edu/ projects/glove/. Adagrad (with the default learning rate of 0.01) as our optimizer, with a minibatch size of 300. Addi ...


Relatedness

C Barrière - Natural Language Understanding in a Semantic Web …, 2016

... GloVe has some datasets trained on Wikipedia 2014 + Gigaword 5 (large news corpus) for a total of 6 billion tokens, covering a 400K vocabulary. It has other datasets based on an even larger corpus, the Common Crawl. To ...


Reordering space design in statistical machine translation

N Pécheux, A Allauzen, J Niehues, F Yvon - Language Resources and Evaluation

... in (Allauzen et al. 2013), and, for English-Czech, the Europarl and CommonCrawl parallel WMT'12 corpora. For each task, a 4-gram language model is estimated using the target side of the training data. We use Ncode with ...


Richer Interpolative Smoothing Based on Modified Kneser-Ney Language Modeling

... Interdependency of m, data size, and discounts To explore the correlation between these factors we selected the German and investigated this correlation on two different training data sizes: Europarl (61M words), and CommonCrawl 2014 (984M words). ...


Scaling Up Word Clustering

... The parallel data comes from the WMT-2015 Common Crawl Corpus, News Commentary, Yandex 1M Corpus, and the Wiki Headlines Corpus.7 The monolingual data consists of 2007– 2014 News Commentary and News Crawl articles. ...


Selecting Domain-Specific Concepts for Question Generation With Lightly-Supervised Methods

... 3 Datasets We make use of two datasets obtained from the In- ternet. One is 200k company profiles from CrunchBase. Another is 57k common crawl business news articles. We refer to these two corpora as “Company Profile Corpus” and “News Corpus”. ...


Page 1. Semantic Snippets via Query-Biased Ranking of Linked Data Entities Mazen Alsarem To cite this version: Mazen Alsarem. Semantic Snippets via Query-Biased Ranking of Linked Data Entities. In- formation Retrieval [cs.IR]. ...


Semantic word embedding neural network language models for automatic speech recognition

K Audhkhasi, A Sethy, B Ramabhadran - 2016 IEEE International Conference on …, 2016

... The Gigaword corpus was a suitable choice because of its focus on news domain data in- stead of generic data sets such as Wikipedia or Common Crawl. We used a symmetric window size of 10 words for constructing the word co-occurrence matrix. ...


Semantics derived automatically from language corpora necessarily contain human biases

A Caliskan-Islam, JJ Bryson, A Narayanan - arXiv preprint arXiv:1608.07187, 2016

... Page 9. GloVe authors provide trained embeddings, which is a “Common Crawl” corpus obtained from a large-scale crawl of the web, containing 840 billion tokens (roughly, words). Tokens in this corpus are case-sensitive and ...


Session: P44-Corpus Creation and Querying (1)

MK Bingel, P Banski, A Witt, C Data, F Lefevre, M Diab…

... 960 Roland Schäfer CommonCOW: Massively Huge Web Corpora from CommonCrawl Data and a Method to Distribute them Freely under Restrictive EU Copyright Laws 990 Ioannis Manousos Katakis, Georgios Petasis and Vangelis Karkaletsis ...


Shared Task on Quality Assessment for Text Simplification

S Štajner, M Popovic, H Saggion, L Specia, M Fishel - Training

... The parameters for the ensemble were obtained using particle swarm optimisation under multiple cross-validation scenarios. 2. Treelstm – The metric uses GloVe word vectors8 trained on the Common Crawl corpus and dependency parse trees. ...


Sheffield Systems for the English-Romanian Translation Task

... For the two last, we use subsets of both the News Commentary (93%) and the Common Crawl (13%), selected using XenC- v2.12 (Rousseau, 2013) in mode 23 with the parallel corpora (Europarl7, SETimes2) as in-domain data. ...


SoK: Applying Machine Learning in Security-A Survey

H Jiang, J Nagra, P Ahammad - arXiv preprint arXiv:1611.03186, 2016

Page 1. SoK: Applying Machine Learning in Security - A Survey Heju Jiang* , Jasvir Nagra, Parvez Ahammad ∗ Instart Logic, Inc. {hjiang, jnagra, pahammad }@instartlogic.com ABSTRACT The idea of applying machine learning ...


Source Sentence Simplification for Statistical Machine Translation

E Hasler, A de Gispert, F Stahlberg, A Waite, B Byrne - Computer Speech & Language, 2016

... translation lattices. We trained an English-German system on the WMT 2015 training data (Bojar et al., 2015) comprising 4.2M parallel sentences from the Europarl, News Commentary v10 and Commoncrawl corpora. We word ...


Syntactically Guided Neural Machine Translation

F Stahlberg, E Hasler, A Waite, B Byrne - arXiv preprint arXiv:1605.04569, 2016

... The En-De training set includes Europarl v7, Common Crawl, and News Commentary v10. Sentence pairs with sentences longer than 80 words or length ratios exceeding 2.4:1 were deleted, as were Common Crawl sentences from other languages (Shuyo, 2010). ...


SYSTEMS AND METHODS FOR SPEECH TRANSCRIPTION

A Hannun, C Case, J Casper, B Catanzaro, G Diamos… - US Patent 20,160,171,974, 2016

... the decoding. The language model was trained on 220 million phrases of the Common Crawl (available at commoncrawl.org), selected such that at least 95% of the characters of each phrase were in the alphabet. Only the most ...


TAIPAN: Automatic Property Mapping for Tabular Data

... RAM. Gold Standard We aimed to use T2D entity-level Gold Standard (T2D), a reference dataset which consists of 1 748 tables and reflects the actual distribution of the data in the Common Crawl,5 to evaluate our algorithms. ...


Target-Side Context for Discriminative Models in Statistical Machine Translation

A Tamchyna, A Fraser, O Bojar, M Junczys-Dowmunt - arXiv preprint arXiv: …, 2016

... Our English-German system is trained on the data available for the WMT14 translation task: Europarl (Koehn, 2005) and the Common Crawl corpus,3 roughly 4.3 million sentence pairs altogether. We tune the system on the WMT13 test set and we test on the WMT14 set. ...


TAXI at SemEval-2016 Task 13: a Taxonomy Induction Method based on Lexico-Syntactic Patterns, Substrings and Focused Crawling

A Panchenko, S Faralli, E Ruppert, S Remus, H Naets…

... 59G 59.2 – – – CommonCrawl 168000.0 ‡ – – – FocusedCrawl Food 22.8 7.9 3.4 3.6 ... WebISA. In addition to PattaMaika and PatternSim, we used a publicly available database of English hypernym relations extracted from the CommonCrawl corpus (Seitner et al., 2016). ...


Temporal Attention-Gated Model for Robust Sequence Classification

W Pei, T Baltrušaitis, DMJ Tax, LP Morency - arXiv preprint arXiv:1612.00385, 2016

... 4.2.2 Experimental Setup We utilize 300-d Glove word vectors pretrained over the Common Crawl [27] as the features for each word of the sentences. Our model is well suitable to perform sentiment analysis using sentence-level labels. ...


The 2016 KIT IWSLT Speech-to-Text Systems for English and German

TS Nguyen, M Müller, M Sperber, T Zenkel, K Kilgour…

... Page 4. Text corpus # Words TED 3.6m Fisher 10.4m Switchboard 1.4m TEDLIUM dataselection 155m News + News-commentary + -crawl 4,478m Commoncrawl 185m GIGA 2323m Table 3: English language modeling data. Text corpus # Words ...


The AFRL-MITLL WMT16 News-Translation Task Systems

J Gwinnup, T Anderson, G Erdmann, K Young, M Kazi… - Proceedings of the First …, 2016

... to build a monolithic language model from the following sources: Yandex4, Commoncrawl (Smith et al., 2013), LDC Gigaword English v5 (Parker et al., 2011) and News Commentary. Submission system 1 included the data selected from the large Commoncrawl corpus as ...


The CogALex-V Shared Task on the Corpus-Based Identification of Semantic Relations

E Santus, A Gladkova, S Evert, A Lenci - COLING 2016, 2016

... Team Method (s) Corpus size Corpus GHHH Word analogies, linear regression and multi-task CNN 100B 6B 840B Google News (pre-trained word2vec embeddings, 300 dim.); Wikipedia+ Gigaword 5 (pre-trained GloVe embeddings, 300 dim.), Common Crawl (pre-trained ...


The Edinburgh/LMU Hierarchical Machine Translation System for WMT 2016

M Huck, A Fraser, B Haddow - Proc. of the ACL 2016 First Conf. on Machine …, 2016

... CommonCrawl LM training data in background LM ... Utilizing a larger amount of target-side monolingual resources by appending the CommonCrawl corpus to the background LM's training data is very beneficial and increases the BLEU scores by around one point. ...


The Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16

F Stahlberg, E Hasler, B Byrne - arXiv preprint arXiv:1606.04963, 2016

This paper presents the University of Cambridge submission to WMT16. Motivated by the complementary nature of syntactical machine translation and neural machine translation (NMT), we exploit the synergies of Hiero and NMT in different …


The ILSP/ARC submission to the WMT 2016 Bilingual Document Alignment Shared Task

V Papavassiliou, P Prokopidis, S Piperidis - Proceedings of the First Conference on …, 2016

... 1http://commoncrawl.org/ 2http://nlp.ilsp.gr/redmine/ilsp-fc/ 3Including modules for metadata extraction, language identification, boilerplate removal, document clean-up, text classification and sentence alignment 733 ... Dirt cheap web-scale parallel text from the common crawl...


The JHU Machine Translation Systems for WMT 2016

S Ding, K Duh, H Khayrallah, P Koehn, M Post - … of the First Conference on Machine …, 2016

... In addition, we included a large language model based on the CommonCrawl monolingual data ... of the language model trained on the monomlingual corpora extracted from Common Crawl... year, large corpora of monolingual data were extracted from Common Crawl (Buck et ...


The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2016

TL Ha, E Cho, J Niehues, M Mediani, M Sperber… - Proceedings of the First …, 2016

... To im- prove the quality of the Common Crawl corpus be- ing used in training, we filtered out noisy sentence pairs using an SVM classifier as described in (Me- diani et al., 2011). All of our translation systems are basically phrase-based. ...


The NTNU-YZU System in the AESW Shared Task: Automated Evalua-tion of Scientific Writing Using a Convolutional Neural Network

... For the GloVe representation, we adopted 4 different datasets for training the vectors including one from Wikipedia 2014 and Gigaword 5 (400K vo- cabulary), two common crawl datasets (uncased 1.9M vocabulary, and cased 2.2M vocabulary) and one Twitter dataset (1.2M ...


The RWTH Aachen Machine Translation System for IWSLT 2016

JT Peter, A Guta, N Rossenbach, M Graça, H Ney

... ich war fünf Mal dort oben . . Figure 1: An example of multiple phrasal segmentations taken from the common crawl corpus. The JTR sequence is indicated by blue arcs. The distinct phrasal segmentations are shown in red and shaded green colour. log-linear framework. ...


Topics of Controversy: An Empirical Analysis of Web Censorship Lists

Z Weinberg, M Sharif, J Szurdi, N Christin - Proceedings on Privacy Enhancing …, 2017

... Common Crawl Finally, this is the closest available ap- proximation to an unbiased sample of the entire Web. The Common Crawl Foundation continuously operates a large-scale Web crawl and publishes the results [27]. Each crawl contains at least a billion pages. ...


Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder

TL Ha, J Niehues, A Waibel - arXiv preprint arXiv:1611.04798, 2016

... translation and the web-crawled parallel data (CommonCrawl). ... network. We mix the TED parallel corpus and the substantial monolingual corpus (EPPS+NC+ CommonCrawl) and train a mix-source NMT system from those data. ...


Towards a Complete View of the Certificate Ecosystem

B VanderSloot, J Amann, M Bernhard, Z Durumeric… - 2016

... In 36th IEEE Symposium on Security and Privacy, May 2015. [5] Certificate Transparency: Extended validation in Chrome. https://www.certificate-transparency.org/ev-ct-plan. [6] Common Crawl. https://commoncrawl.org/. [7] The DROWN attack. https://drownattack.com/. ...


Towards More Accurate Statistical Profiling of Deployed schema. org Microdata

R Meusel, D Ritze, H Paulheim - Journal of Data and Information Quality (JDIQ), 2016

... Springer International Publishing. 44. Alex Stolz and Martin Hepp. 2015. Towards crawling the web for structured data: Pitfalls of common crawl for e-commerce. In Proceedings of the 6th International Workshop on Consuming Linked Data (COLD ISWC'15). ...


Translation of Unknown Words in Low Resource Languages

... worthy trade-off. Word Embedding: For this technique, we collect Hindi monolingual data from Wikipedia dump (Al-Rfou et al., 2013) and Commoncrawl (Buck et al., 2014),2 with a total of about 29 million tokens. For Uzbek, the ...


TripleSent: a triple store of events associated with their prototypical sentiment

V Hoste, E Lefever, S van der Waart van Gulik… - eKNOW 2016: The Eighth …, 2016

... These events will be obtained by extracting patterns for highly explicit sentiment expressions (eg, “I hate” or “I love”) or from large web data crawls (eg, commoncrawl.org), which will subsequently be syntactically and semantically parsed to extract events and sentiment triples. ...


Undercounting File Downloads from Institutional Repositories

P OBrien, K Arlitsch, L Sterman, J Mixter, J Wheeler… - Journal of Library …, 2016

Page 1. © Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda Address correspondence to Patrick OBrien, Semantic Web Research Director, Montana State University, PO Box 173320, Bozeman, MT 59717-3320, USA. ...


User Modeling in Language Learning with Macaronic Texts

A Renduchintala, R Knowles, P Koehn, J Eisner - Proceedings of ACL, 2016

... We translated each German sentence using the Moses Statistical Machine Translation (SMT) toolkit (Koehn et al., 2007). The SMT system was trained on the German-English Commoncrawl parallel text used in WMT 2015 (Bojar et al., 2015). ...


Using Feedforward and Recurrent Neural Networks to Predict a Blogger's Age

... The embedding matrix L ∈ RV ×d is initialized for d = 300 with GloVe word vectors trained on the Common Crawl data set [9]. If a token does not correspond to any pre-trained word vector, a random word vector is generated with Xavier initialization [2]. The unembedded vector ...


Vive la petite différence! Exploiting small differences for gender attribution of short texts

F Gralinski, R Jaworski, Ł Borchmann, P Wierzchon

... The procedure of preparing the HSSS corpus was to take Common Crawl-based Web corpus1 of Polish [4] and grep for lines ... Classification with Deep Learning (2015) 4. Buck, C., Heafield, K., van Ooyen, B.: N-gram counts and language models from the common crawl...


Vive la Petite Différence!

F Graliński, R Jaworski, Ł Borchmann, P Wierzchoń - International Conference on …, 2016

... The research was conducted on the publicly available corpus called “He Said She Said”, consisting of a large number of short texts from the Polish version of Common Crawl... Keywords. Gender attribution Text classification Corpus Common Crawl Research reproducibility. ...


VoldemortKG: Mapping schema. org and Web Entities to Linked Open Data

A Tonon, V Felder, DE Difallah, P Cudré-Mauroux

... that apply to the Common Crawl corpus.12 4 The VoldemortKG Knowledge Graph To demonstrate the potential of the dataset we release, we built a proof of concept knowledge graph called VoldemortKG. VoldemortKG integrates schema.org 12 http://commoncrawl.org/terms ...


What does the Web remember of its deleted past? An archival reconstruction of the former Yugoslav top-level domain

... The completeness of the reconstruction effort could have been aided by consulting other large repositories of temporal Web data, such as Common Crawl, or by simply contacting the Internet Archive and requesting for all domains in the .yu domain. ...


What Makes Word-level Neural Machine Translation Hard: A Case Study on English-German Translation

... 5.1 Dataset & Preprocessing Our models were trained on the data provided by the 2014 Workshop on Machine Translation (WMT). Specifically, we used the Europarl v7, Common Crawl, and News Commentary corpora. Our ...


WHAT: A Big Data Approach for Accounting of Modern Web Services

M Trevisan, I Drago, M Mellia, HH Song, M Baldi - 2016

... [5] D. Plonka and P. Barford, “Flexible Traffic and Host Profiling via DNS Rendezvous,” in Proc. of the SATIN, 2011, pp. 1–8. [6] “Common Crawl,” http://commoncrawl.org/. [7] A. Finamore et al., “Experiences of Internet Traffic Monitoring with Tstat,” IEEE Netw., vol. 25, no. 3, pp. ...


Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM

... Memory (BLSTM) neural network for end-to-end processing.9 The input layer relies on pre-trained word embeddings, in particular GloVe (Pennington et al., 2014) trained on 840B tokens from Common Crawl;10 the embedding weights are further updated during training. ...


Wikipedia mining of hidden links between political leaders

KM Frahm, K Jaffrès-Runser, DL Shepelyansky - arXiv preprint arXiv:1609.01948, 2016

... At present directed networks of real systems can be very large (about 4.2 million articles for the English Wikipedia edition in 2013 [10] or 3.5 billion web pages for a publicly accessible web crawl that was gathered by the Common Crawl Foundation in 2012 [28]). ...


WOLVESAAR at SemEval-2016 Task 1: Replicating the Success of Monolingual Word Alignment and Neural Embeddings for Semantic Textual Similarity

H Bechara, R Gupta, L Tan, C Orasan, R Mitkov… - Proceedings of SemEval, 2016

... 2We use the 300 dimensions vectors from the GloVe model trained on the Commoncrawl Corpus with 840B tokens, 2.2M vocabulary. distributions p and pθ using regularised KullbackLeibler (KL) divergence. J(θ) = 1 n n ∑ i=1KL(p(i)∣ ∣ ∣ ∣ p (i) θ ) + λ2||θ||2 2 (8) ...


Word Representation on Small Background Texts

L Li, Z Jiang, Y Liu, D Huang - Chinese National Conference on Social Media …, 2016

... For example, Pennington et al. (2014) used Wikipedia, Gigaword 5 and Common Crawl to learn word representations, each of which contained billions of tokens. There was not always a monotonic increase in performance as the amount of background texts increased. ...


Word2Vec vs DBnary: Augmenting METEOR using Vector Representations or Lexical Resources?

C Servan, A Berard, Z Elloumi, H Blanchon, L Besacier - arXiv preprint arXiv: …, 2016

C Servan, A Bérard, Z Elloumi, H Blanchon, L Besacier

... German–English Europarl V7 + news commentary V10 2.1 M 57.2 M 59.7 M Russian–English Common Crawl + news commentary V10 + Yandex 2.0 M 47.2 M 50.3 M Table 2: Bilingual corpora used to train the word embeddings for each language pair. ...


Yandex School of Data Analysis approach to English-Turkish translation at WMT16 News Translation Task

A Dvorkovich, S Gubanov, I Galinskaya - Proceedings of the First Conference on …, 2016

... 2.7 Data For training translation model, language models, and NMT reranker, we used only the provided constrained data (SETIMES 2 parallel TurkishEnglish corpus, and monolingual Turkish and En- glish Common Crawl corpora). ...