Discovering roll-up dependencies


Ordering patterns by combining opinions from multiple sources Pattern ordering is an important task in data mining because the number of patterns extracted by standard data mining algorithms often exceeds our capacity to manually analyze them . In this paper , we present an effective approach to address the pattern ordering problem by combining the rank information gathered from disparate sources . Although rank aggregation techniques have been developed for applications such as meta-search engines , they are not directly applicable to pattern ordering for two reasons . First , the techniques are mostly supervised , i.e. , they require a sufficient amount of labeled data . Second , the objects to be ranked are assumed to be independent and identically distributed ( i.i. d ) , an assumption that seldom holds in pattern ordering . The method proposed in this paper is an adaptation of the original Hedge algorithm , modified to work in an unsupervised learning setting . Techniques for addressing the i.i.d. violation in pattern ordering are also presented . Experimental results demonstrate that our unsupervised Hedge algorithm outperforms many alternative techniques such as those based on weighted average ranking and singular value decomposition .

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Colibri : fast mining of large static and dynamic graphs Low-rank approximations of the adjacency matrix of a graph are essential in finding patterns ( such as communities ) and detecting anomalies . Additionally , it is desirable to track the low-rank structure as the graph evolves over time , efficiently and within limited storage . Real graphs typically have thousands or millions of nodes , but are usually very sparse . However , standard decompositions such as SVD do not preserve sparsity . This has led to the development of methods such as CUR and CMD , which seek a non-orthogonal basis by sampling the columns and\/or rows of the sparse matrix . However , these approaches will typically produce overcomplete bases , which wastes both space and time . In this paper we propose the family of Colibri methods to deal with these challenges . Our version for static graphs , Colibri-S , iteratively finds a non-redundant basis and we prove that it has no loss of accuracy compared to the best competitors ( CUR and CMD ) , while achieving significant savings in space and time : on real data , Colibri-S requires much less space and is orders of magnitude faster ( in proportion to the square of the number of non-redundant columns ) . Additionally , we propose an efficient update algorithm for dynamic , time-evolving graphs , Colibri-D . Our evaluation on a large , real network traffic dataset shows that Colibri-D is over 100 times faster than the best published competitor ( CMD ) .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graph mining
low-rank approximation
scalability


A probabilistic framework for semi-supervised clustering Unsupervised clustering can be significantly improved using supervision in the form of pairwise constraints , i.e. , pairs of instances labeled as belonging to same or different clusters . In recent years , a number of algorithms have been proposed for enhancing clustering quality by employing such supervision . Such methods use the constraints to either modify the objective function , or to learn the distance measure . We propose a probabilistic model for semi-supervised clustering based on Hidden Markov Random Fields ( HMRFs ) that provides a principled framework for incorporating supervision into prototype-based clustering . The model generalizes a previous approach that combines constraints and Euclidean distance learning , and allows the use of a broad range of clustering distortion measures , including Bregman divergences ( e.g. , Euclidean distance and I-divergence ) and directional similarity measures ( e.g. , cosine similarity ) . We present an algorithm that performs partitional semi-supervised clustering of data by minimizing an objective function derived from the posterior energy of the HMRF model . Experimental results on several text data sets demonstrate the advantages of the proposed framework .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

distance metric learning
semi-supervised clustering


Unsupervised learning on k-partite graphs Various data mining applications involve data objects of multiple types that are related to each other , which can be naturally formulated as a k-partite graph . However , the research on mining the hidden structures from a k-partite graph is still limited and preliminary . In this paper , we propose a general model , the relation summary network , to find the hidden structures ( the local cluster structures and the global community structures ) from a k-partite graph . The model provides a principal framework for unsupervised learning on k-partite graphs of various structures . Under this model , we derive a novel algorithm to identify the hidden structures of a k-partite graph by constructing a relation summary network to approximate the original k-partite graph under a broad range of distortion measures . Experiments on both synthetic and real datasets demonstrate the promise and effectiveness of the proposed model and algorithm . We also establish the connections between existing clustering approaches and the proposed model to provide a unified view to the clustering approaches .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

relation summary network
unsupervised learning


Evaluating the novelty of text-mined rules using lexical knowledge In this paper , we present a new method of estimating the novelty of rules discovered by data-mining methods using WordNet , a lexical knowledge-base of English words . We assess the novelty of a rule by the average semantic distance in a knowledge hierarchy between the words in the antecedent and the consequent of the rule - the more the average distance , more is the novelty of the rule . The novelty of rules extracted by the DiscoTEX text-mining system on Amazon.com book descriptions were evaluated by both human subjects and by our algorithm . By computing correlation coefficients between pairs of human ratings and between human and automatic ratings , we found that the automatic scoring of rules based on our novelty measure correlates with human judgments about as well as human judgments correlate with one another . @Text mining

O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O B O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O

knowledge hierarchy
novelty
semantic distance
wordnet


A new two-phase sampling based algorithm for discovering association rules This paper introduces FAST , a novel two-phase sampling-based algorithm for discovering association rules in large databases . In Phase I a large initial sample of transactions is collected and used to quickly and accurately estimate the support of each individual item in the database . In Phase II these estimated supports are used to either trim `` outlier '' transactions or select `` representative '' transactions from the initial sample , thereby forming a small final sample that more accurately reflects the statistical characteristics ( i.e. , itemset supports ) of the entire database . The expensive operation of discovering association rules is then performed on the final sample . In an empirical study , FAST was able to achieve 90 -- 95 % accuracy using a final sample having a size of only 15 -- 33 % of that of a comparable random sample . This efficiency gain resulted in a speedup by roughly a factor of 10 over previous algorithms that require expensive processing of the entire database -- even efficient algorithms that exploit sampling . Our new sampling technique can be used in conjunction with almost any standard association-rule algorithm , and can potentially render scalable other algorithms that mine `` count '' data .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Sparsification of influence networks We present Spine , an efficient algorithm for finding the `` backbone '' of an influence network . Given a social graph and a log of past propagations , we build an instance of the independent-cascade model that describes the propagations . We aim at reducing the complexity of that model , while preserving most of its accuracy in describing the data . We show that the problem is inapproximable and we present an optimal , dynamic-programming algorithm , whose search space , albeit exponential , is typically much smaller than that of the brute force , exhaustive-search approach . Seeking a practical , scalable approach to sparsification , we devise Spine , a greedy , efficient algorithm with practically little compromise in quality . We claim that sparsification is a fundamental data-reduction operation with many applications , ranging from visualization to exploratory and descriptive data analysis . As a proof of concept , we use Spine on real-world datasets , revealing the backbone of their influence-propagation networks . Moreover , we apply Spine as a pre-processing step for the influence-maximization problem , showing that computations on sparsified models give up little accuracy , but yield significant improvements in terms of scalability .

O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Handling very large numbers of association rules in the analysis of microarray data The problem of analyzing microarray data became one of important topics in bioinformatics over the past several years , and different data mining techniques have been proposed for the analysis of such data . In this paper , we propose to use association rule discovery methods for determining associations among expression levels of different genes . One of the main problems related to the discovery of these associations is the scalability issue . Microarrays usually contain very large numbers of genes that are sometimes measured in 10,000 s. Therefore , analysis of such data can generate a very large number of associations that can often be measured in millions . The paper addresses this problem by presenting a method that enables biologists to evaluate these very large numbers of discovered association rules during the post-analysis stage of the data mining process . This is achieved by providing several rule evaluation operators , including rule grouping , filtering , browsing , and data inspection operators , that allow biologists to validate multiple individual gane regulation patterns at a time . By iteratively applying these operators , biologists can explore a significant part of all the initially generated rules in an acceptable period of time and thus answer biological questions that are of a particular interest to him or her . To validate our method , we tested our system on the microarray data pertaining to the studies of environmental hazards and their influence of gane expression processes . As a result , we managed to answer several questions that were of interest to the biologists that had collected this data .

O O O O O B I O O B I I I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

analysis of microarray data
association rules
bioinformatics
rule grouping

deduction
expert-driven rule validation
post-processing of discovered rules
rule filtering


Experiences with mining temporal event sequences from electronic medical records : initial successes and some challenges The standardization and wider use of electronic medical records ( EMR ) creates opportunities for better understanding patterns of illness and care within and across medical systems . Our interest is in the temporal history of event codes embedded in patients ' records , specifically investigating frequently occurring sequences of event codes across patients . In studying data from more than 1.6 million patient histories at the University of Michigan Health system we quickly realized that frequent sequences , while providing one level of data reduction , still constitute a serious analytical challenge as many involve alternate serializations of the same sets of codes . To further analyze these sequences , we designed an approach where a partial order is mined from frequent sequences of codes . We demonstrate an EMR mining system called EMRView that enables exploration of the precedence relationships to quickly identify and visualize partial order information encoded in key classes of patients . We demonstrate some important nuggets learned through our approach and also outline key challenges for future research based on our experiences .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

medical informatics
partial orders
temporal data mining


Pattern-based similarity search for microarray data One fundamental task in near-neighbor search as well as other similarity matching efforts is to find a distance function that can efficiently quantify the similarity between two objects in a meaningful way . In DNA microarray analysis , the expression levels of two closely related genes may rise and fall synchronously in response to a set of experimental stimuli . Although the magnitude of their expression levels may not be close , the patterns they exhibit can be very similar . Unfortunately , none of the conventional distance metrics such as the Lp norm can model this similarity effectively . In this paper , we study the near-neighbor search problem based on this new type of similarity . We propose to measure the distance between two genes by subspace pattern similarity , i.e. , whether they exhibit a synchronous pattern of rise and fall on a subset of dimensions . We then present an efficient algorithm for subspace near-neighbor search based on pattern similarity distance , and we perform tests on various data sets to show its effectiveness .

O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Using structure indices for efficient approximation of network properties Statistics on networks have become vital to the study of relational data drawn from areas such as bibliometrics , fraud detection , bioinformatics , and the Internet . Calculating many of the most important measures - such as betweenness centrality , closeness centrality , and graph diameter-requires identifying short paths in these networks . However , finding these short paths can be intractable for even moderate-size networks . We introduce the concept of a network structure index ( NSI ) , a composition of ( 1 ) a set of annotations on every node in the network and ( 2 ) a function that uses the annotations to estimate graph distance between pairs of nodes . We present several varieties of NSIs , examine their time and space complexity , and analyze their performance on synthetic and real data sets . We show that creating an NSI for a given network enables extremely efficient and accurate estimation of a wide variety of network statistics on that network .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

knowledge discovery in graphs
social network analysis


Bayesian networks for lossless dataset compression

uncertainty, fuzzy, and probabilistic reasoning


Application of kernels to link analysis The application of kernel methods to link analysis is explored . In particular , Kandola et al. 's Neumann kernels are shown to subsume not only the co-citation and bibliographic coupling relatedness but also Kleinberg 's HITS importance . These popular measures of relatedness and importance correspond to the Neumann kernels at the extremes of their parameter range , and hence these kernels can be interpreted as defining a spectrum of link analysis measures intermediate between co-citation\/bibliographic coupling and HITS . We also show that the kernels based on the graph Laplacian , including the regularized Laplacian and diffusion kernels , provide relatedness measures that overcome some limitations of co-citation relatedness . The property of these kernel-based link analysis measures is examined with a network of bibliographic citations . Practical issues in applying these methods to real data are discussed , and possible solutions are proposed .

O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O

co-citation coupling
graph kernel
information search and retrieval


Direct mining of discriminative and essential frequent patterns via model-based search tree Frequent patterns provide solutions to datasets that do not have well-structured feature vectors . However , frequent pattern mining is non-trivial since the number of unique patterns is exponential but many are non-discriminative and correlated . Currently , frequent pattern mining is performed in two sequential steps : enumerating a set of frequent patterns , followed by feature selection . Although many methods have been proposed in the past few years on how to perform each separate step efficiently , there is still limited success in eventually finding highly compact and discriminative patterns . The culprit is due to the inherent nature of this widely adopted two-step approach . This paper discusses these problems and proposes a new and different method . It builds a decision tree that partitions the data onto different nodes . Then at each node , it directly discovers a discriminative pattern to further divide its examples into purer subsets . Since the number of examples towards leaf level is relatively small , the new approach is able to examine patterns with extremely low global support that could not be enumerated on the whole dataset by the two-step method . The discovered feature vectors are more accurate on some of the most difficult graph as well as frequent itemset problems than most recently proposed algorithms but the total size is typically 50 % or more smaller . Importantly , the minimum support of some discriminative patterns can be extremely low ( e.g. 0.03 % ) . In order to enumerate these low support patterns , state-of-the-art frequent pattern algorithm either can not finish due to huge memory consumption or have to enumerate 101 to 103 times more patterns before they can even be found . Software and datasets are available by contacting the author .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Partial least squares regression for graph mining Attributed graphs are increasingly more common in many application domains such as chemistry , biology and text processing . A central issue in graph mining is how to collect informative subgraph patterns for a given learning task . We propose an iterative mining method based on partial least squares regression ( PLS ) . To apply PLS to graph data , a sparse version of PLS is developed first and then it is combined with a weighted pattern mining algorithm . The mining algorithm is iteratively called with different weight vectors , creating one latent component per one mining call . Our method , graph PLS , is efficient and easy to implement , because the weight vector is updated with elementary matrix calculations . In experiments , our graph PLS algorithm showed competitive prediction accuracies in many chemical datasets and its efficiency was significantly superior to graph boosting ( gBoost ) and the naive method based on frequent graph mining .

B I I I O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O

graph boosting
graph mining
partial least squares regression


Supervised probabilistic principal component analysis Principal component analysis ( PCA ) has been extensively applied in data mining , pattern recognition and information retrieval for unsupervised dimensionality reduction . When labels of data are available , e.g. , in a classification or regression task , PCA is however not able to use this information . The problem is more interesting if only part of the input data are labeled , i.e. , in a semi-supervised setting . In this paper we propose a supervised PCA model called SPPCA and a semi-supervised PCA model called S2PPCA , both of which are extensions of a probabilistic PCA model . The proposed models are able to incorporate the label information into the projection phase , and can naturally handle multiple outputs ( i.e. , in multi-task learning problems ) . We derive an efficient EM learning algorithm for both models , and also provide theoretical justifications of the model behaviors . SPPCA and S2PPCA are compared with other supervised projection methods on various learning tasks , and show not only promising performance but also good scalability .

O O B I I B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O

dimensionality reduction
principal component analysis
supervised projection


Frequent subgraph mining in outerplanar graphs In recent years there has been an increased interest in algorithms that can perform frequent pattern discovery in large databases of graph structured objects . While the frequent connected subgraph mining problem for tree datasets can be solved in incremental polynomial time , it becomes intractable for arbitrary graph databases . Existing approaches have therefore resorted to various heuristic strategies and restrictions of the search space , but have not identified a practically relevant tractable graph class beyond trees . In this paper , we define the class of so called tenuous outerplanar graphs , a strict generalization of trees , develop a frequent subgraph mining algorithm for tenuous outerplanar graphs that works in incremental polynomial time , and evaluate the algorithm empirically on the NCI molecular graph dataset .

O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A fast algorithm for finding frequent episodes in event streams Frequent episode discovery is a popular framework for mining data available as a long sequence of events . An episode is essentially a short ordered sequence of event types and the frequency of an episode is some suitable measure of how often the episode occurs in the data sequence . Recently , we proposed a new frequency measure for episodes based on the notion of non-overlapped occurrences of episodes in the event sequence , and showed that , such a definition , in addition to yielding computationally efficient algorithms , has some important theoretical properties in connecting frequent episode discovery with HMM learning . This paper presents some new algorithms for frequent episode discovery under this non-overlapped occurrences-based frequency definition . The algorithms presented here are better ( by a factor of N , where N denotes the size of episodes being discovered ) in terms of both time and space complexities when compared to existing methods for frequent episode discovery . We show through some simulation experiments , that our algorithms are very efficient . The new algorithms presented here have arguably the least possible orders of spaceand time complexities for the task of frequent episode discovery .

O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

non-overlapped occurrences
temporal data mining


Machine learning for online query relaxation In this paper we provide a fast , data-driven solution to the failing query problem : given a query that returns an empty answer , how can one relax the query 's constraints so that it returns a non-empty set of tuples ? We introduce a novel algorithm , loqr , which is designed to relax queries that are in the disjunctive normal form and contain a mixture of discrete and continuous attributes . loqr discovers the implicit relationships that exist among the various domain attributes and then uses this knowledge to relax the constraints from the failing query . In a first step , loqr uses a small , randomly-chosen subset of the target database to learn a set of decision rules that predict whether an attribute 's value satisfies the constraints in the failing query ; this query-driven operation is performed online for each failing query . In the second step , loqr uses nearest-neighbor techniques to find the learned rule that is the most similar to the failing query ; then it uses the attributes ' values from this rule to relax the failing query 's constraints . Our experiments on six application domains show that loqr is both robust and fast : it successfully relaxes more than 95 % of the failing queries , and it takes under a second for processing queries that consist of up to 20 attributes ( larger queries of up to 93 attributes are processed in several seconds ) .

O B O B I I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

failing query
learning
online query relaxation

nearest neighbor
rule learning
web-based information sources


Visual classification : an interactive approach to decision tree construction


SyMP : an efficient clustering approach to identify clusters of arbitrary shapes in large data sets We propose a new clustering algorithm , called SyMP , which is based on synchronization of pulse-coupled oscillators . SyMP represents each data point by an Integrate-and-Fire oscillator and uses the relative similarity between the points to model the interaction between the oscillators . SyMP is robust to noise and outliers , determines the number of clusters in an unsupervised manner , identifies clusters of arbitrary shapes , and can handle very large data sets . The robustness of SyMP is an intrinsic property of the synchronization mechanism . To determine the optimum number of clusters , SyMP uses a dynamic resolution parameter . To identify clusters of various shapes , SyMP models each cluster by multiple Gaussian components . The number of components is automatically determined using a dynamic intra-cluster resolution parameter . Clusters with simple shapes would be modeled by few components while clusters with more complex shapes would require a larger number of components . The scalable version of SyMP uses an efficient incremental approach that requires a simple pass through the data set . The proposed clustering approach is empirically evaluated with several synthetic and real data sets , and its performance is compared with CURE .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Towards NIC-based intrusion detection We present and evaluate a NIC-based network intrusion detection system . Intrusion detection at the NIC makes the system potentially tamper-proof and is naturally extensible to work in a distributed setting . Simple anomaly detection and signature detection based models have been implemented on the NIC firmware , which has its own processor and memory . We empirically evaluate such systems from the perspective of quality and performance ( bandwidth of acceptable messages ) under varying conditions of host load . The preliminary results we obtain are very encouraging and lead us to believe that such NIC-based security schemes could very well be a crucial part of next generation network security systems .

O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O

network intrusion detection
network security

data mining
network interface cards
nics


Spatial scan statistics : approximations and performance study Spatial scan statistics are used to determine hotspots in spatial data , and are widely used in epidemiology and biosurveillance . In recent years , there has been much effort invested in designing efficient algorithms for finding such `` high discrepancy '' regions , with methods ranging from fast heuristics for special cases , to general grid-based methods , and to efficient approximation algorithms with provable guarantees on performance and quality . In this paper , we make a number of contributions to the computational study of spatial scan statistics . First , we describe a simple exact algorithm for finding the largest discrepancy region in a domain . Second , we propose a new approximation algorithm for a large class of discrepancy functions ( including the Kulldorff scan statistic ) that improves the approximation versus run time trade-off of prior methods . Third , we extend our simple exact and our approximation algorithms to data sets which lie naturally on a grid or are accumulated onto a grid . Fourth , we conduct a detailed experimental comparison of these methods with a number of known methods , demonstrating that our approximation algorithm has far superior performance in practice to prior methods , and exhibits a good performance-accuracy trade-off . All extant methods ( including those in this paper ) are suitable for data sets that are modestly sized ; if data sets are of the order of millions of data points , none of these methods scale well . For such massive data settings , it is natural to examine whether small-space streaming algorithms might yield accurate answers . Here , we provide some negative results , showing that any streaming algorithms that even provide approximately optimal answers to the discrepancy maximization problem must use space linear in the input .

B I I O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O

discrepancy
kulldorff scan statistic
spatial scan statistics


Assessment and pruning of hierarchical model based clustering The goal of clustering is to identify distinct groups in a dataset . The basic idea of model-based clustering is to approximate the data density by a mixture model , typically a mixture of Gaussians , and to estimate the parameters of the component densities , the mixing fractions , and the number of components from the data . The number of distinct groups in the data is then taken to be the number of mixture components , and the observations are partitioned into clusters ( estimates of the groups ) using Bayes ' rule . If the groups are well separated and look Gaussian , then the resulting clusters will indeed tend to be `` distinct '' in the most common sense of the word - contiguous , densely populated areas of feature space , separated by contiguous , relatively empty regions . If the groups are not Gaussian , however , this correspondence may break down ; an isolated group with a non-elliptical distribution , for example , may be modeled by not one , but several mixture components , and the corresponding clusters will no longer be well separated . We present methods for assessing the degree of separation between the components of a mixture model and between the corresponding clusters . We also propose a new clustering method that can be regarded as a hybrid between model-based and nonparametric clustering . The hybrid clustering algorithm prunes the cluster tree generated by hierarchical model-based clustering . Starting with the tree corresponding to the mixture model chosen by the Bayesian Information Criterion , it progressively merges clusters that do not appear to correspond to different modes of the data density .

O O O O O O O B O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

density estimation
model-based clustering
nonparametric clustering
unimodality


Density-based clustering of uncertain data In many different application areas , e.g. sensor databases , location based services or face recognition systems , distances between odjects have to be computed based on vague and uncertain data . Commonly , the distances between these uncertain object descriptions are expressed by one numerical distance value . Based on such single-valued distance functions standard data mining algorithms can work without any changes . In this paper , we propose to express the similarity between two fuzzy objects by distance probability functions . These fuzzy distance functions assign a probability value to each possible distance value . By integrating these fuzzy distance functions directly into data mining algorithms , the full information provided by these functions is exploited . In order to demonstrate the benefits of this general approach , we enhance the density-based clustering algorithm DBSCAN so that it can work directly on these fuzzy distance functions . In a detailed experimental evaluation based on artificial and real-world data sets , we show the characteristics and benefits of our new approach .

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O

density-based clustering
probabilistic algorithms


SNARE : a link analytic system for graph labeling and risk detection Classifying nodes in networks is a task with a wide range of applications . It can be particularly useful in anomaly and fraud detection . Many resources are invested in the task of fraud detection due to the high cost of fraud , and being able to automatically detect potential fraud quickly and precisely allows human investigators to work more efficiently . Many data analytic schemes have been put into use ; however , schemes that bolster link analysis prove promising . This work builds upon the belief propagation algorithm for use in detecting collusion and other fraud schemes . We propose an algorithm called SNARE ( Social Network Analysis for Risk Evaluation ) . By allowing one to use domain knowledge as well as link knowledge , the method was very successful for pinpointing misstated accounts in our sample of general ledger data , with a significant improvement over the default heuristic in true positive rates , and a lift factor of up to 6.5 ( more than twice that of the default heuristic ) . We also apply SNARE to the task of graph labeling in general on publicly-available datasets . We show that with only some information about the nodes themselves in a network , we get surprisingly high accuracy of labels . Not only is SNARE applicable in a wide variety of domains , but it is also robust to the choice of parameters and highly scalable-linearly with the number of edges in a graph .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Style mining of electronic messages for multiple authorship discrimination : first results This paper considers the use of computational stylistics for performing authorship attribution of electronic messages , addressing categorization problems with as many as 20 different classes ( authors ) . Effective stylistic characterization of text is potentially useful for a variety of tasks , as language style contains cues regarding the authorship , purpose , and mood of the text , all of which would be useful adjuncts to information retrieval or knowledge-management tasks . We focus here on the problem of determining the author of an anonymous message , based only on the message text . Several multiclass variants of the Winnow algorithm were applied to a vector representation of the message texts to learn models for discriminating different authors . We present results comparing the classification accuracy of the different approaches . The results show that stylistic models can be accurately learned to determine an author 's identity .

O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

authorship attribution
computational stylistics

electronic communication
information search and retrieval
learning
text categorization
text mining


Active learning with direct query construction Active learning may hold the key for solving the data scarcity problem in supervised learning , i.e. , the lack of labeled data . Indeed , labeling data is a costly process , yet an active learner may request labels of only selected instances , thus reducing labeling work dramatically . Most previous works of active learning are , however , pool-based ; that is , a pool of unlabeled examples is given and the learner can only select examples from the pool to query for their labels . This type of active learning has several weaknesses . In this paper we propose novel active learning algorithms that construct examples directly to query for labels . We study both a specific active learner based on the decision tree algorithm , and a general active learner that can work with any base learning algorithm . As there is no restriction on what examples to be queried , our methods are shown to often query fewer examples to reduce the predictive error quickly . This casts doubt on the usefulness of the pool in pool-based active learning . Nevertheless , our methods can be easily adapted to work with a given pool of unlabeled examples .

B I O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O


Classification with partial labels In this paper , we address the problem of learning when some cases are fully labeled while other cases are only partially labeled , in the form of partial labels . Partial labels are represented as a set of possible labels for each training example , one of which is the correct label . We introduce a discriminative learning approach that incorporates partial label information into the conventional margin-based learning framework . The partial label learning problem is formulated as a convex quadratic optimization minimizing the L2-norm regularized empirical risk using hinge loss . We also present an efficient algorithm for classification in the presence of partial labels . Experiments with different data sets show that partial label information improves the performance of classification when there is traditional fully-labeled data , and also yields reasonable performance in the absence of any fully labeled data .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


iSAX : indexing and mining terabyte sized time series Current research in indexing and mining time series data has produced many interesting algorithms and representations . However , the algorithms and the size of data considered have generally not been representative of the increasingly massive datasets encountered in science , engineering , and business domains . In this work , we show how a novel multi-resolution symbolic representation can be used to index datasets which are several orders of magnitude larger than anything else considered in the literature . Our approach allows both fast exact search and ultra fast approximate search . We show how to exploit the combination of both types of search as sub-routines in data mining algorithms , allowing for the exact mining of truly massive real world datasets , containing millions of time series .

O O B O O O O B I O O O B O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O


Maximally informative k-itemsets and their efficient discovery In this paper we present a new approach to mining binary data . We treat each binary feature ( item ) as a means of distinguishing two sets of examples . Our interest is in selecting from the total set of items an itemset of specified size , such that the database is partitioned with as uniform a distribution over the parts as possible . To achieve this goal , we propose the use of joint entropy as a quality measure for itemsets , and refer to optimal itemsets of cardinality k as maximally informative k-itemsets . We claim that this approach maximises distinctive power , as well as minimises redundancy within the feature set . A number of algorithms is presented for computing optimal itemsets efficiently .

O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

feature selection
information theory
learning
maximally informative k-itemsets
subgroup discovery
systems and information theory


Trading representability for scalability : adaptive multi-hyperplane machine for nonlinear classification Support Vector Machines ( SVMs ) are among the most popular and successful classification algorithms . Kernel SVMs often reach state-of-the-art accuracies , but suffer from the curse of kernelization due to linear model growth with data size on noisy data . Linear SVMs have the ability to efficiently learn from truly large data , but they are applicable to a limited number of domains due to low representational power . To fill the representability and scalability gap between linear and nonlinear SVMs , we propose the Adaptive Multi-hyperplane Machine ( AMM ) algorithm that accomplishes fast training and prediction and has capability to solve nonlinear classification problems . AMM model consists of a set of hyperplanes ( weights ) , each assigned to one of the multiple classes , and predicts based on the associated class of the weight that provides the largest prediction . The number of weights is automatically determined through an iterative algorithm based on the stochastic gradient descent algorithm which is guaranteed to converge to a local optimum . Since the generalization bound decreases with the number of weights , a weight pruning mechanism is proposed and analyzed . The experiments on several large data sets show that AMM is nearly as fast during training and prediction as the state-of-the-art linear SVM solver and that it can be orders of magnitude faster than kernel SVM . In accuracy , AMM is somewhere between linear and kernel SVMs . For example , on an OCR task with 8 million highly dimensional training examples , AMM trained in 300 seconds on a single-core processor had 0.54 % error rate , which was significantly lower than 2.03 % error rate of a linear SVM trained in the same time and comparable to 0.43 % error rate of a kernel SVM trained in 2 days on 512 processors . The results indicate that AMM could be an attractive option when solving large-scale classification problems . The software is available at www.dabi.temple.edu\/~vucetic\/AMM.html .

O O O O O O O O O B I B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

nonlinear classification
stochastic gradient descent
support vector machines


A sequential dual method for large scale multi-class linear svms Efficient training of direct multi-class formulations of linear Support Vector Machines is very useful in applications such as text classification with a huge number examples as well as features . This paper presents a fast dual method for this training . The main idea is to sequentially traverse through the training set and optimize the dual variables associated with one example at a time . The speed of training is enhanced further by shrinking and cooling heuristics . Experiments indicate that our method is much faster than state of the art solvers such as bundle , cutting plane and exponentiated gradient methods .

O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

support vector machines
text classification


Correlation search in graph databases Correlation mining has gained great success in many application domains for its ability to capture the underlying dependency between objects . However , the research of correlation mining from graph databases is still lacking despite the fact that graph data , especially in various scientific domains , proliferate in recent years . In this paper , we propose a new problem of correlation mining from graph databases , called Correlated Graph Search ( CGS ) . CGS adopts Pearson 's correlation coefficient as a correlation measure to take into consideration the occurrence distributions of graphs . However , the problem poses significant challenges , since every subgraph of a graph in the database is a candidate but the number of subgraphs is exponential . We derive two necessary conditions which set bounds on the occurrence probability of a candidate in the database . With this result , we design an efficient algorithm that operates on a much smaller projected database and thus we are able to obtain a significantly smaller set of candidates . To further improve the efficiency , we develop three heuristic rules and apply them on the candidate set to further reduce the search space . Our extensive experiments demonstrate the effectiveness of our method on candidate reduction . The results also justify the efficiency of our algorithm in mining correlations from large real and synthetic datasets .

B O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient decision tree construction on streaming data Decision tree construction is a well studied problem in data mining . Recently , there has been much interest in mining streaming data . Domingos and Hulten have presented a one-pass algorithm for decision tree construction . Their work uses Hoeffding inequality to achieve a probabilistic bound on the accuracy of the tree constructed . In this paper , we revisit this problem . We make the following two contributions : 1 ) We present a numerical interval pruning ( NIP ) approach for efficiently processing numerical attributes . Our results show an average of 39 % reduction in execution times . 2 ) We exploit the properties of the gain function entropy ( and gini ) to reduce the sample size required for obtaining a given bound on the accuracy . Our experimental results show a 37 % reduction in the number of data instances required .

O B I O O B I B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The complexity of mining maximal frequent itemsets and maximal frequent patterns Mining maximal frequent itemsets is one of the most fundamental problems in data mining . In this paper we study the complexity-theoretic aspects of maximal frequent itemset mining , from the perspective of counting the number of solutions . We present the first formal proof that the problem of counting the number of distinct maximal frequent itemsets in a database of transactions , given an arbitrary support threshold , is #P - complete , thereby providing strong theoretical evidence that the problem of mining maximal frequent itemsets is NP-hard . This result is of particular interest since the associated decision problem of checking the existence of a maximal frequent itemset is in P. We also extend our complexity analysis to other similar data mining problems dealing with complex data structures , such as sequences , trees , and graphs , which have attracted intensive research interests in recent years . Normally , in these problems a partial order among frequent patterns can be defined in such a way as to preserve the downward closure property , with maximal frequent patterns being those without any successor with respect to this partial order . We investigate several variants of these mining problems in which the patterns of interest are subsequences , subtrees , or subgraphs , and show that the associated problems of counting the number of maximal frequent patterns are all either #P - complete or #P - hard .

O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

complexity
data mining
maximal frequent itemset

maximal frequent pattern
nonnumerical algorithms and problems


Incremental context mining for adaptive document classification Automatic document classification ( DC ) is essential for the management of information and knowledge . This paper explores two practical issues in DC : ( 1 ) each document has its context of discussion , and ( 2 ) both the content and vocabulary of the document database is intrinsically evolving . The issues call for adaptive document classification ( ADC ) that adapts a DC system to the evolving contextual requirement of each document category , so that input documents may be classified based on their contexts of discussion . We present an incremental context mining technique to tackle the challenges of ADC . Theoretical analyses and empirical results show that , given a text hierarchy , the mining technique is efficient in incrementally maintaining the evolving contextual requirement of each category . Based on the contextual requirements mined by the system , higher-precision DC may be achieved with better efficiency .

O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Querying multiple sets of discovered rules Rule mining is an important data mining task that has been applied to numerous real-world applications . Often a rule mining system generates a large number of rules and only a small subset of them is really useful in applications . Although there exist some systems allowing the user to query the discovered rules , they are less suitable for complex ad hoc querying of multiple data mining rulebases to retrieve interesting rules . In this paper , we propose a new powerful rule query language Rule-QL for querying multiple rulebases that is modeled after SQL and has rigorous theoretical foundations of a rule-based calculus . In particular , we first propose a rule-based calculus RC based on the first-order logic , and then present the language Rule-QL that is at least as expressive as the safe fragment of RC . We also propose a number of efficient query evaluation techniques for Rule-QL and test them experimentally on some representative queries to demonstrate the feasibility of Rule-QL .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O


Heterogeneous source consensus learning via decision propagation and negotiation Nowadays , enormous amounts of data are continuously generated not only in massive scale , but also from different , sometimes conflicting , views . Therefore , it is important to consolidate different concepts for intelligent decision making . For example , to predict the research areas of some people , the best results are usually achieved by combining and consolidating predictions obtained from the publication network , co-authorship network and the textual content of their publications . Multiple supervised and unsupervised hypotheses can be drawn from these information sources , and negotiating their differences and consolidating decisions usually yields a much more accurate model due to the diversity and heterogeneity of these models . In this paper , we address the problem of `` consensus learning '' among competing hypotheses , which either rely on outside knowledge ( supervised learning ) or internal structure ( unsupervised clustering ) . We argue that consensus learning is an NP-hard problem and thus propose to solve it by an efficient heuristic method . We construct a belief graph to first propagate predictions from supervised models to the unsupervised , and then negotiate and reach consensus among them . Their final decision is further consolidated by calculating each model 's weight based on its degree of consistency with other models . Experiments are conducted on 20 Newsgroups data , Cora research papers , DBLP author-conference network , and Yahoo ! Movies datasets , and the results show that the proposed method improves the classification accuracy and the clustering quality measure ( NMI ) over the best base model by up to 10 % . Furthermore , it runs in time proportional to the number of instances , which is very efficient for large scale data sets .

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Large-scale matrix factorization with distributed stochastic gradient descent We provide a novel algorithm to approximately factor large matrices with millions of rows , millions of columns , and billions of nonzero elements . Our approach rests on stochastic gradient descent ( SGD ) , an iterative stochastic optimization algorithm . We first develop a novel `` stratified '' SGD variant ( SSGD ) that applies to general loss-minimization problems in which the loss function can be expressed as a weighted sum of `` stratum losses . '' We establish sufficient conditions for convergence of SSGD using results from stochastic approximation theory and regenerative process theory . We then specialize SSGD to obtain a new matrix-factorization algorithm , called DSGD , that can be fully distributed and run on web-scale datasets using , e.g. , MapReduce . DSGD can handle a wide variety of matrix factorizations . We describe the practical techniques used to optimize performance in our DSGD implementation . Experiments suggest that DSGD converges significantly faster and has better scalability properties than alternative algorithms .

O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

distributed matrix factorization
recommendation system


iLink : search and routing in social networks The growth of Web 2.0 and fundamental theoretical breakthroughs have led to an avalanche of interest in social networks . This paper focuses on the problem of modeling how social networks accomplish tasks through peer production style collaboration . We propose a general interaction model for the underlying social networks and then a specific model ( iLink for social search and message routing . A key contribution here is the development of a general learning framework for making such online peer production systems work at scale . The iLink model has been used to develop a system for FAQ generation in a social network ( FAQtory ) , and experience with its application in the context of a full-scale learning-driven workflow application ( CALO ) is reported . We also discuss methods of adapting iLink technology for use in military knowledge sharing portals and other message routing systems . Finally , the paper shows the connection of iLink to SQM , a theoretical model for social search that is a generalization of Markov Decision Processes and the popular Pagerank model .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O

learning
message routing
peer production
social search

expert identification
smart rss
social faq generation


Get another label ? improving data quality and data mining using multiple , noisy labelers This paper addresses the repeated acquisition of labels for data items when the labeling is imperfect . We examine the improvement ( or lack thereof ) in data quality via repeated labeling , and focus especially on the improvement of training labels for supervised induction . With the outsourcing of small tasks becoming easier , for example via Rent-A-Coder or Amazon 's Mechanical Turk , it often is possible to obtain less-than-expert labeling at low cost . With low-cost labeling , preparing the unlabeled part of the data can become considerably more expensive than labeling . We present repeated-labeling strategies of increasing complexity , and show several main results . ( i ) Repeated-labeling can improve label quality and model quality , but not always . ( ii ) When labels are noisy , repeated labeling can be preferable to single labeling even in the traditional setting where labels are not particularly cheap . ( iii ) As soon as the cost of processing the unlabeled data is not free , even the simple strategy of labeling everything multiple times can give considerable advantage . ( iv ) Repeatedly labeling a carefully chosen set of points is generally preferable , and we present a robust technique that combines different notions of uncertainty to select data points for which quality should be improved . The bottom line : the results show clearly that when labeling is not perfect , selective acquisition of multiple labels is a strategy that data miners should have in their repertoire ; for certain label-quality\/cost regimes , the benefit is substantial .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Relational learning via latent social dimensions Social media such as blogs , Facebook , Flickr , etc. , presents data in a network format rather than classical IID distribution . To address the interdependency among data instances , relational learning has been proposed , and collective inference based on network connectivity is adopted for prediction . However , connections in social media are often multi-dimensional . An actor can connect to another actor for different reasons , e.g. , alumni , colleagues , living in the same city , sharing similar interests , etc. . Collective inference normally does not differentiate these connections . In this work , we propose to extract latent social dimensions based on network information , and then utilize them as features for discriminative learning . These social dimensions describe diverse affiliations of actors hidden in the network , and the discriminative learning can automatically determine which affiliations are better aligned with the class labels . Such a scheme is preferred when multiple diverse relations are associated with the same network . We conduct extensive experiments on social media data ( one from a real-world blog site and the other from a popular content sharing site ) . Our model outperforms representative relational learning methods based on collective inference , especially when few labeled data are available . The sensitivity of this model and its connection to existing methods are also examined .

B I O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

relational learning
social dimensions
social media


Efficient methods for topic model inference on streaming document collections Topic models provide a powerful tool for analyzing large text collections by representing high dimensional data in a low dimensional subspace . Fitting a topic model given a set of training documents requires approximate inference techniques that are computationally expensive . With today 's large-scale , constantly expanding document collections , it is useful to be able to infer topic distributions for new documents without retraining the model . In this paper , we empirically evaluate the performance of several methods for topic inference in previously unseen documents , including methods based on Gibbs sampling , variational inference , and a new method inspired by text classification . The classification-based inference method produces results similar to iterative inference methods , but requires only a single matrix multiplication . In addition to these inference methods , we present SparseLDA , an algorithm and data structure for evaluating Gibbs sampling distributions . Empirical results indicate that SparseLDA can be approximately 20 times faster than traditional LDA and provide twice the speedup of previously published fast sampling methods , while also using substantially less memory .

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Data mining criteria for tree-based regression and classification This paper is concerned with the construction of regression and classification trees that are more adapted to data mining applications than conventional trees . To this end , we propose new splitting criteria for growing trees . Conventional splitting criteria attempt to perform well on both sides of a split by attempting a compromise in the quality of fit between the left and the right side . By contrast , we adopt a data mining point of view by proposing criteria that search for interesting subsets of the data , as opposed to modeling all of the data equally well . The new criteria do not split based on a compromise between the left and the right bucket ; they effectively pick the more interesting bucket and ignore the other . As expected , the result is often a simpler characterization of interesting subsets of the data . Less expected is that the new criteria often yield whole trees that provide more interpretable data descriptions . Surprisingly , it is a `` flaw '' that works to their advantage : The new criteria have an increased tendency to accept splits near the boundaries of the predictor ranges . This so-called `` end-cut problem '' leads to the repeated peeling of small layers of data and results in very unbalanced but highly expressive and interpretable trees .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

boston housing data
cart
pima indians diabetes data


Connecting the dots between news articles The process of extracting useful knowledge from large datasets has become one of the most pressing problems in today 's society . The problem spans entire sectors , from scientists to intelligence analysts and web users , all of whom are constantly struggling to keep up with the larger and larger amounts of content published every day . With this much data , it is often easy to miss the big picture . In this paper , we investigate methods for automatically connecting the dots -- providing a structured , easy way to navigate within a new topic and discover hidden connections . We focus on the news domain : given two news articles , our system automatically finds a coherent chain linking them together . For example , it can recover the chain of events starting with the decline of home prices ( January 2007 ) , and ending with the ongoing health-care debate . We formalize the characteristics of a good chain and provide an efficient algorithm ( with theoretical guarantees ) to connect two fixed endpoints . We incorporate user feedback into our framework , allowing the stories to be refined and personalized . Finally , we evaluate our algorithm over real news data . Our user studies demonstrate the algorithm 's effectiveness in helping users understanding the news .

O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O


Neighbor query friendly compression of social networks Compressing social networks can substantially facilitate mining and advanced analysis of large social networks . Preferably , social networks should be compressed in a way that they still can be queried efficiently without decompression . Arguably , neighbor queries , which search for all neighbors of a query vertex , are the most essential operations on social networks . Can we compress social networks effectively in a neighbor query friendly manner , that is , neighbor queries still can be answered in sublinear time using the compression ? In this paper , we develop an effective social network compression approach achieved by a novel Eulerian data structure using multi-position linearizations of directed graphs . Our method comes with a nontrivial theoretical bound on the compression rate . To the best of our knowledge , our approach is the first that can answer both out-neighbor and in-neighbor queries in sublinear time . An extensive empirical study on more than a dozen benchmark real data sets verifies our design .

O O O B O B I O B I O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Weighted Association Rule Mining using weighted support and significance framework We address the issues of discovering significant binary relationships in transaction datasets in a weighted setting . Traditional model of association rule mining is adapted to handle weighted association rule mining problems where each item is allowed to have a weight . The goal is to steer the mining focus to those significant relationships involving items with significant weights rather than being flooded in the combinatornal explosion of insignificant relationships . We identify the challenge of using weights in the iterative process of generating large itemsets . The problem of invalidation of the `` downward closure property '' in the weighted setting is solved by using an improved model of weighted support measurements and exploiting a `` weighted downward closure property '' . A new algorithm called WARM ( Weighted Association Rule Mining ) is developed based on the improved model . The algorithm is both scalable and efficient in discovering significant relationships in weighted settings as illustrated by experiments performed on simulated datasets .

B I I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I I I O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

weighted association rule mining
weighted downward closure property
weighted support


Detecting research topics via the correlation between graphs and texts In this paper we address the problem of detecting topics in large-scale linked document collections . Recently , topic detection has become a very active area of research due to its utility for information navigation , trend analysis , and high-level description of data . We present a unique approach that uses the correlation between the distribution of a term that represents a topic and the link distribution in the citation graph where the nodes are limited to the documents containing the term . This tight coupling between term and graph analysis is distinguished from other approaches such as those that focus on language models . We develop a topic score measure for each term , using the likelihood ratio of binary hypotheses based on a probabilistic description of graph connectivity . Our approach is based on the intuition that if a term is relevant to a topic , the documents containing the term have denser connectivity than a random selection of documents . We extend our algorithm to detect a topic represented by a set of terms , using the intuition that if the co-occurrence of terms represents a new topic , the citation pattern should exhibit the synergistic effect . We test our algorithm on two electronic research literature collections , arXiv and Citeseer . Our evaluation shows that the approach is effective and reveals some novel aspects of topic detection .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O

citation graphs
content analysis and indexing
correlation of text and links
graph mining
probabilistic measure


Fast vertical mining using diffsets A number of vertical mining algorithms have been proposed recently for association mining , which have shown to be very effective and usually outperform horizontal approaches . The main advantage of the vertical format is support for fast frequency counting via intersection operations on transaction ids ( tids ) and automatic pruning of irrelevant data . The main problem with these approaches is when intermediate results of vertical tid lists become too large for memory , thus affecting the algorithm scalability . In this paper we present a novel vertical data representation called Diffset , that only keeps track of differences in the tids of a candidate pattern from its generating frequent patterns . We show that diffsets drastically cut down the size of memory required to store intermediate results . We show how diffsets , when incorporated into previous vertical mining methods , increase the performance significantly .

O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O

association rule mining
database applications
frequent itemsets


Carpenter : finding closed patterns in long biological datasets The growth of bioinformatics has resulted in datasets with new characteristics . These datasets typically contain a large number of columns and a small number of rows . For example , many gene expression datasets may contain 10,000-100 ,000 columns but only 100-1000 rows . Such datasets pose a great challenge for existing ( closed ) frequent pattern discovery algorithms , since they have an exponential dependence on the average row length . In this paper , we describe a new algorithm called CARPENTER that is specially designed to handle datasets having a large number of attributes and relatively small number of rows . Several experiments on real bioinformatics datasets show that CARPENTER is orders of magnitude better than previous closed pattern mining algorithms like CLOSET and CHARM .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O


Predicting rare classes : can boosting make any weak learner strong ? Boosting is a strong ensemble-based learning algorithm with the promise of iteratively improving the classification accuracy using any base learner , as long as it satisfies the condition of yielding weighted accuracy ) 0.5 . In this paper , we analyze boosting with respect to this basic condition on the base learner , to see if boosting ensures prediction of rarely occurring events with high recall and precision . First we show that a base learner can satisfy the required condition even for poor recall or precision levels , especially for very rare classes . Furthermore , we show that the intelligent weight updating mechanism in boosting , even in its strong cost-sensitive form , does not prevent cases where the base learner always achieves high precision but poor recall or high recall but poor precision , when mapped to the original distribution . In either of these cases , we show that the voting mechanism of boosting falls to achieve good overall recall and precision for the ensemble . In effect , our analysis indicates that one can not be blind to the base learner performance , and just rely on the boosting mechanism to take care of its weakness . We validate our arguments empirically on variety of real and synthetic rare class problems . In particular , using AdaCost as the boosting algorithm , and variations of PNrule and RIPPER as the base learners , we show that if algorithm A achieves better recall-precision balance than algorithm B , then using A as the base learner in AdaCost yields significantly better performance than using B as the base learner .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A data mining approach to modeling relationships among categories in image collection This paper proposes a data mining approach to modeling relationships among categories in image collection . In our approach , with image feature grouping , a visual dictionary is created for color , texture , and shape feature attributes respectively . Labeling each training image with the keywords in the visual dictionary , a classification tree is built . Based on the statistical properties of the feature space we define a structure , called α-Semantics Graph , to discover the hidden semantic relationships among the semantic categories embodied in the image collection . With the α-Semantics Graph , each semantic category is modeled as a unique fuzzy set to explicitly address the semantic uncertainty and semantic overlap among the categories in the feature space . The model is utilized in the semantics-intensive image retrieval application . An algorithm using the classification accuracy measures is developed to combine the built classification tree with the fuzzy set modeling method to deliver semantically relevant image retrieval for a given query image . The experimental evaluations have demonstrated that the proposed approach models the semantic relationships effectively and the image retrieval prototype system utilizing the derived model is promising both in effectiveness and efficiency .

O O O O O O B O O O B I O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O

image collection
relationships
semantic category


Fragments of order High-dimensional collections of 0 -- 1 data occur in many applications . The attributes in such data sets are typically considered to be unordered . However , in many cases there is a natural total or partial order ≺ underlying the variables of the data set . Examples of variables for which such orders exist include terms in documents , courses in enrollment data , and paleontological sites in fossil data collections . The observations in such applications are flat , unordered sets ; however , the data sets respect the underlying ordering of the variables . By this we mean that if A ≺ B ≺ C are three variables respecting the underlying ordering ≺ , and both of variables A and C appear in an observation , then , up to noise levels , variable B also appears in this observation . Similarly , if A1 ≺ A2 ≺ ... ≺ Al-1 ≺ Ai is a longer sequence of variables , we do not expect to see many observations for which there are indices i ( j ( k such that Ai and Ak occur in the observation but Aj does not . In this paper we study the problem of discovering fragments of orders of variables implicit in collections of unordered observations . We define measures that capture how well a given order agrees with the observed data . We describe a simple and efficient algorithm for finding all the fragments that satisfy certain conditions . We also discuss the sometimes necessary postprocessing for selecting only the best fragments of order . Also , we relate our method with a sequencing approach that uses a spectral algorithm , and with the consecutive ones problem . We present experimental results on some real data sets ( author lists of database papers , exam results data , and paleontological data ) .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

consecutive ones property
discovering hidden orderings
nonnumerical algorithms and problems
novel data mining algorithms
spectral analysis of data


OLAP on search logs : an infrastructure supporting data-driven applications in search engines Search logs , which contain rich and up-to-date information about users ' needs and preferences , have become a critical data source for search engines . Recently , more and more data-driven applications are being developed in search engines based on search logs , such as query suggestion , keyword bidding , and dissatisfactory query analysis . In this paper , by observing that many data-driven applications in search engines highly rely on online mining of search logs , we develop an OLAP system on search logs which serves as an infrastructure supporting various data-driven applications . An empirical study using real data of over two billion query sessions demonstrates the usefulness and feasibility of our design .

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A refinement approach to handling model misfit in text categorization Text categorization or classification is the automated assigning of text documents to pre-defined classes based on their contents . This problem has been studied in information retrieval , machine learning and data mining . So far , many effective techniques have been proposed . However , most techniques are based on some underlying models and\/or assumptions . When the data fits the model well , the classification accuracy will be high . However , when the data does not fit the model well , the classification accuracy can be very low . In this paper , we propose a refinement approach to dealing with this problem of model misfit . We show that we do not need to change the classification technique itself ( or its underlying model ) to make it more flexible . Instead , we propose to use successive refinements of classification on the training data to correct the model misfit . We apply the proposed technique to improve the classification performance of two simple and efficient text classifiers , the Rocchio classifier and the naïve Bayesian classifier . These techniques are suitable for very large text collections because they allow the data to reside on disk and need only one scan of the data to build a text classifier . Extensive experiments on two benchmark document corpora show that the proposed technique is able to improve text categorization accuracy of the two techniques dramatically . In particular , our refined model is able to improve the naïve Bayesian or Rocchio classifier 's prediction performance by 45 % on average .

O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

naïve bayesian classifier
text categorization

probabilistic algorithms
rocchio algorithm


Mining closed relational graphs with connectivity constraints Relational graphs are widely used in modeling large scale networks such as biological networks and social networks . In this kind of graph , connectivity becomes critical in identifying highly associated groups and clusters . In this paper , we investigate the issues of mining closed frequent graphs with connectivity constraints in massive relational graphs where each graph has around 10K nodes and 1M edges . We adopt the concept of edge connectivity and apply the results from graph theory , to speed up the mining process . Two approaches are developed to handle different mining requests : CloseCut , a pattern-growth approach , and splat , a pattern-reduction approach . We have applied these methods in biological datasets and found the discovered patterns interesting .

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Combining linguistic and statistical analysis to extract relations from web documents The World Wide Web provides a nearly endless source of knowledge , which is mostly given in natural language . A first step towards exploiting this data automatically could be to extract pairs of a given semantic relation from text documents - for example all pairs of a person and her birthdate . One strategy for this task is to find text patterns that express the semantic relation , to generalize these patterns , and to apply them to a corpus to find new pairs . In this paper , we show that this approach profits significantly when deep linguistic structures are used instead of surface text patterns . We demonstrate how linguistic structures can be represented for machine learning , and we provide a theoretical analysis of the pattern matching approach . We show the benefits of our approach by extensive experiments with our prototype system LEILA .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O


An energy-efficient mobile recommender system The increasing availability of large-scale location traces creates unprecedent opportunities to change the paradigm for knowledge discovery in transportation systems . A particularly promising area is to extract energy-efficient transportation patterns ( green knowledge ) , which can be used as guidance for reducing inefficiencies in energy consumption of transportation sectors . However , extracting green knowledge from location traces is not a trivial task . Conventional data analysis tools are usually not customized for handling the massive quantity , complex , dynamic , and distributed nature of location traces . To that end , in this paper , we provide a focused study of extracting energy-efficient transportation patterns from location traces . Specifically , we have the initial focus on a sequence of mobile recommendations . As a case study , we develop a mobile recommender system which has the ability in recommending a sequence of pick-up points for taxi drivers or a sequence of potential parking positions . The goal of this mobile recommendation system is to maximize the probability of business success . Along this line , we provide a Potential Travel Distance ( PTD ) function for evaluating each candidate sequence . This PTD function possesses a monotone property which can be used to effectively prune the search space . Based on this PTD function , we develop two algorithms , LCP and SkyRoute , for finding the recommended routes . Finally , experimental results show that the proposed system can provide effective mobile sequential recommendation and the knowledge extracted from location traces can be used for coaching drivers and leading to the efficient use of energy .

O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O