BGP-lens : patterns and anomalies in internet routing updates The Border Gateway Protocol ( BGP ) is one of the fundamental computer communication protocols . Monitoring and mining BGP update messages can directly reveal the health and stability of Internet routing . Here we make two contributions : firstly we find patterns in BGP updates , like self-similarity , power-law and lognormal marginals ; secondly using these patterns , we find anomalies . Specifically , we develop BGP-lens , an automated BGP updates analysis tool , that has three desirable properties : ( a ) It is effective , able to identify phenomena that would otherwise go unnoticed , such as a peculiar ` clothesline ' behavior or prolonged ` spikes ' that last as long as 8 hours ; ( b ) It is scalable , using algorithms are all linear on the number of time-ticks ; and ( c ) It is admin-friendly , giving useful leads for phenomenon of interest . We showcase the capabilities of BGP-lens by identifying surprising phenomena verified by syadmins , over a massive trace of BGP updates spanning 2 years , from the publicly available site datapository.net .

O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Online novelty detection on temporal sequences In this paper , we present a new framework for online novelty detection on temporal sequences . This framework include a mechanism for associating each detection result with a confidence value . Based on this framework , we develop a concrete online detection algorithm , by modeling the temporal sequence using an online support vector regression algorithm . Experiments on both synthetic and real world data are performed to demonstrate the promising performance of our proposed detection algorithm .

O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O

novelty detection
support vector regression

anomaly detection
design methodology
online algorithm


Mining complex models from arbitrarily large databases in constant time In this paper we propose a scaling-up method that is applicable to essentially any induction algorithm based on discrete search . The result of applying the method to an algorithm is that its running time becomes independent of the size of the database , while the decisions made are essentially identical to those that would be made given infinite data . The method works within pre-specified memory limits and , as long as the data is iid , only requires accessing it sequentially . It gives anytime results , and can be used to produce batch , stream , time-changing and active-learning versions of an algorithm . We apply the method to learning Bayesian networks , developing an algorithm that is faster than previous ones by orders of magnitude , while achieving essentially the same predictive performance . We observe these gains on a series of large databases `` generated from benchmark networks , on the KDD Cup 2000 e-commerce data , and on a Web log containing 100 million requests .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

hoeffding bounds
scalable learning algorithms
subsampling


The distributed boosting algorithm In this paper , we propose a general framework for distributed boosting intended for efficient integrating specialized classifiers learned over very large and distributed homogeneous databases that can not be merged at a single location . Our distributed boosting algorithm can also be used as a parallel classification technique , where a massive database that can not fit into main computer memory is partitioned into disjoint subsets for a more efficient analysis . In the proposed method , at each boosting round the classifiers are first learned from disjoint datasets and then exchanged amongst the sites . Finally the classifiers are combined into a weighted voting ensemble on each disjoint data set . The ensemble that is applied to an unseen test set represents an ensemble of ensembles built on all distributed sites . In experiments performed on four large data sets the proposed distributed boosting method achieved classification accuracy comparable or even slightly better than the standard boosting algorithm while requiring less memory and less computational time . In addition , the communication overhead of the distributed boosting algorithm is very small making it a viable alternative to the standard boosting for large-scale databases .

O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O

classifier ensembles
distributed learning


On the need for time series data mining benchmarks : a survey and empirical demonstration In the last decade there has been an explosion of interest in mining time series data . Literally hundreds of papers have introduced new algorithms to index , classify , cluster and segment time series . In this work we make the following claim . Much of this work has very little utility because the contribution made ( speed in the case of indexing , accuracy in the case of classification and clustering , model accuracy in the case of segmentation ) offer an amount of `` improvement '' that would have been completely dwarfed by the variance that would have been observed by testing on many real world datasets , or the variance that would have been observed by changing minor ( unstated ) implementation details . To illustrate our point , we have undertaken the most exhaustive set of time series experiments ever attempted , re-implementing the contribution of more than two dozen papers , and testing them on 50 real world , highly diverse datasets . Our empirical results strongly support our assertion , and suggest the need for a set of time series benchmarks and more careful empirical evaluation in the data mining community .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O


Detecting time series motifs under uniform scaling Time series motifs are approximately repeated patterns foundwithin the data . Such motifs have utility for many data mining algorithms , including rule-discovery , novelty-detection , summarization and clustering . Since the formalization of the problem and the introduction of efficient linear time algorithms , motif discovery has been successfully applied tomany domains , including medicine , motion capture , robotics and meteorology . In this work we show that most previous applications of time series motifs have been severely limited by the definition 's brittleness to even slight changes of uniform scaling , the speed at which the patterns develop . We introduce a new algorithm that allows discovery of time series motifs with invariance to uniform scaling , and show that it produces objectively superior results in several important domains . Apart from being more general than all other motifdiscovery algorithms , a further contribution of our work isthat it is simpler than previous approaches , in particular we have drastically reduced the number of parameters that need to be specified .

O B I B O B I B I B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Modeling and predicting user behavior in sponsored search Implicit user feedback , including click-through and subsequent browsing behavior , is crucial for evaluating and improving the quality of results returned by search engines . Several recent studies ( 1 , 2 , 3 , 13 , 25 ) have used post-result browsing behavior including the sites visited , the number of clicks , and the dwell time on site in order to improve the ranking of search results . In this paper , we first study user behavior on sponsored search results ( i.e. , the advertisements displayed by search engines next to the organic results ) , and compare this behavior to that of organic results . Second , to exploit post-result user behavior for better ranking of sponsored results , we focus on identifying patterns in user behavior and predict expected on-site actions in future instances . In particular , we show how post-result behavior depends on various properties of the queries , advertisement , sites , and users , and build a classifier using properties such as these to predict certain aspects of the user behavior . Additionally , we develop a generative model to mimic trends in observed user activity using a mixture of pareto distributions . We conduct experiments based on billions of real navigation trails collected by a major search engine 's browser toolbar .

O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning to rank networked entities Several algorithms have been proposed to learn to rank entities modeled as feature vectors , based on relevance feedback . However , these algorithms do not model network connections or relations between entities . Meanwhile , Pagerank and variants find the stationary distribution of a reasonable but arbitrary Markov walk over a network , but do not learn from relevance feedback . We present a framework for ranking networked entities based on Markov walks with parameterized conductance values associated with the network edges . We propose two flavors of conductance learning problems in our framework . In the first setting , relevance feedback comparing node-pairs hints that the user has one or more hidden preferred communities with large edge conductance , and the algorithm must discover these communities . We present a constrained maximum entropy network flow formulation whose dual can be solved efficiently using a cutting-plane approach and a quasi-Newton optimizer . In the second setting , edges have types , and relevance feedback hints that each edge type has a potentially different conductance , but this is fixed across the whole network . Our algorithm learns the conductances using an approximate Newton method .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Ranking-based clustering of heterogeneous information networks with star network schema A heterogeneous information network is an information network composed of multiple types of objects . Clustering on such a network may lead to better understanding of both hidden structures of the network and the individual role played by every object in each cluster . However , although clustering on homogeneous networks has been studied over decades , clustering on heterogeneous networks has not been addressed until recently . A recent study proposed a new algorithm , RankClus , for clustering on bi-typed heterogeneous networks . However , a real-world network may consist of more than two types , and the interactions among multi-typed objects play a key role at disclosing the rich semantics that a network carries . In this paper , we study clustering of multi-typed heterogeneous networks with a star network schema and propose a novel algorithm , NetClus , that utilizes links across multityped objects to generate high-quality net-clusters . An iterative enhancement method is developed that leads to effective ranking-based clustering in such heterogeneous networks . Our experiments on DBLP data show that NetClus generates more accurate clustering results than the baseline topic model algorithm PLSA and the recently proposed algorithm , RankClus . Further , NetClus generates informative clusters , presenting good ranking and cluster membership information for each attribute object in each net-cluster .

O B O O O O O O O O O B I I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

clustering
heterogeneous information network


Online discovery and maintenance of time series motifs The detection of repeated subsequences , time series motifs , is a problem which has been shown to have great utility for several higher-level data mining algorithms , including classification , clustering , segmentation , forecasting , and rule discovery . In recent years there has been significant research effort spent on efficiently discovering these motifs in static offline databases . However , for many domains , the inherent streaming nature of time series demands online discovery and maintenance of time series motifs . In this paper , we develop the first online motif discovery algorithm which monitors and maintains motifs exactly in real time over the most recent history of a stream . Our algorithm has a worst-case update time which is linear to the window size and is extendible to maintain more complex pattern structures . In contrast , the current offline algorithms either need significant update time or require very costly pre-processing steps which online algorithms simply can not afford . Our core ideas allow useful extensions of our algorithm to deal with arbitrary data rates and discovering multidimensional motifs . We demonstrate the utility of our algorithms with a variety of case studies in the domains of robotics , acoustic monitoring and online compression .

O O O O O B I B O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O B I B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O


Utility-based anonymization using local recoding Privacy becomes a more and more serious concern in applications involving microdata . Recently , efficient anonymization has attracted much research work . Most of the previous methods use global recoding , which maps the domains of the quasi-identifier attributes to generalized or changed values . However , global recoding may not always achieve effective anonymization in terms of discernability and query answering accuracy using the anonymized data . Moreover , anonymized data is often for analysis . As well accepted in many analytical applications , different attributes in a data set may have different utility in the analysis . The utility of attributes has not been considered in the previous methods . In this paper , we study the problem of utility-based anonymization . First , we propose a simple framework to specify utility of attributes . The framework covers both numeric and categorical data . Second , we develop two simple yet efficient heuristic local recoding methods for utility-based anonymization . Our extensive performance study using both real data sets and synthetic data sets shows that our methods outperform the state-of-the-art multidimensional global recoding methods in both discernability and query answering accuracy . Furthermore , our utility-based method can boost the quality of analysis using the anonymized data .

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Nantonac collaborative filtering : recommendation based on order responses A recommender system suggests the items expected to be preferred by the users . Recommender systems use collaborative filtering to recommend items by summarizing the preferences of people who have tendencies similar to the user preference . Traditionally , the degree of preference is represented by a scale , for example , one that ranges from one to five . This type of measuring technique is called the semantic differential ( SD ) method . Web adopted the ranking method , however , rather than the SD method , since the SD method is intrinsically not suited for representing individual preferences . In the ranking method , the preferences are represented by orders , which are sorted item sequences according to the users ' preferences . We here propose some methods to recommed items based on these order responses , and carry out the comparison experiments of these methods .

O B I O O O O B O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O

collaborative filtering
order
recommender system


Enhanced max margin learning on multimodal data mining in a multimedia database The problem of multimodal data mining in a multimedia database can be addressed as a structured prediction problem where we learn the mapping from an input to the structured and interdependent output variables . In this paper , built upon the existing literature on the max margin based learning , we develop a new max margin learning approach called Enhanced Max Margin Learning ( EMML ) framework . In addition , we apply EMML framework to developing an effective and efficient solution to the multimodal data mining problem in a multimedia database . The main contributions include : ( 1 ) we have developed a new max margin learning approach - the enhanced max margin learning framework that is much more efficient in learning with a much faster convergence rate , which is verified in empirical evaluations ; ( 2 ) we have applied this EMML approach to developing an effective and efficient solution to the multimodal data mining problem that is highly scalable in the sense that the query response time is independent of the database scale , allowing facilitating a multimodal data mining querying to a very large scale multimedia database , and excelling many existing multimodal data mining methods in the literature that do not scale up at all ; this advantage is also supported through the complexity analysis as well as empirical evaluations against a state-of-the-art multimodal data mining method from the literature . While EMML is a general framework , for the evaluation purpose , we apply it to the Berkeley Drosophila embryo image database , and report the performance comparison with a state-of-the-art multimodal data mining method .

O B I O O B I I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O


Using the fractal dimension to cluster datasets


Toward autonomic grids : analyzing the job flow with affinity streaming The Affinity Propagation ( AP ) clustering algorithm proposed by Frey and Dueck ( 2007 ) provides an understandable , nearly optimal summary of a dataset , albeit with quadratic computational complexity . This paper , motivated by Autonomic Computing , extends AP to the data streaming framework . Firstly a hierarchical strategy is used to reduce the complexity to O ( N1 + ε ) ; the distortion loss incurred is analyzed in relation with the dimension of the data items . Secondly , a coupling with a change detection test is used to cope with non-stationary data distribution , and rebuild the model as needed . The presented approach StrAP is applied to the stream of jobs submitted to the EGEE Grid , providing an understandable description of the job flow and enabling the system administrator to spot online some sources of failures .

O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

affinity propagation
autonomic computing


Sampling from large graphs Given a huge real graph , how can we derive a representative sample ? There are many known algorithms to compute interesting measures ( shortest paths , centrality , betweenness , etc. ) , but several of them become impractical for large graphs . Thus graph sampling is essential . The natural questions to ask are ( a ) which sampling method to use , ( b ) how small can the sample size be , and ( c ) how to scale up the measurements of the sample ( e.g. , the diameter ) , to get estimates for the large graph . The deeper , underlying question is subtle : how do we measure success ? . We answer the above questions , and test our answers by thorough experiments on several , diverse datasets , spanning thousands nodes and edges . We consider several sampling methods , propose novel methods to check the goodness of sampling , and develop a set of scaling laws that describe relations between the properties of the original and the sample . In addition to the theoretical contributions , the practical conclusions from our work are : Sampling strategies based on edge selection do not perform well ; simple uniform random node selection performs surprisingly well . Overall , best performing methods are the ones based on random-walks and `` forest fire '' ; they match very accurately both static as well as evolutionary graph patterns , with sample sizes down to about 15 % of the original graph .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Cross-sell : a fast promotion-tunable customer-item recommendation method based on conditionally independent probabilities

collaborative filtering
cross-sell
electronic commerce
imputation


A quickstart in frequent structure mining can make a difference Given a database , structure mining algorithms search for substructures that satisfy constraints such as minimum frequency , minimum confidence , minimum interest and maximum frequency . Examples of substructures include graphs , trees and paths . For these substructures many mining algorithms have been proposed . In order to make graph mining more efficient , we investigate the use of the `` quickstart principle '' , which is based on the fact that these classes of structures are contained in each other , thus allowing for the development of structure mining algorithms that split the search into steps of increasing complexity . We introduce the GrAph\/Sequence\/Tree extractiON ( Gaston ) algorithm that implements this idea by searching first for frequent paths , then frequent free trees and finally cyclic graphs . We investigate two alternatives for computing the frequency of structures and present experimental results to relate these alternatives .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O


Mining knowledge-sharing sites for viral marketing Viral marketing takes advantage of networks of influence among customers to inexpensively achieve large changes in behavior . Our research seeks to put it on a firmer footing by mining these networks from data , building probabilistic models of them , and using these models to choose the best viral marketing plan . Knowledge-sharing sites , where customers review products and advise each other , are a fertile source for this type of data mining . In this paper we extend our previous techniques , achieving a large reduction in computational cost , and apply them to data from a knowledge-sharing site . We optimize the amount of marketing funds spent on each customer , rather than just making a binary decision on whether to market to him . We take into account the fact that knowledge of the network is partial , and that gathering that knowledge can itself have a cost . Our results show the robustness and utility of our approach .

O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

direct marketing
knowledge sharing
linear models
social networks


GPLAG : detection of software plagiarism by program dependence graph analysis Along with the blossom of open source projects comes the convenience for software plagiarism . A company , if less self-disciplined , may be tempted to plagiarize some open source projects for its own products . Although current plagiarism detection tools appear sufficient for academic use , they are nevertheless short for fighting against serious plagiarists . For example , disguises like statement reordering and code insertion can effectively confuse these tools . In this paper , we develop a new plagiarism detection tool , called GPLAG , which detects plagiarism by mining program dependence graphs ( PDGs ) . A PDG is a graphic representation of the data and control dependencies within a procedure . Because PDGs are nearly invariant during plagiarism , GPLAG is more effective than state-of-the-art tools for plagiarism detection . In order to make GPLAG scalable to large programs , a statistical lossy filter is proposed to prune the plagiarism search space . Experiment study shows that GPLAG is both effective and efficient : It detects plagiarism that easily slips over existing tools , and it usually takes a few seconds to find ( simulated ) plagiarism in programs having thousands of lines of code .

O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graph mining
software plagiarism detection


Mining relational data through correlation-based multiple view validation Commercial relational databases currently store vast amounts of real-world data . The data within these relational repositories are represented by multiple relations , which are inter-connected by means of foreign key joins . The mining of such interrelated data poses a major challenge to the data mining community . Unfortunately , traditional data mining algorithms usually only explore one relation , the so-called target relation , thus excluding crucial knowledge embedded in the related so-called background relations . In this paper , we propose a novel approach for classifying relational such domains . This strategy employs multiple views to capture crucial information not only from the target relation , but also from related relations . This information is integrated into the relational mining process . The framework presented here , firstly , explore the relational domain to partition its features space into multiple subsets . Subsequently , these subsets are used to construct multiple uncorrelated views , based on a novel correlation-based view validation method , against the target concept . Finally , the knowledge possessed by multiple views are incorporated into a meta-learning mechanism to augment one another . Based on this framework , a wide range of conventional data mining methods can be applied to mine relational databases . Our experiments on benchmark real-world data sets show that the proposed method achieves promising results both in terms of overall accuracy obtained and run time , when compared with two other relational data mining approaches .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

classification
multi-relational data mining
multi-view learning
relational database


Summarizing itemset patterns using probabilistic models In this paper , we propose a novel probabilistic approach to summarize frequent itemset patterns . Such techniques are useful for summarization , post-processing , and end-user interpretation , particularly for problems where the resulting set of patterns are huge . In our approach items in the dataset are modeled as random variables . We then construct a Markov Random Fields ( MRF ) on these variables based on frequent itemsets and their occurrence statistics . The summarization proceeds in a level-wise iterative fashion . Occurrence statistics of itemsets at the lowest level are used to construct an initial MRF . Statistics of itemsets at the next level can then be inferred from the model . We use those patterns whose occurrence can not be accurately inferred from the model to augment the model in an iterative manner , repeating the procedure until all frequent itemsets can be modeled . The resulting MRF model affords a concise and useful representation of the original collection of itemsets . Extensive empirical study on real datasets show that the new approach can effectively summarize a large number of itemsets and typically significantly outperforms extant approaches .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

itemset pattern summarization
markov random field
probabilistic graphical model


Selecting the right interestingness measure for association patterns Many techniques for association rule mining and feature selection require a suitable metric to capture the dependencies among variables in a data set . For example , metrics such as support , confidence , lift , correlation , and collective strength are often used to determine the interestingness of association patterns . However , many such measures provide conflicting information about the interestingness of a pattern , and the best metric to use for a given application domain is rarely known . In this paper , we present an overview of various measures proposed in the statistics , machine learning and data mining literature . We describe several key properties one should examine in order to select the right measure for a given application domain . A comparative study of these properties is made using twenty one of the existing measures . We show that each measure has different properties which make them useful for some application domains , but not for others . We also present two scenarios in which most of the existing measures agree with each other , namely , support-based pruning and table standardization . Finally , we present an algorithm to select a small set of tables such that an expert can select a desirable measure by looking at just this small set of tables .

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Mining closed episodes with simultaneous events Sequential pattern discovery is a well-studied field in data mining . Episodes are sequential patterns describing events that often occur in the vicinity of each other . Episodes can impose restrictions to the order of the events , which makes them a versatile technique for describing complex patterns in the sequence . Most of the research on episodes deals with special cases such as serial , parallel , and injective episodes , while discovering general episodes is understudied . In this paper we extend the definition of an episode in order to be able to represent cases where events often occur simultaneously . We present an efficient and novel miner for discovering frequent and closed general episodes . Such a task presents unique challenges . Firstly , we can not define closure based on frequency . We solve this by computing a more conservative closure that we use to reduce the search space and discover the closed episodes as a postprocessing step . Secondly , episodes are traditionally presented as directed acyclic graphs . We argue that this representation has drawbacks leading to redundancy in the output . We solve these drawbacks by defining a subset relationship in such a way that allows us to remove the redundant episodes . We demonstrate the efficiency of our algorithm and the need for using closed episodes empirically on synthetic and real-world datasets .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O


Learning nonstationary models of normal network traffic for detecting novel attacks Traditional intrusion detection systems ( IDS ) detect attacks by comparing current behavior to signatures of known attacks . One main drawback is the inability of detecting new attacks which do not have known signatures . In this paper we propose a learning algorithm that constructs models of normal behavior from attack-free network traffic . Behavior that deviates from the learned normal model signals possible novel attacks . Our IDS is unique in two respects . First , it is nonstationary , modeling probabilities based on the time since the last event rather than on average rate . This prevents alarm floods . Second , the IDS learns protocol vocabularies ( at the data link through application layers ) in order to detect unknown attacks that attempt to exploit implementation errors in poorly tested features of the target software . On the 1999 DARPA IDS evaluation data set ( 9 ) , we detect 70 of 180 attacks ( with 100 false alarms ) , about evenly divided between user behavioral anomalies ( IP addresses and ports , as modeled by most other systems ) and protocol anomalies . Because our methods are unconventional there is a significant non-overlap of our IDS with the original DARPA participants , which implies that they could be combined to increase coverage .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

security and protection
unauthorized access


Detection of emerging space-time clusters We propose a new class of spatio-temporal cluster detection methods designed for the rapid detection of emerging space-time clusters . We focus on the motivating application of prospective disease surveillance : detecting space-time clusters of disease cases resulting from an emerging disease outbreak . Automatic , real-time detection of outbreaks can enable rapid epidemiological response , potentially reducing rates of morbidity and mortality . Building on the prior work on spatial and space-time scan statistics , our methods combine time series analysis ( to determine how many cases we expect to observe for a given spatial region in a given time interval ) with new `` emerging cluster '' space-time scan statistics ( to decide whether an observed increase in cases in a region is significant ) , enabling fast and accurate detection of emerging outbreaks . We evaluate these methods on two types of simulated outbreaks : aerosol release of inhalational anthrax ( e.g. from a bioterrorist attack ) and FLOO ( `` Fictional Linear Onset Outbreak '' ) , injected into actual baseline data ( Emergency Department records and over-the-counter drug sales data from Allegheny County ) . We demonstrate that our methods are successful in rapidly detecting both outbreak types while keeping the number of false positives low , and show that our new `` emerging cluster '' scan statistics consistently outperform the standard `` persistent cluster '' scan statistics approach .

O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

biosurveillance
space-time scan statistics


Construct robust rule sets for classification We study the problem of computing classification rule sets from relational databases so that accurate predictions can be made on test data with missing attribute values . Traditional classifiers perform badly when test data are not as complete as the training data because they tailor a training database too much . We introduce the concept of one rule set being more robust than another , that is , able to make more accurate predictions on test data with missing attribute values . We show that the optimal class association rule set is as robust as the complete class association rule set . We then introduce the k-optimal rule set , which provides predictions exactly the same as the optimal class association rule set on test data with up to k missing attribute values . This leads to a hierarchy of k-optimal rule sets in which decreasing size corresponds to decreasing robustness , and they all more robust than a traditional classification rule set . We introduce two methods to find k-optimal rule sets , i.e. an optimal association rule mining approach and a heuristic approximate approach . We show experimentally that a k-optimal rule set generated by the optimal association rule mining approach performs better than that by the heuristic approximate approach and both rule sets perform significantly better than a typical classification rule set ( C4 .5 Rules ) on incomplete test data .

O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O


Web usage mining based on probabilistic latent semantic analysis The primary goal of Web usage mining is the discovery of patterns in the navigational behavior of Web users . Standard approaches , such as clustering of user sessions and discovering association rules or frequent navigational paths , do not generally provide the ability to automatically characterize or quantify the unobservable factors that lead to common navigational patterns . It is , therefore , necessary to develop techniques that can automatically discover hidden semantic relationships among users as well as between users and Web objects . Probabilistic Latent Semantic Analysis ( PLSA ) is particularly useful in this context , since it can uncover latent semantic associations among users and pages based on the co-occurrence patterns of these pages in user sessions . In this paper , we develop a unified framework for the discovery and analysis of Web navigational patterns based on PLSA . We show the flexibility of this framework in characterizing various relationships among users and Web objects . Since these relationships are measured in terms of probabilities , we are able to use probabilistic inference to perform a variety of analysis tasks such as user segmentation , page classification , as well as predictive tasks such as collaborative recommendations . We demonstrate the effectiveness of our approach through experiments performed on real-world data sets .

B I I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fast mining of high dimensional expressive contrast patterns using zero-suppressed binary decision diagrams Patterns of contrast are a very important way of comparing multi-dimensional datasets . Such patterns are able to capture regions of high difference between two classes of data , and are useful for human experts and the construction of classifiers . However , mining such patterns is particularly challenging when the number of dimensions is large . This paper describes a new technique for mining several varieties of contrast pattern , based on the use of Zero-Suppressed Binary Decision Diagrams ( ZBDDs ) , a powerful data structure for manipulating sparse data . We study the mining of both simple contrast patterns , such as emerging patterns , and more novel and complex contrasts , which we call disjunctive emerging patterns . A performance study demonstrates our ZBDD technique is highly scalable , substantially improves on state of the art mining for emerging patterns and can be effective for discovering complex contrasts from datasets with thousands of attributes .

O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

contrast patterns
disjunctive emerging patterns


Apolo

interactive

large

graph

sensemaking

by

combining

machine

learning

and

visualization

We

present

APOLO

a

system

that

uses

a

mixed-initiative

approach

to

help

people

interactively

explore

and

make

sense

of

large

network

datasets

It

combines

visualization

rich

user

interaction

and

machine

learning

to

engage

the

user

in

bottom-up

sensemaking

to

gradually

build

up

an

understanding

over

time

by

starting

small

rather

than

starting

big

and

drilling

down

APOLO

helps

users

find

relevant

information

by

specifying

exemplars

and

then

using

a

machine

learning

method

called

Belief

Propagation

to

infer

which

other

nodes

may

be

of

interest

We

demonstrate

APOLO

's

usage

and

benefits

using

a

Google

Scholar

citation

graph

consisting

of

articles

nodes

and

citations

relationships

A

demo

video

of

APOLO

is

available

at

http:\/\/www.cs.cmu.edu\/~dchau\/apolo\/apolo.mp4

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

belief propagation
large network
sensemaking


Finding similar files in large document repositories Hewlett-Packard has many millions of technical support documents in a variety of collections . As part of content management , such collections are periodically merged and groomed . In the process , it becomes important to identify and weed out support documents that are largely duplicates of newer versions . Doing so improves the quality of the collection , eliminates chaff from search results , and improves customer satisfaction . The technical challenge is that through workflow and human processes , the knowledge of which documents are related is often lost . We required a method that could identify similar documents based on their content alone , without relying on metadata , which may be corrupt or missing . We present an approach for finding similar files that scales up to large document repositories . It is based on chunking the byte stream to find unique signatures that may be shared in multiple files . An analysis of the file-chunk graph yields clusters of related files . An optional bipartite graph partitioning algorithm can be applied to greatly increase scalability .

O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

content analysis and indexing
document management
near duplicate detection
similarity


Structure and evolution of online social networks In this paper , we consider the evolution of structure within large online social networks . We present a series of measurements of two such networks , together comprising in excess of five million people and ten million friendship links , annotated with metadata capturing the time of every event in the life of the network . Our measurements expose a surprising segmentation of these networks into three regions : singletons who do not participate in the network ; isolated communities which overwhelmingly display star structure ; and a giant component anchored by a well-connected core region which persists even in the absence of stars . We present a simple model of network growth which captures these aspects of component structure . The model follows our experimental results , characterizing users as either passive members of the network ; inviters who encourage offline friends and acquaintances to migrate online ; and linkers who fully participate in the social evolution of the network .

O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graph evolution
graph mining
small-world phenomenon


Automatic mining of fruit fly embryo images We present FEMine , an automatic system for image-based gene expression analysis . We perform experiments on the largest publicly available collection of Drosophila ISH ( in situ hybridization ) images , showing that our FEMine system achieves excellent performance in classification , clustering , and content-based image retrieval . The major innovation of FEMine is the use of automatically discovered latent spatial `` themes '' of gene expressions , LGEs , in the whole-embryo context , as opposed to patterns in nearly disjoint portions of an embryo proposed in previous methods .

O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

embryonic image analysis
independent component analysis


Mining uncertain data with probabilistic guarantees Data uncertainty is inherent in applications such as sensor monitoring systems , location-based services , and biological databases . To manage this vast amount of imprecise information , probabilistic databases have been recently developed . In this paper , we study the discovery of frequent patterns and association rules from probabilistic data under the Possible World Semantics . This is technically challenging , since a probabilistic database can have an exponential number of possible worlds . We propose two effcient algorithms , which discover frequent patterns in bottom-up and top-down manners . Both algorithms can be easily extended to discover maximal frequent patterns . We also explain how to use these patterns to generate association rules . Extensive experiments , using real and synthetic datasets , were conducted to validate the performance of our methods .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A scalable modular convex solver for regularized risk minimization A wide variety of machine learning problems can be described as minimizing a regularized risk functional , with different algorithms using different notions of risk and different regularizers . Examples include linear Support Vector Machines ( SVMs ) , Logistic Regression , Conditional Random Fields ( CRFs ) , and Lasso amongst others . This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems . It can be parallelized on a cluster of workstations , allows for data-locality , and can deal with regularizers such as l1 and l2 penalties . At present , our solver implements 20 different estimation problems , can be easily extended , scales to millions of observations , and is up to 10 times faster than specialized solvers for many applications . The open source code is freely available as part of the ELEFANT toolbox .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Real-time bidding algorithms for performance-based display ad allocation We describe a real-time bidding algorithm for performance-based display ad allocation . A central issue in performance display advertising is matching campaigns to ad impressions , which can be formulated as a constrained optimization problem that maximizes revenue subject to constraints such as budget limits and inventory availability . The current practice is to solve the optimization problem offline at a tractable level of impression granularity ( e.g. , the page level ) , and to serve ads online based on the precomputed static delivery scheme . Although this offline approach takes a global view to achieve optimality , it fails to scale to ad allocation at the individual impression level . Therefore , we propose a real-time bidding algorithm that enables fine-grained impression valuation ( e.g. , targeting users with real-time conversion data ) , and adjusts value-based bids according to real-time constraint snapshots ( e.g. , budget consumption levels ) . Theoretically , we show that under a linear programming ( LP ) primal-dual formulation , the simple real-time bidding algorithm is indeed an online solver to the original primal problem by taking the optimal solution to the dual problem as input . In other words , the online algorithm guarantees the offline optimality given the same level of knowledge an offline optimization would have . Empirically , we develop and experiment with two real-time bid adjustment approaches to adapting to the non-stationary nature of the marketplace : one adjusts bids against real-time constraint satisfaction levels using control-theoretic methods , and the other adjusts bids also based on the statistically modeled historical bidding landscape . Finally , we show experimental results with real-world ad delivery data that support our theoretical conclusions .

O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

ad exchange
combinatorial optimization
real-time bidding


Tumor cell identification using features rules Advances in imaging techniques have led to large repositories of images . There is an increasing demand for automated systems that can analyze complex medical images and extract meaningful information for mining patterns . Here , we describe a real-life image mining application to the problem of tumor cell counting . The quantitative analysis of tumor cells is fundamental to characterizing the activity of tumor cells . Existing approaches are mostly manual , time-consuming and subjective . Efforts to automate the process of cell counting have largely focused on using image processing techniques only . Our studies indicate that image processing alone is unable to give accurate results . In this paper , we examine the use of extracted features rules to aid in the process of tumor cell counting . We propose a robust local adaptive thresholding and dynamic water immersion algorithms to segment regions of interesting from background . Meaningful features are then extracted from the segmented regions . A number of base classifiers are built to generate features rules to help identify the tumor cell . Two voting strategies are implemented to combine the base classifiers into a meta-classifier . Experiment results indicate that this process of using extracted features rules to help identify tumor cell leads to better accuracy than pure image processing techniques alone .

O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O

dynamic water immersion
features rules
identification
local adaptive thresholding

majority vote
meta classifier
weighted vote


Regression error characteristic surfaces This paper presents a generalization of Regression Error Characteristic ( REC ) curves . REC curves describe the cumulative distribution function of the prediction error of models and can be seen as a generalization of ROC curves to regression problems . REC curves provide useful information for analyzing the performance of models , particularly when compared to error statistics like for instance the Mean Squared Error . In this paper we present Regression Error Characteristic ( REC ) surfaces that introduce a further degree of detail by plotting the cumulative distribution function of the errors across the distribution of the target variable , i.e. the joint cumulative distribution function of the errors and the target variable . This provides a more detailed analysis of the performance of models when compared to REC curves . This extra detail is particularly relevant in applications with non-uniform error costs , where it is important to study the performance of models for specific ranges of the target variable . In this paper we present the notion of REC surfaces , describe how to use them to compare the performance of models , and illustrate their use with an important practical class of applications : the prediction of rare extreme values .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A fast kernel-based multilevel algorithm for graph clustering Graph clustering ( also called graph partitioning ) -- clustering the nodes of a graph -- is an important problem in diverse data mining applications . Traditional approaches involve optimization of graph clustering objectives such as normalized cut or ratio association ; spectral methods are widely used for these objectives , but they require eigenvector computation which can be slow . Recently , graph clustering with a general cut objective has been shown to be mathematically equivalent to an appropriate weighted kernel k-means objective function . In this paper , we exploit this equivalence to develop a very fast multilevel algorithm for graph clustering . Multilevel approaches involve coarsening , initial partitioning and refinement phases , all of which may be specialized to different graph clustering objectives . Unlike existing multilevel clustering approaches , such as METIS , our algorithm does not constrain the cluster sizes to be nearly equal . Our approach gives a theoretical guarantee that the refinement step decreases the graph cut objective under consideration . Experiments show that we achieve better final objective function values as compared to a state-of-the-art spectral clustering algorithm : on a series of benchmark test graphs with up to thirty thousand nodes and one million edges , our algorithm achieves lower normalized cut values in 67 % of our experiments and higher ratio association values in 100 % of our experiments . Furthermore , on large graphs , our algorithm is significantly faster than spectral methods . Finally , our algorithm requires far less memory than spectral methods ; we cluster a 1.2 million node movie network into 5000 clusters , which due to memory requirements can not be done directly with spectral methods .

O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On effective classification of strings with wavelets In recent years , the technological advances in mapping genes have made it increasingly easy to store and use a wide variety of biological data . Such data are usually in the form of very long strings for which it is difficult to determine the most relevant features for a classification task . For example , a typical DNA string may be millions of characters long , and there may be thousands of such strings in a database . In many cases , the classification behavior of the data may be hidden in the compositional behavior of certain segments of the string which can not be easily determined apriori . Another problem which complicates the classification task is that in some cases the classification behavior is reflected in global behavior of the string , whereas in others it is reflected in local patterns . Given the enormous variation in the behavior of the strings over different data sets , it is useful to develop an approach which is sensitive to both the global and local behavior of the strings for the purpose of classification . For this purpose , we will exploit the multi-resolution property of wavelet decomposition in order to create a scheme which can mine classification characteristics at different levels of granularity . The resulting scheme turns out to be very effective in practice on a wide range of problems .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Locating secret messages in images Steganography involves hiding messages in innocuous media such as images , while steganalysis is the field of detecting these secret messages . The ultimate goal of steganalysis is two-fold : making a binary classification of a file as stego-bearing or innocent , and secondly , locating the hidden message with an aim to extracting , sterilizing or manipulating it . Almost all steganalysis approaches ( known as attacks ) focus on the first of these two issues . In this paper , we explore the difficult related problem : given that we know an image file contains steganography , locate which pixels contain the message . We treat the hidden message location problem as outlier detection using probability\/energy measures of images motivated by the image restoration community . Pixels contributing the most to the energy calculations of an image are deemed outliers . Typically , of the top third of one percent of most energized pixels ( outliers ) , we find that 87 % are stego-bearing in color images and 61 % in grayscale images . In all image types only 1 % of all pixels are stego-bearing indicating our techniques provides a substantial lift over random guessing .

O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

outlier detection
steganalysis
steganography


Discovering associations with numeric variables This paper further develops Aumann and Lindell 's ( 3 ) proposal for a variant of association rules for which the consequent is a numeric variable . It is argued that these rules can discover useful interactions with numeric data that can not be discovered directly using traditional association rules with discretization . Alternative measures for identifying interesting rules are proposed . Efficient algorithms are presented that enable these rules to be discovered for dense data sets for which application of Auman and Lindell 's algorithm is infeasible .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

association rule
impact rule
information search and retrieval
learning
search


Fast approximate spectral clustering Spectral clustering refers to a flexible class of clustering procedures that can produce high-quality clusterings on small data sets but which has limited applicability to large-scale problems due to its computational complexity of O ( n3 ) in general , with n the number of data points . We extend the range of spectral clustering by developing a general framework for fast approximate spectral clustering in which a distortion-minimizing local transformation is first applied to the data . This framework is based on a theoretical analysis that provides a statistical characterization of the effect of local distortion on the mis-clustering rate . We develop two concrete instances of our general framework , one based on local k-means clustering ( KASP ) and one based on random projection trees ( RASP ) . Extensive experiments show that these algorithms can achieve significant speedups with little degradation in clustering accuracy . Specifically , our algorithms outperform k-means by a large margin in terms of accuracy , and run several times faster than approximate spectral clustering based on the Nystrom method , with comparable accuracy and significantly smaller memory footprint . Remarkably , our algorithms make it possible for a single machine to spectral cluster data sets with a million observations within several minutes .

O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data quantization
learning
unsupervised learning


Exploring social tagging graph for web object classification This paper studies web object classification problem with the novel exploration of social tags . Automatically classifying web objects into manageable semantic categories has long been a fundamental preprocess for indexing , browsing , searching , and mining these objects . The explosive growth of heterogeneous web objects , especially non-textual objects such as products , pictures , and videos , has made the problem of web classification increasingly challenging . Such objects often suffer from a lack of easy-extractable features with semantic information , interconnections between each other , as well as training examples with category labels . In this paper , we explore the social tagging data to bridge this gap . We cast web object classification problem as an optimization problem on a graph of objects and tags . We then propose an efficient algorithm which not only utilizes social tags as enriched semantic features for the objects , but also infers the categories of unlabeled objects from both homogeneous and heterogeneous labeled objects , through the implicit connection of social tags . Experiment results show that the exploration of social tags effectively boosts web object classification . Our algorithm significantly outperforms the state-of-the-art of general classification methods .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O

general
optimization
social tagging
web classification


Trajectory pattern mining The increasing pervasiveness of location-acquisition technologies ( GPS , GSM networks , etc. ) is leading to the collection of large spatio-temporal datasets and to the opportunity of discovering usable knowledge about movement behavior , which fosters novel applications and services . In this paper , we move towards this direction and develop an extension of the sequential pattern mining paradigm that analyzes the trajectories of moving objects . We introduce trajectory patterns as concise descriptions of frequent behaviors , in terms of both space ( i.e. , the regions of space visited during movements ) and time ( i.e. , the duration of movements ) . In this setting , we provide a general formal statement of the novel mining problem and then study several different instantiations of different complexity . The various approaches are then empirically evaluated over real data and synthetic benchmarks , comparing their strengths and weaknesses .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A general approach to incorporate data quality matrices into data mining algorithms Data quality is a central issue for many information-oriented organizations . Recent advances in the data quality field reflect the view that a database is the product of a manufacturing process . While routine errors , such as non-existent zip codes , can be detected and corrected using traditional data cleansing tools , many errors systemic to the manufacturing process can not be addressed . Therefore , the product of the data manufacturing process is an imprecise recording of information about the entities of interest ( i.e. customers , transactions or assets ) . In this way , the database is only one ( flawed ) version of the entities it is supposed to represent . Quality assurance systems such as Motorola 's Six-Sigma and other continuous improvement methods document the data manufacturing process 's shortcomings . A widespread method of documentation is quality matrices . In this paper , we explore the use of the readily available data quality matrices for the data mining classification task . We first illustrate that if we do not factor in these quality matrices , then our results for prediction are sub-optimal . We then suggest a general-purpose ensemble approach that perturbs the data according to these quality matrices to improve the predictive accuracy and show the improvement is due to a reduction in variance .

O O O O O B I O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

decision trees
ensemble approaches
six-sigma


Generation of synthetic data sets for evaluating the accuracy of knowledge discovery systems Information Discovery and Analysis Systems ( IDAS ) are designed to correlate multiple sources of data and use data mining techniques to identify potential significant events . Application domains for IDAS are numerous and include the emerging area of homeland security . Developing test cases for an IDAS requires background data sets into which hypothetical future scenarios can be overlaid . The IDAS can then be measured in terms of false positive and false negative error rates . Obtaining the test data sets can be an obstacle due to both privacy issues and also the time and cost associated with collecting a diverse set of data sources . In this paper , we give an overview of the design and architecture of an IDAS Data Set Generator ( IDSG ) that enables a fast and comprehensive test of an IDAS . The IDSG generates data using statistical and rule-based algorithms and also semantic graphs that represent interdependencies between attributes . A credit card transaction application is used to illustrate the approach .

O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


PROXIMUS : a framework for analyzing very high dimensional discrete-attributed datasets This paper presents an efficient framework for error-bounded compression of high-dimensional discrete attributed datasets . Such datasets , which frequently arise in a wide variety of applications , pose some of the most significant challenges in data analysis . Subsampling and compression are two key technologies for analyzing these datasets . PROXIMUS provides a technique for reducing large datasets into a much smaller set of representative patterns , on which traditional ( expensive ) analysis algorithms can be applied with minimal loss of accuracy . We show desirable properties of PROXIMUS in terms of runtime , scalability to large datasets , and performance in terms of capability to represent data in a compact form . We also demonstrate applications of PROXIMUS in association rule mining . In doing so , we establish PROXIMUS as a tool for preprocessing data before applying computationally expensive algorithms or as a tool for directly extracting correlated patterns . Our experimental results show that use of the compressed data for association rule mining provides excellent precision and recall values ( near 100 % ) across a range of support thresholds while reducing the time required for association rule mining drastically .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

clustering
compressing discrete-valued vectors
non-orthogonal matrix decompositions
semi-discrete decomposition


Feature bagging for outlier detection Outlier detection has recently become an important problem in many industrial and financial applications . In this paper , a novel feature bagging approach for detecting outliers in very large , high dimensional and noisy databases is proposed . It combines results from multiple outlier detection algorithms that are applied using different set of features . Every outlier detection algorithm uses a small subset of features that are randomly selected from the original feature set . As a result , each outlier detector identifies different outliers , and thus assigns to all data records outlier scores that correspond to their probability of being outliers . The outlier scores computed by the individual outlier detection algorithms are then combined in order to find the better quality outliers . Experiments performed on several synthetic and real life data sets show that the proposed methods for combining outputs from multiple outlier detection algorithms provide non-trivial improvements over the base algorithm .

O B O B I B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O

detection rate
false alarm
feature subsets
integration


Bias and controversy : beyond the statistical deviation In this paper , we investigate how deviation in evaluation activities may reveal bias on the part of reviewers and controversy on the part of evaluated objects . We focus on a ` data-centric approach ' where the evaluation data is assumed to represent the ` ground truth ' . The standard statistical approaches take evaluation and deviation at face value . We argue that attention should be paid to the subjectivity of evaluation , judging the evaluation score not just on ` what is being said ' ( deviation ) , but also on ` who says it ' ( reviewer ) as well as on ` whom it is said about ' ( object ) . Furthermore , we observe that bias and controversy are mutually dependent , as there is more bias if there is higher deviation on a less controversial object . To address this mutual dependency , we propose a reinforcement model to identify bias and controversy . We test our model on real-life data to verify its applicability .

B O B O O O O O O O O O O O O O O B O O O B O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O

information systems applications
social and behavioral sciences
social network


High-precision phrase-based document classification on a modern scale We present a document classification system that employs lazy learning from labeled phrases , and argue that the system can be highly effective whenever the following property holds : most of information on document labels is captured in phrases . We call this property near sufficiency . Our research contribution is twofold : ( a ) we quantify the near sufficiency property using the Information Bottleneck principle and show that it is easy to check on a given dataset ; ( b ) we reveal that in all practical cases -- from small-scale to very large-scale -- manual labeling of phrases is feasible : the natural language constrains the number of common phrases composed of a vocabulary to grow linearly with the size of the vocabulary . Both these contributions provide firm foundation to applicability of the phrase-based classification ( PBC ) framework to a variety of large-scale tasks . We deployed the PBC system on the task of job title classification , as a part of LinkedIn 's data standardization effort . The system significantly outperforms its predecessor both in terms of precision and coverage . It is currently being used in LinkedIn 's ad targeting product , and more applications are being developed . We argue that PBC excels in high explainability of the classification results , as well as in low development and low maintenance costs . We benchmark PBC against existing high-precision document classification algorithms and conclude that it is most useful in multilabel classification .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

high-precision classification
large-scale classification
multilabel text classification


Incorporating site-level knowledge for incremental crawling of web forums : a list-wise strategy We study in this paper the problem of incremental crawling of web forums , which is a very fundamental yet challenging step in many web applications . Traditional approaches mainly focus on scheduling the revisiting strategy of each individual page . However , simply assigning different weights for different individual pages is usually inefficient in crawling forum sites because of the different characteristics between forum sites and general websites . Instead of treating each individual page independently , we propose a list-wise strategy by taking into account the site-level knowledge . Such site-level knowledge is mined through reconstructing the linking structure , called sitemap , for a given forum site . With the sitemap , posts from the same thread but distributed on various pages can be concatenated according to their timestamps . After that , for each thread , we employ a regression model to predict the time when the next post arrives . Based on this model , we develop an efficient crawler which is 260 % faster than some state-of-the-art methods in terms of fetching new generated content ; and meanwhile our crawler also ensure a high coverage ratio . Experimental results show promising performance of Coverage , Bandwidth utilization , and Timeliness of our crawler on 18 various forums .

O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Rapid detection of significant spatial clusters Given an N x N grid of squares , where each square has a count cij and an underlying population pij , our goal is to find the rectangular region with the highest density , and to calculate its significance by randomization . An arbitrary density function D , dependent on a region 's total count C and total population P , can be used . For example , if each count represents the number of disease cases occurring in that square , we can use Kulldorff 's spatial scan statistic DK to find the most significant spatial disease cluster . A naive approach to finding the maximum density region requires O ( N4 ) time , and is generally computationally infeasible . We present a multiresolution algorithm which partitions the grid into overlapping regions using a novel overlap-kd tree data structure , bounds the maximum score of subregions contained in each region , and prunes regions which can not contain the maximum density region . For sufficiently dense regions , this method finds the maximum density region in O ( ( N log N ) 2 ) time , in practice resulting in significant ( 20-2000x ) speedups on both real and simulated datasets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

biosurveillance
cluster detection
spatial data mining algorithms
spatial scan statistics


Efficient anomaly monitoring over moving object trajectory streams Lately there exist increasing demands for online abnormality monitoring over trajectory streams , which are obtained from moving object tracking devices . This problem is challenging due to the requirement of high speed data processing within limited space cost . In this paper , we present a novel framework for monitoring anomalies over continuous trajectory streams . First , we illustrate the importance of distance-based anomaly monitoring over moving object trajectories . Then , we utilize the local continuity characteristics of trajectories to build local clusters upon trajectory streams and monitor anomalies via efficient pruning strategies . Finally , we propose a piecewise metric index structure to reschedule the joining order of local clusters to further reduce the time cost . Our extensive experiments demonstrate the effectiveness and efficiency of our methods .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

outlier detection
similarity search
temporal data


Using association rules for product assortment decisions : a case study

association rules
product assortment decisions


Improving spatial locality of programs via data mining In most computer systems , page fault rate is currently minimized by generic page replacement algorithms which try to model the temporal locality inherent in programs . In this paper , we propose two algorithms , one greedy and the other stochastic , designed for program specific code restructuring as a means of increasing spatial locality within a program . Both algorithms effectively decrease average working set size and hence the page fault rate . Our methods are more effective than traditional approaches due to use of domain information . We illustrate the efficacy of our algorithms on actual data mining algorithms .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

database applications
page clustering
program locality


Effective label acquisition for collective classification Information diffusion , viral marketing , and collective classification all attempt to model and exploit the relationships in a network to make inferences about the labels of nodes . A variety of techniques have been introduced and methods that combine attribute information and neighboring label information have been shown to be effective for collective labeling of the nodes in a network . However , in part because of the correlation between node labels that the techniques exploit , it is easy to find cases in which , once a misclassification is made , incorrect information propagates throughout the network . This problem can be mitigated if the system is allowed to judiciously acquire the labels for a small number of nodes . Unfortunately , under relatively general assumptions , determining the optimal set of labels to acquire is intractable . Here we propose an acquisition method that learns the cases when a given collective classification algorithm makes mistakes , and suggests acquisitions to correct those mistakes . We empirically show on both real and synthetic datasets that this method significantly outperforms a greedy approximate inference approach , a viral marketing approach , and approaches based on network structural measures such as node degree and network clustering . In addition to significantly improving accuracy with just a small amount of labeled data , our method is tractable on large networks .

O B I O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

collective classification
label acquisition


Data mining to predict and prevent errors in health insurance claims processing Health insurance costs across the world have increased alarmingly in recent years . A major cause of this increase are payment errors made by the insurance companies while processing claims . These errors often result in extra administrative effort to re-process ( or rework ) the claim which accounts for up to 30 % of the administrative staff in a typical health insurer . We describe a system that helps reduce these errors using machine learning techniques by predicting claims that will need to be reworked , generating explanations to help the auditors correct these claims , and experiment with feature selection , concept drift , and active learning to collect feedback from the auditors to improve over time . We describe our framework , problem formulation , evaluation metrics , and experimental results on claims data from a large US health insurer . We show that our system results in an order of magnitude better precision ( hit rate ) over existing approaches which is accurate enough to potentially result in over $ 15-25 million in savings for a typical insurer . We also describe interesting research problems in this domain as well as design choices made to make the system easily deployable across health insurance companies .

O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

health insurance claims
machine learning

claim rework identification
decision support


Shrinkage estimator generalizations of Proximal Support Vector Machines We give a statistical interpretation of Proximal Support Vector Machines ( PSVM ) proposed at KDD2001 as linear approximaters to ( nonlinear ) Support Vector Machines ( SVM ) . We prove that PSVM using a linear kernel is identical to ridge regression , a biased-regression method known in the statistical community for more than thirty years . Techniques from the statistical literature to estimate the tuning constant that appears in the SVM and PSVM framework are discussed . Better shrinkage strategies that incorporate more than one tuning constant are suggested . For nonlinear kernels , the minimization problem posed in the PSVM framework is equivalent to finding the posterior mode of a Bayesian model defined through a Gaussian process on the predictor space . Apart from providing new insights , these interpretations help us attach an estimate of uncertainty to our predictions and enable us to build richer classes of models . In particular , we propose a new algorithm called PSVMMIX which is a combination of ridge regression and a Gaussian process model . Extension to the case of continuous response is straightforward and illustrated with example datasets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O

bayesian models
bias-variance tradeoff
classification
correlation
probability and statistics


A Web page prediction model based on click-stream tree representation of user behavior Predicting the next request of a user as she visits Web pages has gained importance as Web-based activity increases . Markov models and their variations , or models based on sequence mining have been found well suited for this problem . However , higher order Markov models are extremely complicated due to their large number of states whereas lower order Markov models do not capture the entire behavior of a user in a session . The models that are based on sequential pattern mining only consider the frequent sequences in the data set , making it difficult to predict the next request following a page that is not in the sequential pattern . Furthermore , it is hard to find models for mining two different kinds of information of a user session . We propose a new model that considers both the order information of pages in a session and the time spent on them . We cluster user sessions based on their pair-wise similarity and represent the resulting clusters by a click-stream tree . The new user session is then assigned to a cluster based on a similarity measure . The click-stream tree of that cluster is used to generate the recommendation set . The model can be used as part of a cache prefetching system as well as a recommendation model .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

design methodology
graph based clustering
two dimensional sequential model
web usage mining


Exploiting Wikipedia as external knowledge for document clustering In traditional text clustering methods , documents are represented as `` bags of words '' without considering the semantic information of each document . For instance , if two documents use different collections of core words to represent the same topic , they may be falsely assigned to different clusters due to the lack of shared core words , although the core words they use are probably synonyms or semantically associated in other forms . The most common way to solve this problem is to enrich document representation with the background knowledge in an ontology . There are two major issues for this approach : ( 1 ) the coverage of the ontology is limited , even for WordNet or Mesh , ( 2 ) using ontology terms as replacement or additional features may cause information loss , or introduce noise . In this paper , we present a novel text clustering method to address these two issues by enriching document representation with Wikipedia concept and category information . We develop two approaches , exact match and relatedness-match , to map text documents to Wikipedia concepts , and further to Wikipedia categories . Then the text documents are clustered based on a similarity metric which combines document content information , concept information as well as category information . The experimental results using the proposed clustering framework on three datasets ( 20-newsgroup , TDT2 , and LA Times ) show that clustering performance improves significantly by enriching document representation with Wikipedia concepts and categories .

O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O

document representation
text clustering
wikipedia