Optimizing web traffic via the media scheduling problem Website traffic varies through time in consistent and predictable ways , with highest traffic in the middle of the day . When providing media content to visitors , it is important to present repeat visitors with new content so that they keep coming back . In this paper we present an algorithm to balance the need to keep a website fresh with new content with the desire to present the best content to the most visitors at times of peak traffic . We formulate this as the media scheduling problem , where we attempt to maximize total clicks , given the overall traffic pattern and the time varying clickthrough rates of available media content . We present an efficient algorithm to perform this scheduling under certain conditions and apply this algorithm to real data obtained from server logs , showing evidence of significant improvements in traffic from our algorithmic schedules . Finally , we analyze the click data , presenting models for why and how the clickthrough rate for new content declines as it ages .

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On privacy preservation against adversarial data mining Privacy preserving data processing has become an important topic recently because of advances in hardware technology which have lead to widespread proliferation of demographic and sensitive data . A rudimentary way to preserve privacy is to simply hide the information in some of the sensitive fields picked by a user . However , such a method is far from satisfactory in its ability to prevent adversarial data mining . Real data records are not randomly distributed . As a result , some fields in the records may be correlated with one another . If the correlation is sufficiently high , it may be possible for an adversary to predict some of the sensitive fields using other fields . In this paper , we study the problem of privacy preservation against adversarial data mining , which is to hide a minimal set of entries so that the privacy of the sensitive fields are satisfactorily preserved . In other words , even by data mining , an adversary still can not accurately recover the hidden data entries . We model the problem concisely and develop an efficient heuristic algorithm which can find good solutions in practice . An extensive performance study is conducted on both synthetic and real data sets to examine the effectiveness of our approach .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Probabilistic workflow mining In several organizations , it has become increasingly popular to document and log the steps that makeup a typical business process . In some situations , a normative workflow model of such processes is developed , and it becomes important to know if such a model is actually being followed by analyzing the available activity logs . In other scenarios , no model is available and , with the purpose of evaluating cases or creating new production policies , one is interested in learning a workflow representation of such activities . In either case , machine learning tools that can mine workflow models are of great interest and still relatively unexplored . We present here a probabilistic workflow model and a corresponding learning algorithm that runs in polynomial time . We illustrate the algorithm on example data derived from a real world workflow .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

causal models
graphical models
probability and statistics


A framework for analysis of dynamic social networks Finding patterns of social interaction within a population has wide-ranging applications including : disease modeling , cultural and information transmission , and behavioral ecology . Social interactions are often modeled with networks . A key characteristic of social interactions is their continual change . However , most past analyses of social networks are essentially static in that all information about the time that social interactions take place is discarded . In this paper , we propose a new mathematical and computational framework that enables analysis of dynamic social networks and that explicitly makes use of information about when social interactions occur .

O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O


Nomograms for visualizing support vector machines We propose a simple yet potentially very effective way of visualizing trained support vector machines . Nomograms are an established model visualization technique that can graphically encode the complete model on a single page . The dimensionality of the visualization does not depend on the number of attributes , but merely on the properties of the kernel . To represent the effect of each predictive feature on the log odds ratio scale as required for the nomograms , we employ logistic regression to convert the distance from the separating hyperplane into a probability . Case studies on selected data sets show that for a technique thought to be a black-box , nomograms can clearly expose its internal structure . By providing an easy-to-interpret visualization the analysts can gain insight and study the effects of predictive factors .

O O O B I I O O O O O O O O O O O O B I I O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O


Aggregation-based feature invention and relational concept classes Model induction from relational data requires aggregation of the values of attributes of related entities . This paper makes three contributions to the study of relational learning . ( 1 ) It presents a hierarchy of relational concepts of increasing complexity , using relational schema characteristics such as cardinality , and derives classes of aggregation operators that are needed to learn these concepts . ( 2 ) Expanding one level of the hierarchy , it introduces new aggregation operators that model the distributions of the values to be aggregated and ( for classification problems ) the differences in these distributions by class . ( 3 ) It demonstrates empirically on a noisy business domain that more-complex aggregation methods can increase generalization performance . Constructing features using target-dependent aggregations can transform relational prediction tasks so that well-understood feature-vector-based modeling algorithms can be applied successfully .

O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O

constructive induction
feature construction
propositionalization
relational learning


Making every bit count : fast nonlinear axis scaling Existing axis scaling and dimensionality methods focus on preserving structure , usually determined via the Euclidean distance . In other words , they inherently assume that the Euclidean distance is already correct . We instead propose a novel nonlinear approach driven by an information-theoretic viewpoint , which we show is also strongly linked to intrinsic dimensionality , or degrees of freedom ; and uniformity . Nonlinear transformations based on common probability distributions , combined with information-driven selection , simultaneously reduce the number of dimensions required and increase the value of those we retain . Experiments on real data confirm that this approach reveals correlations , finds novel attributes , and scales well .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Exploiting unlabeled data in ensemble methods An adaptive semi-supervised ensemble method , ASSEMBLE , is proposed that constructs classification ensembles based on both labeled and unlabeled data . ASSEMBLE alternates between assigning `` pseudo-classes '' to the unlabeled data using the existing ensemble and constructing the next base classifier using both the labeled and pseudolabeled data . Mathematically , this intuitive algorithm corresponds to maximizing the classification margin in hypothesis space as measured on both the labeled and unlabeled of data . Unlike alternative approaches , ASSEMBLE does not require a semi-supervised learning method for the base classifier . ASSEMBLE can be used in conjunction with any cost-sensitive classification algorithm for both two-class and multi-class problems . ASSEMBLE using decision trees won the NIPS 2001 Unlabeled Data Competition . In addition , strong results on several benchmark datasets using both decision trees and neural networks support the proposed method .

O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

boosting
ensemble learning
self-modifying machines
semi-supervised learning


A probabilistic model for personalized tag prediction Social tagging systems have become increasingly popular for sharing and organizing web resources . Tag prediction is a common feature of social tagging systems . Social tagging by nature is an incremental process , meaning that once a user has saved a web page with tags , the tagging system can provide more accurate predictions for the user , based on user 's incremental behaviors . However , existing tag prediction methods do not consider this important factor , in which their training and test datasets are either split by a fixed time stamp or randomly sampled from a larger corpus . In our temporal experiments , we perform a time-sensitive sampling on an existing public dataset , resulting in a new scenario which is much closer to `` real-world '' . In this paper , we address the problem of tag prediction by proposing a probabilistic model for personalized tag prediction . The model is a Bayesian approach , and integrates three factors - ego-centric effect , environmental effects and web page content . Two methods - both intuitive calculation and learning optimization - are provided for parameter estimation . Pure graphbased methods which may have significant constraints ( such as every user , every item and every tag has to occur in at least p posts ) , can not make a prediction in most of `` real world '' cases while our model improves the F-measure by over 30 % compared to a leading algorithm , in our `` real-world '' use case .

O O O O B I I B I O O O O O O O O O O O O B I O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

personalized tag prediction
social tagging
tag prediction


BBM : bayesian browsing model from petabyte-scale data Given a quarter of petabyte click log data , how can we estimate the relevance of each URL for a given query ? In this paper , we propose the Bayesian Browsing Model ( BBM ) , a new modeling technique with following advantages : ( a ) it does exact inference ; ( b ) it is single-pass and parallelizable ; ( c ) it is effective . We present two sets of experiments to test model effectiveness and efficiency . On the first set of over 50 million search instances of 1.1 million distinct queries , BBM out-performs the state-of-the-art competitor by 29.2 % in log-likelihood while being 57 times faster . On the second click-log set , spanning a quarter of petabyte data , we showcase the scalability of BBM : we implemented it on a commercial MapReduce cluster , and it took only 3 hours to compute the relevance for 1.15 billion distinct query-URL pairs .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

bayesian models
click log analysis
web search


Assessing data mining results via swap randomization The problem of assessing the significance of data mining results on high-dimensional 0-1 data sets has been studied extensively in the literature . For problems such as mining frequent sets and finding correlations , significance testing can be done by , e.g. , chi-square tests , or many other methods . However , the results of such tests depend only on the specific attributes and not on the dataset as a whole . Moreover , the tests are more difficult to apply to sets of patterns or other complex results of data mining . In this paper , we consider a simple randomization technique that deals with this shortcoming . The approach consists of producing random datasets that have the same row and column margins with the given dataset , computing the results of interest on the randomized instances , and comparing them against the results on the actual data . This randomization technique can be used to assess the results of many different types of data mining algorithms , such as frequent sets , clustering , and rankings . To generate random datasets with given margins , we use variations of a Markov chain approach , which is based on a simple swap operation . We give theoretical results on the efficiency of different randomization methods , and apply the swap randomization method to several well-known datasets . Our results indicate that for some datasets the structure discovered by the data mining algorithms is a random artifact , while for other datasets the discovered structure conveys meaningful information .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


To buy or not to buy : mining airfare data to minimize ticket purchase price As product prices become increasingly available on the World Wide Web , consumers attempt to understand how corporations vary these prices over time . However , corporations change prices based on proprietary algorithms and hidden variables ( e.g. , the number of unsold seats on a flight ) . Is it possible to develop data mining techniques that will enable consumers to predict price changes under these conditions ? This paper reports on a pilot study in the domain of airline ticket prices where we recorded over 12,000 price observations over a 41 day period . When trained on this data , Hamlet -- our multi-strategy data mining algorithm -- generated a predictive model that saved 341 simulated passengers $ 198,074 by advising them when to buy and when to postpone ticket purchases . Remarkably , a clairvoyant algorithm with complete knowledge of future prices could save at most $ 320,572 in our simulation , thus HAMLET 's savings were 61.8 % of optimal . The algorithm 's savings of $ 198,074 represents an average savings of 23.8 % for the 341 passengers for whom savings are possible . Overall , HAMLET saved 4.4 % of the ticket price averaged over the entire set of 4,488 simulated passengers . Our pilot study suggests that mining of price data available over the web has the potential to save consumers substantial sums of money per annum .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

airline price prediction
internet
learning
price mining
web mining


Maximum profit mining and its application in software development While most software defects ( i.e. , bugs ) are corrected and tested as part of the lengthy software development cycle , enterprise software vendors often have to release software products before all reported defects are corrected , due to deadlines and limited resources . A small number of these defects will be escalated by customers and they must be resolved immediately by the software vendors at a very high cost . In this paper , we develop an Escalation Prediction ( EP ) system that mines historic defect report data and predict the escalation risk of the defects for maximum net profit . More specifically , we first describe a simple and general framework to convert the maximum net profit problem to cost-sensitive learning . We then apply and compare several well-known cost-sensitive learning approaches for EP . Our experiments suggest that the cost-sensitive decision tree is the best method for producing the highest positive net profit and comprehensible results . The EP system has been deployed successfully in the product group of an enterprise software vendor .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Spotting out emerging artists using geo-aware analysis of P2P query strings Record label companies would like to identify potential artists as early as possible in their careers , before other companies approach the artists with competing contracts . The vast number of candidates makes the process of identifying the ones with high success potential time consuming and laborious . This paper demonstrates how datamining of P2P query strings can be used in order to mechanize most of this detection process . Using a unique intercepting system over the Gnutella network , we were able to capture an unprecedented amount of geographically identified ( geo-aware ) queries , allowing us to investigate the diffusion of music related queries in time and space . Our solution is based on the observation that emerging artists , especially rappers , have a discernible stronghold of fans in their hometown area , where they are able to perform and market their music . In a file sharing network , this is reflected as a delta function spatial distribution of content queries . Using this observation , we devised a detection algorithm for emerging artists , that looks for performers with sharp increase in popularity in a small geographic region though still unnoticable nation wide . The algorithm can suggest a short list of artists with breakthrough potential , from which we showed that about 30 % translate the potential to national success .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Exploiting dictionaries in named entity extraction : combining semi-Markov extraction processes and data integration methods We consider the problem of improving named entity recognition ( NER ) systems by using external dictionaries -- more specifically , the problem of extending state-of-the-art NER systems by incorporating information about the similarity of extracted entities to entities in an external dictionary . This is difficult because most high-performance named entity recognition systems operate by sequentially classifying words as to whether or not they participate in an entity name ; however , the most useful similarity measures score entire candidate names . To correct this mismatch we formalize a semi-Markov extraction process , which is based on sequentially classifying segments of several adjacent words , rather than single words . In addition to allowing a natural way of coupling high-performance NER methods and high-performance similarity functions , this formalism also allows the direct use of other useful entity-level features , and provides a more natural formulation of the NER problem than sequential word classification . Experiments in multiple domains show that the new model can substantially improve extraction performance over previous methods for using external dictionaries in NER .

O O O O O O O O O O O O B I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data integration
named entity recognition

information extraction
learning
sequential learning


On community outliers and their efficient detection in information networks Linked or networked data are ubiquitous in many applications . Examples include web data or hypertext documents connected via hyperlinks , social networks or user profiles connected via friend links , co-authorship and citation information , blog data , movie reviews and so on . In these datasets ( called `` information networks '' ) , closely related objects that share the same properties or interests form a community . For example , a community in blogsphere could be users mostly interested in cell phone reviews and news . Outlier detection in information networks can reveal important anomalous and interesting behaviors that are not obvious if community information is ignored . An example could be a low-income person being friends with many rich people even though his income is not anomalously low when considered over the entire population . This paper first introduces the concept of community outliers ( interesting points or rising stars for a more positive sense ) , and then shows that well-known baseline approaches without considering links or community information can not find these community outliers . We propose an efficient solution by modeling networked data as a mixture model composed of multiple normal communities and a set of randomly generated outliers . The probabilistic model characterizes both data and links simultaneously by defining their joint distribution based on hidden Markov random fields ( HMRF ) . Maximizing the data likelihood and the posterior of the model gives the solution to the outlier inference problem . We apply the model on both synthetic data and DBLP data sets , and the results demonstrate importance of this concept , as well as the effectiveness and efficiency of the proposed approach .

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Succinct summarization of transactional databases : an overlapped hyperrectangle scheme Transactional data are ubiquitous . Several methods , including frequent itemsets mining and co-clustering , have been proposed to analyze transactional databases . In this work , we propose a new research problem to succinctly summarize transactional databases . Solving this problem requires linking the high level structure of the database to a potentially huge number of frequent itemsets . We formulate this problem as a set covering problem using overlapped hyperrectangles ; we then prove that this problem and its several variations are NP-hard . We develop an approximation algorithm HYPER which can achieve a ln ( k ) + 1 approximation ratio in polynomial time . We propose a pruning strategy that can significantly speed up the processing of our algorithm . Additionally , we propose an efficient algorithm to further summarize the set of hyperrectangles by allowing false positive conditions . A detailed study using both real and synthetic datasets shows the effectiveness and efficiency of our approaches in summarizing transactional databases .

O B O B I O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O

hyperrectangle
summarization
transactional databases


Mining data records in Web pages A large amount of information on the Web is contained in regularly structured objects , which we call data records . Such data records are important because they often present the essential information of their host pages , e.g. , lists of products or services . It is useful to mine such data records in order to extract information from them to provide value-added services . Existing automatic techniques are not satisfactory because of their poor accuracies . In this paper , we propose a more effective technique to perform the task . The technique is based on two observations about data records on the Web and a string matching algorithm . The proposed technique is able to mine both contiguous and non-contiguous data records . Our experimental results show that the proposed technique outperforms existing techniques substantially .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

database applications
web data records
web information integration
web mining


Frequent-subsequence-based prediction of outer membrane proteins A number of medically important disease-causing bacteria ( collectively called Gram-negative bacteria ) are noted for the extra `` outer '' membrane that surrounds their cell . Proteins resident in this membrane ( outer membrane proteins , or OMPs ) are of primary research interest for antibiotic and vaccine drug design as they are on the surface of the bacteria and so are the most accessible targets to develop new drugs against . With the development of genome sequencing technology and bioinformatics , biologists can now deduce all the proteins that are likely produced in a given bacteria and have attempted to classify where proteins are located in a bacterial cell . However such protein localization programs are currently least accurate when predicting OMPs , and so there is a current need for the development of a better OMP classifier . Data mining research suggests that the use of frequent patterns has good performance in aiding the development of accurate and efficient classification algorithms . In this paper , we present two methods to identify OMPs based on frequent subsequences and test them on all Gram-negative bacterial proteins whose localizations have been determined by biological experiments . One classifier follows an association rule approach , while the other is based on support vector machines ( SVMs ) . We compare the proposed methods with the state-of-the-art methods in the biological domain . The results demonstrate that our methods are better both in terms of accurately identifying OMPs and providing biological insights that increase our understanding of the structures and functions of these important proteins .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

database applications
outer membrane protein
subcellular localization
support vector machine


Incremental quantile estimation for massive tracking

applications
customer profiles
customer relationship management
dynamic database
equi-depth histograms
ewma
massive data
model validation and analysis
percentiles
sequential estimation
simulation output analysis
stochastic approximation
transaction data


Towards exploratory test instance specific algorithms for high dimensional classification In an interactive classification application , a user may find it more valuable to develop a diagnostic decision support method which can reveal significant classification behavior of exemplar records . Such an approach has the additional advantage of being able to optimize the decision process for the individual record in order to design more effective classification methods . In this paper , we propose the Subspace Decision Path method which provides the user with the ability to interactively explore a small number of nodes of a hierarchical decision process so that the most significant classification characteristics for a given test instance are revealed . In addition , the SD-Path method can provide enormous interpretability by constructing views of the data in which the different classes are clearly separated out . Even in cases where the classification behavior of the test instance is ambiguous , the SD-Path method provides a diagnostic understanding of the characteristics which result in this ambiguity . Therefore , this method combines the abilities of the human and the computer in creating an effective diagnostic tool for instance-centered high dimensional classification .

O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

database applications
visual data mining


Mining preferences from superior and inferior examples Mining user preferences plays a critical role in many important applications such as customer relationship management ( CRM ) , product and service recommendation , and marketing campaigns . In this paper , we identify an interesting and practical problem of mining user preferences : in a multidimensional space where the user preferences on some categorical attributes are unknown , from some superior and inferior examples provided by a user , can we learn about the user 's preferences on those categorical attributes ? We model the problem systematically and show that mining user preferences from superior and inferior examples is challenging . Although the problem has great potential in practice , to the best of our knowledge , it has not been explored systematically before . As the first attempt to tackle the problem , we propose a greedy method and show that our method is practical using real data sets and synthetic data sets .

O B O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Algorithms for storytelling We formulate a new data mining problem called it storytelling as a generalization of redescription mining . In traditional redescription mining , we are given a set of objects and a collection of subsets defined over these objects . The goal is to view the set system as a vocabulary and identify two expressions in this vocabulary that induce the same set of objects . Storytelling , on the other hand , aims to explicitly relate object sets that are disjoint ( and hence , maximally dissimilar ) by finding a chain of ( approximate ) redescriptions between the sets . This problem finds applications in bioinformatics , for instance , where the biologist is trying to relate a set of genes expressed in one experiment to another set , implicated in a different pathway . We outline an efficient storytelling implementation that embeds the CART wheels redescription mining algorithm in an A \* search procedure , using the former to supply next move operators on search branches to the latter . This approach is practical and effective for mining large datasets and , at the same time , exploits the structure of partitions imposed by the given vocabulary . Three application case studies are presented : a study of word overlaps in large English dictionaries , exploring connections between genesets in a bioinformatics dataset , and relating publications in the PubMed index of abstracts .

O O B O O O O O O O O O B O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Constructing comprehensive summaries of large event sequences Event sequences capture system and user activity over time . Prior research on sequence mining has mostly focused on discovering local patterns . Though interesting , these patterns reveal local associations and fail to give a comprehensive summary of the entire event sequence . Moreover , the number of patterns discovered can be large . In this paper , we take an alternative approach and build short summaries that describe the entire sequence , while revealing local associations among events . We formally define the summarization problem as an optimization problem that balances between shortness of the summary and accuracy of the data description . We show that this problem can be solved optimally in polynomial time by using a combination of two dynamic-programming algorithms . We also explore more efficient greedy alternatives and demonstrate that they work well on large datasets . Experiments on both synthetic and real datasets illustrate that our algorithms are efficient and produce high-quality results , and reveal interesting local structures in the data .

O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dynamic programming
log mining
minimum description length


Natural communities in large linked networks We are interested in finding natural communities in large-scale linked networks . Our ultimate goal is to track changes over time in such communities . For such temporal tracking , we require a clustering algorithm that is relatively stable under small perturbations of the input data . We have developed an efficient , scalable agglomerative strategy and applied it to the citation graph of the NEC CiteSeer database ( 250,000 papers ; 4.5 million citations ) . Agglomerative clustering techniques are known to be unstable on data in which the community structure is not strong . We find that some communities are essentially random and thus unstable while others are natural and will appear in most clusterings . These natural communities will enable us to track the evolution of communities over time .

B I O B I I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O

large linked networks
natural communities

hierarchical agglomerative clustering
information search and retrieval
stability


Experiments with random projections for machine learning Dimensionality reduction via Random Projections has attracted considerable attention in recent years . The approach has interesting theoretical underpinnings and offers computational advantages . In this paper we report a number of experiments to evaluate Random Projections in the context of inductive supervised learning . In particular , we compare Random Projections and PCA on a number of different datasets and using different machine learning methods . While we find that the random projection approach predictively underperforms PCA , its computational advantages may make it attractive for certain applications .

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O

dimensionality reduction
random projection

pattern recognition
probability and statistics


V-Miner : using enhanced parallel coordinates to mine product design and test data Analyzing data to find trends , correlations , and stable patterns is an important task in many industrial applications . This paper proposes a new technique based on parallel coordinate visualization . Previous work on parallel coordinate methods has shown that they are effective only when variables that are correlated and\/or show similar patterns are displayed adjacently . Although current parallel coordinate tools allow the user to manually rearrange the order of variables , this process is very time-consuming when the number of variables is large . Automated assistance is required . This paper introduces an edit-distance based technique to rearrange variables so that interesting change patterns can be easily detected visually . The Visual Miner ( V-Miner ) software includes both automated methods for visualizing common patterns and a query tool that enables the user to describe specific target patterns to be mined or displayed by the system . In addition , the system can filter data according to rules sets imported from other data mining tools . This feature was found very helpful in practice , because it enables decision makers to visually identify interesting rules and data segments for further analysis or data mining . This paper begins with an introduction to the proposed techniques and the V-Miner system . Next , a case study illustrates how V-Miner has been used at Motorola to guide product design and test decisions .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

change patterns
parallel coordinate visualization
rules


1-dimensional splines as building blocks for improving accuracy of risk outcomes models Transformation of both the response variable and the predictors is commonly used in fitting regression models . However , these transformation methods do not always provide the maximum linear correlation between the response variable and the predictors , especially when there are non-linear relationships between predictors and the response such as the medical data set used in this study . A spline based transformation method is proposed that is second order smooth , continuous , and minimizes the mean squared error between the response and each predictor . Since the computation time for generating this spline is O ( n ) , the processing time is reasonable with massive data sets . In contrast to cubic smoothing splines , the resulting transformation equations also display a high level of efficiency for scoring . Data used for predicting health outcomes contains an abundance of non-linear relationships between predictors and the outcomes requiring an algorithm for modeling them accurately . Thus , a transformation that fits an adaptive cubic spline to each of a set of variables is proposed . These curves are used as a set of transformation functions on the predictors . A case study of how the transformed variables can be fed into a simple linear regression model to predict risk outcomes is presented . The results show significant improvement over the performance of the original variables in both linear and non-linear models .

O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O

data mining
learning
linear model
prediction
variable transformation


Distributed data mining in a chain store database of short transactions In this paper , we broaden the horizon of traditional rule mining by introducing a new framework of causality rule mining in a distributed chain store database . Specifically , the causality rule explored in this paper consists of a sequence of triggering events and a set of consequential events , and is designed with the capability of mining non-sequential , inter-transaction information . Hence , the causality rule mining provides a very general framework for rule derivation . Note , however , that the procedure of causality rule mining is very costly particularly in the presence of a huge number of candidate sets and a distributed database , and in our opinion , can not be dealt with by direct extensions from existing rule mining methods . Consequently , we devise in this paper a series of level matching algorithms , including Level Matching ( abbreviatedly as LM ) , Level Matching with Selective Scan ( abbreviatedly as LMS ) , and Distributed Level Matching ( abbreviatedly as Distibuted LM ) , to minimize the computing cost needed for the distributed data mining of causality rules . In addition , the phenomena of time window constraints are also taken into consideration for the development of our algorithms . As a result of properly employing the technologies of level matching and selective scan , the proposed algorithms present good efficiency and scalability in the mining of local and global causality rules . Scale-up experiments show that the proposed algorithms scale well with the number of sites and the number of customer transactions . Index Terms : knowledge discovery , distributed data mining causality rules , triggering events , consequential events

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

deduction
number-theoretic computations
representations


Adversarial classification Essentially all data mining algorithms assume that the data-generating process is independent of the data miner 's activities . However , in many domains , including spam detection , intrusion detection , fraud detection , surveillance and counter-terrorism , this is far from the case : the data is actively manipulated by an adversary seeking to make the classifier produce false negatives . In these domains , the performance of a classifier can degrade rapidly after it is deployed , as the adversary learns to defeat it . Currently the only solution to this is repeated , manual , ad hoc reconstruction of the classifier . In this paper we develop a formal framework and algorithms for this problem . We view classification as a game between the classifier and the adversary , and produce a classifier that is optimal given the adversary 's optimal strategy . Experiments in a spam detection domain show that this approach can greatly outperform a classifier learned in the standard way , and ( within the parameters of the problem ) automatically adapt the classifier to the adversary 's evolving manipulations .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

cost-sensitive learning
game theory
integer linear programming
naive bayes


GPCA : an efficient dimension reduction scheme for image compression and retrieval Recent years have witnessed a dramatic increase in the quantity of image data collected , due to advances in fields such as medical imaging , reconnaissance , surveillance , astronomy , multimedia etc. . With this increase has come the need to be able to store , transmit , and query large volumes of image data efficiently . A common operation on image databases is the retrieval of all images that are similar to a query image . For this , the images in the database are often represented as vectors in a high-dimensional space and a query is answered by retrieving all image vectors that are proximal to the query image in this space , under a suitable similarity metric . To overcome problems associated with high dimensionality , such as high storage and retrieval times , a dimension reduction step is usually applied to the vectors to concentrate relevant information in a small number of dimensions . Principal Component Analysis ( PCA ) is a well-known dimension reduction scheme . However , since it works with vectorized representations of images , PCA does not take into account the spatial locality of pixels in images . In this paper , a new dimension reduction scheme , called Generalized Principal Component Analysis ( GPCA ) , is presented . This scheme works directly with images in their native state , as two-dimensional matrices , by projecting the images to a vector space that is the tensor product of two lower-dimensional vector spaces . Experiments on databases of face images show that , for the same amount of storage , GPCA is superior to PCA in terms of quality of the compressed images , query precision , and computational cost .

O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dimension reduction
image compression
principal component analysis
tensor product
vector space

information search and retrieval
singular value decomposition


Improved robustness of signature-based near-replica detection via lexicon randomization Detection of near duplicate documents is an important problem in many data mining and information filtering applications . When faced with massive quantities of data , traditional duplicate detection techniques relying on direct inter-document similarity computation ( e.g. , using the cosine measure ) are often not feasible given the time and memory performance constraints . On the other hand , fingerprint-based methods , such as I-Match , are very attractive computationally but may be brittle with respect to small changes to document content . We focus on approaches to near-replica detection that are based upon large-collection statistics and present a general technique of increasing their robustness via multiple lexicon randomization . In experiments with large web-page and spam-email datasets the proposed method is shown to consistently outperform traditional I-Match , with the relative improvement in duplicate-document recall reaching as high as 40-60 % . The large gains in detection accuracy are offset by only small increases in computational requirements .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data cleaning
deduplication
spam filtering
web mining


Evolutionary clustering We consider the problem of clustering data over time . An evolutionary clustering should simultaneously optimize two potentially conflicting criteria : first , the clustering at any point in time should remain faithful to the current data as much as possible ; and second , the clustering should not shift dramatically from one timestep to the next . We present a generic framework for this problem , and discuss evolutionary versions of two widely-used clustering algorithms within this framework : k-means and agglomerative hierarchical clustering . We extensively evaluate these algorithms on real data sets and show that our algorithms can simultaneously attain both high accuracy in capturing today 's data , and high fidelity in reflecting yesterday 's clustering .

O B O O O O O B O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

information search and retrieval
k-means
temporal evolution


An iterative method for multi-class cost-sensitive learning Cost-sensitive learning addresses the issue of classification in the presence of varying costs associated with different types of misclassification . In this paper , we present a method for solving multi-class cost-sensitive learning problems using any binary classification algorithm . This algorithm is derived using hree key ideas : 1 ) iterative weighting ; 2 ) expanding data space ; and 3 ) gradient boosting with stochastic ensembles . We establish some theoretical guarantees concerning the performance of this method . In particular , we show that a certain variant possesses the boosting property , given a form of weak learning assumption on the component binary classifier . We also empirically evaluate the performance of the proposed method using benchmark data sets and verify that our method generally achieves better results than representative methods for cost-sensitive learning , in terms of predictive performance ( cost minimization ) and , in many cases , computational efficiency .

O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O

cost-sensitive learning
multi-class classification


Dimension induced clustering It is commonly assumed that high-dimensional datasets contain points most of which are located in low-dimensional manifolds . Detection of low-dimensional clusters is an extremely useful task for performing operations such as clustering and classification , however , it is a challenging computational problem . In this paper we study the problem of finding subsets of points with low intrinsic dimensionality . Our main contribution is to extend the definition of fractal correlation dimension , which measures average volume growth rate , in order to estimate the intrinsic dimensionality of the data in local neighborhoods . We provide a careful analysis of several key examples in order to demonstrate the properties of our measure . Based on our proposed measure , we introduce a novel approach to discover clusters with low dimensionality . The resulting algorithms extend previous density based measures , which have been successfully used for clustering . We demonstrate the effectiveness of our algorithms for discovering low-dimensional m-flats embedded in high dimensional spaces , and for detecting low-rank sub-matrices .

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O


Efficiently learning the accuracy of labeling sources for selective sampling Many scalable data mining tasks rely on active learning to provide the most useful accurately labeled instances . However , what if there are multiple labeling sources ( ` oracles ' or ` experts ' ) with different but unknown reliabilities ? With the recent advent of inexpensive and scalable online annotation tools , such as Amazon 's Mechanical Turk , the labeling process has become more vulnerable to noise - and without prior knowledge of the accuracy of each individual labeler . This paper addresses exactly such a challenge : how to jointly learn the accuracy of labeling sources and obtain the most informative labels for the active learning task at hand minimizing total labeling effort . More specifically , we present IEThresh ( Interval Estimate Threshold ) as a strategy to intelligently select the expert ( s ) with the highest estimated labeling accuracy . IEThresh estimates a confidence interval for the reliability of each expert and filters out the one ( s ) whose estimated upper-bound confidence interval is below a threshold - which jointly optimizes expected accuracy ( mean ) and need to better estimate the expert 's accuracy ( variance ) . Our framework is flexible enough to work with a wide range of different noise levels and outperforms baselines such as asking all available experts and random expert selection . In particular , IEThresh achieves a given level of accuracy with less than half the queries issued by all-experts labeling and less than a third the queries required by random expert selection on datasets such as the UCI mushroom one . The results show that our method naturally balances exploration and exploitation as it gains knowledge of which experts to rely upon , and selects them with increasing frequency .

O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

estimation
labeler selection
noisy labelers


Anomaly pattern detection in categorical datasets We propose a new method for detecting patterns of anomalies in categorical datasets . We assume that anomalies are generated by some underlying process which affects only a particular subset of the data . Our method consists of two steps : we first use a `` local anomaly detector '' to identify individual records with anomalous attribute values , and then detect patterns where the number of anomalous records is higher than expected . Given the set of anomalies flagged by the local anomaly detector , we search over all subsets of the data defined by any set of fixed values of a subset of the attributes , in order to detect self-similar patterns of anomalies . We wish to detect any such subset of the test data which displays a significant increase in anomalous activity as compared to the normal behavior of the system ( as indicated by the training data ) . We perform significance testing to determine if the number of anomalies in any subset of the test data is significantly higher than expected , and propose an efficient algorithm to perform this test over all such subsets of the data . We show that this algorithm is able to accurately detect anomalous patterns in real-world hospital , container shipping and network intrusion data .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Probabilistic modeling of transaction data with applications to profiling , visualization , and prediction Transaction data is ubiquitous in data mining applications . Examples include market basket data in retail commerce , telephone call records in telecommunications , and Web logs of individual page-requests at Web sites . Profiling consists of using historical transaction data on individuals to construct a model of each individual 's behavior . Simple profiling techniques such as histograms do not generalize well from sparse transaction data . In this paper we investigate the application of probabilistic mixture models to automatically generate profiles from large volumes of transaction data . In effect , the mixture model represents each individual 's behavior as a linear combination of `` basis transactions . '' We evaluate several variations of the model on a large retail transaction data set and show that the proposed model provides improved predictive power over simpler histogram-based techniques , as well as being relatively scalable , interpretable , and flexible . In addition we point to applications in outlier detection , customer ranking , interactive visualization , and so forth . The paper concludes by comparing and relating the proposed framework to other transaction-data modeling techniques such as association rules .

O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

mixture models
profiles
transaction data

electronic commerce
em algorithm
types of simulation


Mining advisor-advisee relationships from research publication networks Information network contains abundant knowledge about relationships among people or entities . Unfortunately , such kind of knowledge is often hidden in a network where different kinds of relationships are not explicitly categorized . For example , in a research publication network , the advisor-advisee relationships among researchers are hidden in the coauthor network . Discovery of those relationships can benefit many interesting applications such as expert finding and research community analysis . In this paper , we take a computer science bibliographic network as an example , to analyze the roles of authors and to discover the likely advisor-advisee relationships . In particular , we propose a time-constrained probabilistic factor graph model ( TPFG ) , which takes a research publication network as input and models the advisor-advisee relationship mining problem using a jointly likelihood objective function . We further design an efficient learning algorithm to optimize the objective function . Based on that our model suggests and ranks probable advisors for every author . Experimental results show that the proposed approach infer advisor-advisee relationships efficiently and achieves a state-of-the-art accuracy ( 80-90 % ) . We also apply the discovered advisor-advisee relationships to bole search , a specific expert finding task and empirical study shows that the search performance can be effectively improved ( +4.09 % by NDCG@5 ) .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

advisor-advisee prediction
time-constrained factor graph


MARK : a boosting algorithm for heterogeneous kernel models Support Vector Machines and other kernel methods have proven to be very effective for nonlinear inference . Practical issues are how to select the type of kernel including any parameters and how to deal with the computational issues caused by the fact that the kernel matrix grows quadratically with the data . Inspired by ensemble and boosting methods like MART , we propose the Multiple Additive Regression Kernels ( MARK ) algorithm to address these issues . MARK considers a large ( potentially infinite ) library of kernel matrices formed by different kernel functions and parameters . Using gradient boosting\/column generation , MARK constructs columns of the heterogeneous kernel matrix ( the base hypotheses ) on the fly and then adds them into the kernel ensemble . Regularization methods such as used in SVM , kernel ridge regression , and MART , are used to prevent overfitting . We investigate how MARK is applied to heterogeneous kernel ridge regression . The resulting algorithm is simple to implement and efficient . Kernel parameter selection is handled within MARK . Sampling and `` weak '' kernels are used to further enhance the computational efficiency of the resulting additive algorithm . The user can incorporate and potentially extract domain knowledge by restricting the kernel library to interpretable kernels . MARK compares very favorably with SVM and kernel ridge regression on several benchmark datasets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Scalable mining of large disk-based graph databases Mining frequent structural patterns from graph databases is an interesting problem with broad applications . Most of the previous studies focus on pruning unfruitful search subspaces effectively , but few of them address the mining on large , disk-based databases . As many graph databases in applications can not be held into main memory , scalable mining of large , disk-based graph databases remains a challenging problem . In this paper , we develop an effective index structure , ADI ( for ( u ) ad ( \/ u ) jacency ( u ) i ( \/ u ) ndex ) , to support mining various graph patterns over large databases that can not be held into main memory . The index is simple and efficient to build . Moreover , the new index structure can be easily adopted in various existing graph pattern mining algorithms . As an example , we adapt the well-known gSpan algorithm by using the ADI structure . The experimental results show that the new index structure enables the scalable graph pattern mining over large databases . In one set of the experiments , the new disk-based method can mine graph databases with one million graphs , while the original gSpan algorithm can only handle databases of up to 300 thousand graphs . Moreover , our new method is faster than gSpan when both can run in main memory .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

frequent graph pattern
graph database
graph mining


Information extraction from Wikipedia : moving down the long tail Not only is Wikipedia a comprehensive source of quality information , it has several kinds of internal structure ( e.g. , relational summaries known as infoboxes ) , which enable self-supervised information extraction . While previous efforts at extraction from Wikipedia achieve high precision and recall on well-populated classes of articles , they fail in a larger number of cases , largely because incomplete articles and infrequent use of infoboxes lead to insufficient training data . This paper presents three novel techniques for increasing recall from Wikipedia 's long tail of sparse classes : ( 1 ) shrinkage over an automatically-learned subsumption taxonomy , ( 2 ) a retraining technique for improving the training data , and ( 3 ) supplementing results by extracting from the broader Web . Our experiments compare design variations and show that , used in concert , these techniques increase recall by a factor of 1.76 to 8.71 while maintaining or increasing precision .

B I O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

information systems applications
semantic web


Consistent bipartite graph co-partitioning for star-structured high-order heterogeneous data co-clustering Heterogeneous data co-clustering has attracted more and more attention in recent years due to its high impact on various applications . While the co-clustering algorithms for two types of heterogeneous data ( denoted by pair-wise co-clustering ) , such as documents and terms , have been well studied in the literature , the work on more types of heterogeneous data ( denoted by high-order co-clustering ) is still very limited . As an attempt in this direction , in this paper , we worked on a specific case of high-order co-clustering in which there is a central type of objects that connects the other types so as to form a star structure of the inter-relationships . Actually , this case could be a very good abstract for many real-world applications , such as the co-clustering of categories , documents and terms in text mining . In our philosophy , we treated such kind of problems as the fusion of multiple pair-wise co-clustering sub-problems with the constraint of the star structure . Accordingly , we proposed the concept of consistent bipartite graph co-partitioning , and developed an algorithm based on semi-definite programming ( SDP ) for efficient computation of the clustering results . Experiments on toy problems and real data both verified the effectiveness of our proposed method .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

co-clustering
consistency
high-order heterogeneous data
spectral graph


Mining quantitative correlated patterns using an information-theoretic approach Existing research on mining quantitative databases mainly focuses on mining associations . However , mining associations is too expensive to be practical in many cases . In this paper , we study mining correlations from quantitative databases and show that it is a more effective approach than mining associations . We propose a new notion of Quantitative Correlated Patterns ( QCPs ) , which is founded on two formal concepts , mutual information and all-confidence . We first devise a normalization on mutual information and apply it to QCP mining to capture the dependency between the attributes . We further adopt all-confidence as a quality measure to control , at a finer granularity , the dependency between the attributes with specific quantitative intervals . We also propose a supervised method to combine the consecutive intervals of the quantitative attributes based on mutual information , such that the interval combining is guided by the dependency between the attributes . We develop an algorithm , QCoMine , to efficiently mine QCPs by utilizing normalized mutual information and all-confidence to perform a two-level pruning . Our experiments verify the efficiency of QCoMine and the quality of the QCPs .

O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O

correlated patterns
mutual information
quantitative databases


The UCI KDD archive of large data sets for data mining research and experimentation


Combinational collaborative filtering for personalized community recommendation Rapid growth in the amount of data available on social networking sites has made information retrieval increasingly challenging for users . In this paper , we propose a collaborative filtering method , Combinational Collaborative Filtering ( CCF ) , to perform personalized community recommendations by considering multiple types of co-occurrences in social data at the same time . This filtering method fuses semantic and user information , then applies a hybrid training strategy that combines Gibbs sampling and Expectation-Maximization algorithm . To handle the large-scale dataset , parallel computing is used to speed up the model training . Through an empirical study on the Orkut dataset , we show CCF to be both effective and scalable .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

miscellaneous
personalized recommendation
probabilistic models


Beyond heuristics : learning to classify vulnerabilities and predict exploits The security demands on modern system administration are enormous and getting worse . Chief among these demands , administrators must monitor the continual ongoing disclosure of software vulnerabilities that have the potential to compromise their systems in some way . Such vulnerabilities include buffer overflow errors , improperly validated inputs , and other unanticipated attack modalities . In 2008 , over 7,400 new vulnerabilities were disclosed -- well over 100 per week . While no enterprise is affected by all of these disclosures , administrators commonly face many outstanding vulnerabilities across the software systems they manage . Vulnerabilities can be addressed by patches , reconfigurations , and other workarounds ; however , these actions may incur down-time or unforeseen side-effects . Thus , a key question for systems administrators is which vulnerabilities to prioritize . From publicly available databases that document past vulnerabilities , we show how to train classifiers that predict whether and how soon a vulnerability is likely to be exploited . As input , our classifiers operate on high dimensional feature vectors that we extract from the text fields , time stamps , cross references , and other entries in existing vulnerability disclosure reports . Compared to current industry-standard heuristics based on expert knowledge and static formulas , our classifiers predict much more accurately whether and how soon individual vulnerabilities are likely to be exploited .

O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O

security and protection
supervised learning
svm


Canonicalization of database records using adaptive similarity measures It is becoming increasingly common to construct databases from information automatically culled from many heterogeneous sources . For example , a research publication database can be constructed by automatically extracting titles , authors , and conference information from online papers . A common difficulty in consolidating data from multiple sources is that records are referenced in a variety of ways ( e.g. abbreviations , aliases , and misspellings ) . Therefore , it can be difficult to construct a single , standard representation to present to the user . We refer to the task of constructing this representation as canonicalization . Despite its importance , there is little existing work on canonicalization . In this paper , we explore the use of edit distance measures to construct a canonical representation that is `` central '' in the sense that it is most similar to each of the disparate records . This approach reduces the impact of noisy records on the canonical representation . Furthermore , because the user may prefer different styles of canonicalization , we show how different edit distance costs can result in different forms of canonicalization . For example , reducing the cost of character deletions can result in representations that favor abbreviated forms over expanded forms ( e.g. KDD versus Conference on Knowledge Discovery and Data Mining ) . We describe how to learn these costs from a small amount of manually annotated data using stochastic hill-climbing . Additionally , we investigate feature-based methods to learn ranking preferences over canonicalizations . These approaches can incorporate arbitrary textual evidence to select a canonical record . We evaluate our approach on a real-world publications database and show that our learning method results in a canonicalization solution that is robust to errors and easily customizable to user preferences .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On the privacy of anonymized networks The proliferation of online social networks , and the concomitant accumulation of user data , give rise to hotly debated issues of privacy , security , and control . One specific challenge is the sharing or public release of anonymized data without accidentally leaking personally identifiable information ( PII ) . Unfortunately , it is often difficult to ascertain that sophisticated statistical techniques , potentially employing additional external data sources , are unable to break anonymity . In this paper , we consider an instance of this problem , where the object of interest is the structure of a social network , i.e. , a graph describing users and their links . Recent work demonstrates that anonymizing node identities may not be sufficient to keep the network private : the availability of node and link data from another domain , which is correlated with the anonymized network , has been used to re-identify the anonymized nodes . This paper is about conditions under which such a de-anonymization process is possible . We attempt to shed light on the following question : can we assume that a sufficiently sparse network is inherently anonymous , in the sense that even with unlimited computational power , de-anonymization is impossible ? Our approach is to introduce a random graph model for a version of the de-anonymization problem , which is parameterized by the expected node degree and a similarity parameter that controls the correlation between two graphs over the same vertex set . We find simple conditions on these parameters delineating the boundary of privacy , and show that the mean node degree need only grow slightly faster than log n with network size n for nodes to be identifiable . Our results have policy implications for sharing of anonymized network information .

O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

de-anonymization
general
graph sampling
network privacy
random graphs


Predicting customer shopping lists from point-of-sale purchase data This paper describes a prototype that predicts the shopping lists for customers in a retail store . The shopping list prediction is one aspect of a larger system we have developed for retailers to provide individual and personalized interactions with customers as they navigate through the retail store . Instead of using traditional personalization approaches , such as clustering or segmentation , we learn separate classifiers for each customer from historical transactional data . This allows us to make very fine-grained and accurate predictions about what items a particular individual customer will buy on a given shopping trip . We formally frame the shopping list prediction as a classification problem , describe the algorithms and methodology behind our system , its impact on the business case in which we frame it , and explore some of the properties of the data source that make it an interesting testbed for KDD algorithms . Our results show that we can predict a shopper 's shopping list with high levels of accuracy , precision , and recall . We believe that this work impacts both the data mining and the retail business community . The formulation of shopping list prediction as a machine learning problem results in algorithms that should be useful beyond retail shopping list prediction . For retailers , the result is not only a practical system that increases revenues by up to 11 % , but also enhances customer experience and loyalty by giving them the tools to individually interact with customers and anticipate their needs .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Mining progressive confident rules Many real world objects have states that change over time . By tracking the state sequences of these objects , we can study their behavior and take preventive measures before they reach some undesirable states . In this paper , we propose a new kind of pattern called progressive confident rules to describe sequences of states with an increasing confidence that lead to a particular end state . We give a formal definition of progressive confident rules and their concise set . We devise pruning strategies to reduce the enormous search space . Experiment result shows that the proposed algorithm is efficient and scalable . We also demonstrate the application of progressive confident rules in classification .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O


Evolutionary algorithms in data mining : multi-objective performance modeling for direct marketing

database marketing
evolutionary computation
multiple objectives
pareto-optimal models


A microeconomic data mining problem : customer-oriented catalog segmentation The microeconomic framework for data mining ( 7 ) assumes that an enterprise chooses a decision maximizing the overall utility over all customers where the contribution of a customer is a function of the data available on that customer . In Catalog Segmentation , the enterprise wants to design k product catalogs of size r that maximize the overall number of catalog products purchased . However , there are many applications where a customer , once attracted to an enterprise , would purchase more products beyond the ones contained in the catalog . Therefore , in this paper , we investigate an alternative problem formulation , that we call Customer-Oriented Catalog Segmentation , where the overall utility is measured by the number of customers that have at least a specified minimum interest t in the catalogs . We formally introduce the Customer-Oriented Catalog Segmentation problem and discuss its complexity . Then we investigate two different paradigms to design efficient , approximate algorithms for the Customer-Oriented Catalog Segmentation problem , greedy ( deterministic ) and randomized algorithms . Since greedy algorithms may be trapped in a local optimum and randomized algorithms crucially depend on a reasonable initial solution , we explore a combination of these two paradigms . Our experimental evaluation on synthetic and real data demonstrates that the new algorithms yield catalogs of significantly higher utility compared to classical Catalog Segmentation algorithms .

O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O

catalog segmentation
microeconomic data mining


Density-based clustering for real-time stream data Existing data-stream clustering algorithms such as CluStream arebased on k-means . These clustering algorithms are incompetent tofind clusters of arbitrary shapes and can not handle outliers . Further , they require the knowledge of k and user-specified time window . To address these issues , this paper proposes D-Stream , a framework for clustering stream data using adensity-based approach . The algorithm uses an online component which maps each input data record into a grid and an offline component which computes the grid density and clusters the grids based on the density . The algorithm adopts a density decaying technique to capture the dynamic changes of a data stream . Exploiting the intricate relationships between the decay factor , data density and cluster structure , our algorithm can efficiently and effectively generate and adjust the clusters in real time . Further , a theoretically sound technique is developed to detect and remove sporadic grids mapped to by outliers in order to dramatically improve the space and time efficiency of the system . The technique makes high-speed data stream clustering feasible without degrading the clustering quality . The experimental results show that our algorithm has superior quality and efficiency , can find clusters of arbitrary shapes , and can accurately recognize the evolving behaviors of real-time data streams .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

d-stream
density-based clustering
stream data mining


Data selection for support vector machine classifiers

concave minimization
data classification
support vector machines


Scalable robust covariance and correlation estimates for data mining Covariance and correlation estimates have important applications in data mining . In the presence of outliers , classical estimates of covariance and correlation matrices are not reliable . A small fraction of outliers , in some cases even a single outlier , can distort the classical covariance and correlation estimates making them virtually useless . That is , correlations for the vast majority of the data can be very erroneously reported ; principal components transformations can be misleading ; and multidimensional outlier detection via Mahalanobis distances can fail to detect outliers . There is plenty of statistical literature on robust covariance and correlation matrix estimates with an emphasis on affine-equivariant estimators that possess high breakdown points and small worst case biases . All such estimators have unacceptable exponential complexity in the number of variables and quadratic complexity in the number of observations . In this paper we focus on several variants of robust covariance and correlation matrix estimates with quadratic complexity in the number of variables and linear complexity in the number of observations . These estimators are based on several forms of pairwise robust covariance and correlation estimates . The estimators studied include two fast estimators based on coordinate-wise robust transformations embedded in an overall procedure recently proposed by ( 14 ) . We show that the estimators have attractive robustness properties , and give an example that uses one of the estimators in the new Insightful Miner data mining product .

O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

robust estimators
robust statistics
scalable algorithm


A family of dissimilarity measures between nodes generalizing both the shortest-path and the commute-time distances This work introduces a new family of link-based dissimilarity measures between nodes of a weighted directed graph . This measure , called the randomized shortest-path ( RSP ) dissimilarity , depends on a parameter θ and has the interesting property of reducing , on one end , to the standard shortest-path distance when θ is large and , on the other end , to the commute-time ( or resistance ) distance when θ is small ( near zero ) . Intuitively , it corresponds to the expected cost incurred by a random walker in order to reach a destination node from a starting node while maintaining a constant entropy ( related to θ ) spread in the graph . The parameter θ is therefore biasing gradually the simple random walk on the graph towards the shortest-path policy . By adopting a statistical physics approach and computing a sum over all the possible paths ( discrete path integral ) , it is shown that the RSP dissimilarity from every node to a particular node of interest can be computed efficiently by solving two linear systems of n equations , where n is the number of nodes . On the other hand , the dissimilarity between every couple of nodes is obtained by inverting an n x n matrix . The proposed measure can be used for various graph mining tasks such as computing betweenness centrality , finding dense communities , etc , as shown in the experimental section .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O

biased random walk
commute-time distance
kernel on a graph
resistance distance
shortest path


Aggregating time partitions Partitions of sequential data exist either per se or as a result of sequence segmentation algorithms . It is often the case that the same timeline is partitioned in many different ways . For example , different segmentation algorithms produce different partitions of the same underlying data points . In such cases , we are interested in producing an aggregate partition , i.e. , a segmentation that agrees as much as possible with the input segmentations . Each partition is defined as a set of continuous non-overlapping segments of the timeline . We show that this problem can be solved optimally in polynomial time using dynamic programming . We also propose faster greedy heuristics that work well in practice . We experiment with our algorithms and we demonstrate their utility in clustering the behavior of mobile-phone users and combining the results of different segmentation algorithms on genomic sequences .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A general framework for accurate and fast regression by data summarization in random decision trees Predicting the values of continuous variable as a function of several independent variables is one of the most important problems for data mining . A very large number of regression methods , both parametric and nonparametric , have been proposed in the past . However , since the list is quite extensive and many of these models make rather explicit , strong yet different assumptions about the type of applicable problems and involve a lot of parameters and options , choosing the appropriate regression methodology and then specifying the parameter values is a none-trivial , sometimes frustrating , task for data mining practitioners . Choosing the inappropriate methodology can have rather disappointing results . This issue is against the general utility of data mining software . For example , linear regression methods are straightforward and well-understood . However , since the linear assumption is very strong , its performance is compromised for complicated non-linear problems . Kernel-based methods perform quite well if the kernel functions are selected correctly . In this paper , we propose a straightforward approach based on summarizing the training data using an ensemble of random decisions trees . It requires very little knowledge from the user , yet is applicable to every type of regression problem that we are currently aware of . We have experimented on a wide range of problems including those that parametric methods performwell , a large selection of benchmark datasets for nonparametric regression , as well as highly non-linear stochastic problems . Our results are either significantly better than or identical to many approaches that are known to perform well on these problems .

O O O O O O O B O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fast coordinate descent methods with variable selection for non-negative matrix factorization Nonnegative Matrix Factorization ( NMF ) is an effective dimension reduction method for non-negative dyadic data , and has proven to be useful in many areas , such as text mining , bioinformatics and image processing . NMF is usually formulated as a constrained non-convex optimization problem , and many algorithms have been developed for solving it . Recently , a coordinate descent method , called FastHals , has been proposed to solve least squares NMF and is regarded as one of the state-of-the-art techniques for the problem . In this paper , we first show that FastHals has an inefficiency in that it uses a cyclic coordinate descent scheme and thus , performs unneeded descent steps on unimportant variables . We then present a variable selection scheme that uses the gradient of the objective function to arrive at a new coordinate descent method . Our new method is considerably faster in practice and we show that it has theoretical convergence guarantees . Moreover when the solution is sparse , as is often the case in real applications , our new method benefits by selecting important variables to update more often , thus resulting in higher speed . As an example , on a text dataset RCV1 , our method is 7 times faster than FastHals , and more than 15 times faster when the sparsity is increased by adding an L1 penalty . We also develop new coordinate descent methods when error in NMF is measured by KL-divergence by applying the Newton method to solve the one-variable sub-problems . Experiments indicate that our algorithm for minimizing the KL-divergence is faster than the Lee & Seung multiplicative rule by a factor of 10 on the CBCL image dataset .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

learning
non-negative matrix factorization