Temporal causal modeling with graphical granger methods The need for mining causality , beyond mere statistical correlations , for real world problems has been recognized widely . Many of these applications naturally involve temporal data , which raises the challenge of how best to leverage the temporal information for causal modeling . Recently graphical modeling with the concept of `` Granger causality '' , based on the intuition that a cause helps predict its effects in the future , has gained attention in many domains involving time series data analysis . With the surge of interest in model selection methodologies for regression , such as the Lasso , as practical alternatives to solving structural learning of graphical models , the question arises whether and how to combine these two notions into a practically viable approach for temporal causal modeling . In this paper , we examine a host of related algorithms that , loosely speaking , fall under the category of graphical Granger methods , and characterize their relative performance from multiple viewpoints . Our experiments show , for instance , that the Lasso algorithm exhibits consistent gain over the canonical pairwise graphical Granger method . We also characterize conditions under which these variants of graphical Granger methods perform well in comparison to other benchmark methods . Finally , we apply these methods to a real world data set involving key performance indicators of corporations , and present some concrete results .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

causal modeling
graphical models
time series data


Feature selection methods for text classification We consider feature selection for text classification both theoretically and empirically . Our main result is an unsupervised feature selection strategy for which we give worst-case theoretical guarantees on the generalization power of the resultant classification function f with respect to the classification function f obtained when keeping all the features . To the best of our knowledge , this is the first feature selection method with such guarantees . In addition , the analysis leads to insights as to when and why this feature selection strategy will perform well in practice . We then use the TechTC-100 , 20-Newsgroups , and Reuters-RCV2 data sets to evaluate empirically the performance of this and two simpler but related feature selection strategies against two commonly-used strategies . Our empirical evaluation shows that the strategy with provable performance guarantees performs well in comparison with other commonly-used feature selection strategies . In addition , it performs better on certain datasets under very aggressive feature selection .

B I O O B I O O B I O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O

regularized least squares classification


Privacy-preserving Bayesian network structure computation on distributed heterogeneous data As more and more activities are carried out using computers and computer networks , the amount of potentially sensitive data stored by business , governments , and other parties increases . Different parties may wish to benefit from cooperative use of their data , but privacy regulations and other privacy concerns may prevent the parties from sharing their data . Privacy-preserving data mining provides a solution by creating distributed data mining algorithms in which the underlying data is not revealed . In this paper , we present a privacy-preserving protocol for a particular data mining task : learning the Bayesian network structure for distributed heterogeneous data . In this setting , two parties owning confidential databases wish to learn the structure of Bayesian network on the combination of their databases without revealing anything about their data to each other . We give an efficient and privacy-preserving version of the K2 algorithm to construct the structure of a Bayesian network for the parties ' joint data .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O

distributed databases
privacy-preserving data mining


Interestingness of frequent itemsets using Bayesian networks as background knowledge The paper presents a method for pruning frequent itemsets based on background knowledge represented by a Bayesian network . The interestingness of an itemset is defined as the absolute difference between its support estimated from data and from the Bayesian network . Efficient algorithms are presented for finding interestingness of a collection of frequent itemsets , and for finding all attribute sets with a given minimum interestingness . Practical usefulness of the algorithms and their efficiency have been verified experimentally .

B O O O O O O O B B O O O O O O O O O O O B B O O O B I O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O

background
bayesian network
interestingness
knowledge


On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms


Creating social networks to improve peer-to-peer networking We use knowledge discovery techniques to guide the creation of efficient overlay networks for peer-to-peer file sharing . An overlay network specifies the logical connections among peers in a network and is distinct from the physical connections of the network . It determines the order in which peers will be queried when a user is searching for a specific file . To better understand the role of the network overlay structure in the performance of peer-to-peer file sharing protocols , we compare several methods for creating overlay networks . We analyze the networks using data from a campus network for peer-to-peer file sharing that recorded anonymized data on 6,528 users sharing 291,925 music files over an 81-day period . We propose a novel protocol for overlay creation based on a model of user preference identified by latent-variable clustering with hierarchical Dirichlet processes ( HDPs ) . Our simulations and empirical studies show that the clusters of songs created by HDPs effectively model user behavior and can be used to create desirable network overlays that outperform alternative approaches .

O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

hierarchical dirichlet processes
overlay networks
social networks

distributed hash tables
learning
peer-to-peer networks


Exploiting place features in link prediction on location-based social networks Link prediction systems have been largely adopted to recommend new friends in online social networks using data about social interactions . With the soaring adoption of location-based social services it becomes possible to take advantage of an additional source of information : the places people visit . In this paper we study the problem of designing a link prediction system for online location-based social networks . We have gathered extensive data about one of these services , Gowalla , with periodic snapshots to capture its temporal evolution . We study the link prediction space , finding that about 30 % of new links are added among `` place-friends '' , i.e. , among users who visit the same places . We show how this prediction space can be made 15 times smaller , while still 66 % of future connections can be discovered . Thus , we define new prediction features based on the properties of the places visited by users which are able to discriminate potential future links among them . Building on these findings , we describe a supervised learning framework which exploits these prediction features to predict new links among friends-of-friends and place-friends . Our evaluation shows how the inclusion of information about places and related user activity offers high link prediction performance . These results open new directions for real-world link recommendation systems on location-based social networks .

O O O O B I O O B I B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O

database applications
location-based services


Eliminating noisy information in Web pages for data mining A commercial Web page typically contains many information blocks . Apart from the main content blocks , it usually has such blocks as navigation panels , copyright and privacy notices , and advertisements ( for business purposes and for easy user access ) . We call these blocks that are not the main content blocks of the page the noisy blocks . We show that the information contained in these noisy blocks can seriously harm Web data mining . Eliminating these noises is thus of great importance . In this paper , we propose a noise elimination technique based on the following observation : In a given Web site , noisy blocks usually share some common contents and presentation styles , while the main content blocks of the pages are often diverse in their actual contents and\/or presentation styles . Based on this observation , we propose a tree structure , called Style Tree , to capture the common presentation styles and the actual contents of the pages in a given Web site . By sampling the pages of the site , a Style Tree can be built for the site , which we call the Site Style Tree ( SST ) . We then introduce an information based measure to determine which parts of the SST represent noises and which parts represent the main contents of the site . The SST is employed to detect and eliminate noises in any Web page of the site by mapping this page to the SST . The proposed technique is evaluated with two data mining tasks , Web page clustering and classification . Experimental results show that our noise elimination technique is able to improve the mining results significantly .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O


New EM derived from Kullback-Leibler divergence We introduce a new EM framework in which it is possible not only to optimize the model parameters but also the number of model components . A key feature of our approach is that we use nonparametric density estimation to improve parametric density estimation in the EM framework . While the classical EM algorithm estimates model parameters empirically using the data points themselves , we estimate them using nonparametric density estimates . There exist many possible applications that require optimal adjustment of model components . We present experimental results in two domains . One is polygonal approximation of laser range data , which is an active research topic in robot navigation . The other is grouping of edge pixels to contour boundaries , which still belongs to unsolved problems in computer vision .

O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

expectation maximization
general
kullback-leibler divergence


Mining coherent gene clusters from gene-sample-time microarray data Extensive studies have shown that mining microarray data sets is important in bioinformatics research and biomedical applications . In this paper , we explore a novel type of gene-sample-time microarray data sets , which records the expression levels of various genes under a set of samples during a series of time points . In particular , we propose the mining of coherent gene clusters from such data sets . Each cluster contains a subset of genes and a subset of samples such that the genes are coherent on the samples along the time series . The coherent gene clusters may identify the samples corresponding to some phenotypes ( e.g. , diseases ) , and suggest the candidate genes correlated to the phenotypes . We present two efficient algorithms , namely the Sample-Gene Search and the Gene-Sample Search , to mine the complete set of coherent gene clusters . We empirically evaluate the performance of our approaches on both a real microarray data set and synthetic data sets . The test results have shown that our approaches are both efficient and effective to find meaningful coherent gene clusters .

O O O O O O B I O O O O O O B I O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O


MIME : a framework for interactive visual pattern mining We present a framework for interactive visual pattern mining . Our system enables the user to browse through the data and patterns easily and intuitively , using a toolbox consisting of interestingness measures , mining algorithms and post-processing algorithms to assist in identifying interesting patterns . By mining interactively , we enable the user to combine their subjective interestingness measure and background knowledge with a wide variety of objective measures to easily and quickly mine the most important and interesting patterns . Basically , we enable the user to become an essential part of the mining algorithm . Our demo currently applies to mining interesting itemsets and association rules , and its extension to episodes and decision trees is ongoing .

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

interactive visual mining
pattern exploration


Unsupervised feature selection for multi-cluster data In many data analysis tasks , one is often confronted with very high dimensional data . Feature selection techniques are designed to find the relevant feature subset of the original features which can facilitate clustering , classification and retrieval . In this paper , we consider the feature selection problem in unsupervised learning scenario , which is particularly difficult due to the absence of class labels that would guide the search for relevant information . The feature selection problem is essentially a combinatorial optimization problem which is computationally expensive . Traditional unsupervised feature selection methods address this issue by selecting the top ranked features based on certain scores computed independently for each feature . These approaches neglect the possible correlation between different features and thus can not produce an optimal feature subset . Inspired from the recent developments on manifold learning and L1-regularized models for subset selection , we propose in this paper a new approach , called Multi-Cluster Feature Selection ( MCFS ) , for unsupervised feature selection . Specifically , we select those features such that the multi-cluster structure of the data can be best preserved . The corresponding optimization problem can be efficiently solved since it only involves a sparse eigen-problem and a L1-regularized least squares problem . Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithm .

B B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

clustering
feature selection
unsupervised


IMDS : intelligent malware detection system The proliferation of malware has presented a serious threat to the security of computer systems . Traditional signature-based anti-virus systems fail to detect polymorphic and new , previously unseen malicious executables . In this paper , resting on the analysis of Windows API execution sequences called by PE files , we develop the Intelligent Malware Detection System ( IMDS ) using Objective-Oriented Association ( OOA ) mining based classification . IMDS is an integrated system consisting of three major modules : PE parser , OOA rule generator , and rule based classifier . An OOA algorithm is adapted to efficiently generate OOA rules for classification . A comprehensive experimental study on a large collection of PE files obtained from the anti-virus laboratory of King-Soft Corporation is performed to compare various malware detection approaches . Promising experimental results demonstrate that the accuracy and efficiency of our IMDS system out perform popular anti-virus software such as Norton AntiVirus and McAfee VirusScan , as well as previous data mining based detection systems which employed Naive Bayes , Support Vector Machine ( SVM ) and Decision Tree techniques .

O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

learning
ooa mining
pe file
windows api sequence


Naïve filterbots for robust cold-start recommendations The goal of a recommender system is to suggest items of interest to a user based on historical behavior of a community of users . Given detailed enough history , item-based collaborative filtering ( CF ) often performs as well or better than almost any other recommendation method . However , in cold-start situations - where a user , an item , or the entire system is new - simple non-personalized recommendations often fare better . We improve the scalability and performance of a previous approach to handling cold-start situations that uses filterbots , or surrogate users that rate items based only on user or item attributes . We show that introducing a very small number of simple filterbots helps make CF algorithms more robust . In particular , adding just seven global filterbots improves both user-based and item-based CF in cold-start user , cold-start item , and cold-start system settings . Performance is better when data is scarce , performance is no worse when data is plentiful , and algorithm efficiency is negligibly affected . We systematically compare a non-personalized baseline , user-based CF , item-based CF , and our bot-augmented user - and item-based CF algorithms using three data sets ( Yahoo ! Movies , MovieLens , and EachMovie ) with the normalized MAE metric in three types of cold-start situations . The advantage of our `` naïve filterbot '' approach is most pronounced for the Yahoo ! data , the sparsest of the three data sets .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

collaborative filtering
naïve filterbots

hybrid content and collaborative filtering


Privacy-preservation for gradient descent methods Gradient descent is a widely used paradigm for solving many optimization problems . Stochastic gradient descent performs a series of iterations to minimize a target function in order to reach a local minimum . In machine learning or data mining , this function corresponds to a decision model that is to be discovered . The gradient descent paradigm underlies many commonly used techniques in data mining and machine learning , such as neural networks , Bayesian networks , genetic algorithms , and simulated annealing . To the best of our knowledge , there has not been any work that extends the notion of privacy preservation or secure multi-party computation to gradient-descent-based techniques . In this paper , we propose a preliminary approach to enable privacy preservation in gradient descent methods in general and demonstrate its feasibility in specific gradient descent methods .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O

gradient descent method
regression
secure multi-party computation


Maximizing the spread of influence through a social network Models for the processes by which ideas and influence propagate through a social network have been studied in a number of domains , including the diffusion of medical and technological innovations , the sudden and widespread adoption of various strategies in game-theoretic settings , and the effects of `` word of mouth '' in the promotion of new products . Recently , motivated by the design of viral marketing strategies , Domingos and Richardson posed a fundamental algorithmic problem for such social network processes : if we can try to convince a subset of individuals to adopt a new product or innovation , and the goal is to trigger a large cascade of further adoptions , which set of individuals should we target ? We consider this problem in several of the most widely studied models in social network analysis . The optimization problem of selecting the most influential nodes is NP-hard here , and we provide the first provable approximation guarantees for efficient algorithms . Using an analysis framework based on submodular functions , we show that a natural greedy strategy obtains a solution that is provably within 63 % of optimal for several classes of models ; our framework suggests a general approach for reasoning about the performance guarantees of algorithms for these types of influence problems in social networks . We also provide computational experiments on large collaboration networks , showing that in addition to their provable guarantees , our approximation algorithms significantly out-perform node-selection heuristics based on the well-studied notions of degree centrality and distance centrality from the field of social networks .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O

approximation algorithms
social networks
viral marketing

diffusion of innovations
nonnumerical algorithms and problems


Detecting anomalous records in categorical datasets We consider the problem of detecting anomalies in high aritycategorical datasets . In most applications , anomalies are defined as datapoints that are `` abnormal '' . Quite often we have access to data which consists mostly of normal records , a long with a small percentage of unlabelled anomalous records . We are interested in the problem of unsupervised anomaly detection , where we use the unlabelled data for training , and detect records that do not follow the definition of normality . A standard approach is to create a model of normal data , and compare test records against it . A probabilistic approach builds a likelihood model from the training data . Records are tested for anomalies based on the complete record likelihood given the probability model . For categorical attributes , bayes nets give a standard representation of the likelihood . While this approach is good at finding outliers in the dataset , it often tends to detect records with attribute values that are rare . Sometimes , just detecting rare values of an attribute is not desired and such outliers are not considered as anomalies in that context . We present an alternative definition of anomalies , and propose an approach of comparing against marginal distribution of attribute subsets . We show that this is a more meaningful way of detecting anomalies , and has a better performance over semi-synthetic as well as real world datasets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Spatial-temporal causal modeling for climate change attribution Attribution of climate change to causal factors has been based predominantly on simulations using physical climate models , which have inherent limitations in describing such a complex and chaotic system . We propose an alternative , data centric , approach that relies on actual measurements of climate observations and human and natural forcing factors . Specifically , we develop a novel method to infer causality from spatial-temporal data , as well as a procedure to incorporate extreme value modeling into our method in order to address the attribution of extreme climate events , such as heatwaves . Our experimental results on a real world dataset indicate that changes in temperature are not solely accounted for by solar radiance , but attributed more significantly to CO2 and other greenhouse gases . Combined with extreme value modeling , we also show that there has been a significant increase in the intensity of extreme temperatures , and that such changes in extreme temperature are also attributable to greenhouse gases . These preliminary results suggest that our approach can offer a useful alternative to the simulation-based approach to climate modeling and attribution , and provide valuable insights from a fresh perspective .

O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

climate change attribution
extreme value modeling

graphical granger modeling
spatio-temporal causal modeling


Towards systematic design of distance functions for data mining applications Distance function computation is a key subtask in many data mining algorithms and applications . The most effective form of the distance function can only be expressed in the context of a particular data domain . It is also often a challenging and non-trivial task to find the most effective form of the distance function . For example , in the text domain , distance function design has been considered such an important and complex issue that it has been the focus of intensive research over three decades . The final design of distance functions in this domain has been reached only by detailed empirical testing and consensus over the quality of results provided by the different variations . With the increasing ability to collect data in an automated way , the number of new kinds of data continues to increase rapidly . This makes it increasingly difficult to undertake such efforts for each and every new data type . The most important aspect of distance function design is that since a human is the end-user for any application , the design must satisfy the user requirements with regard to effectiveness . This creates the need for a systematic framework to design distance functions which are sensitive to the particular characteristics of the data domain . In this paper , we discuss such a framework . The goal is to create distance functions in an automated waywhile minimizing the work required from the user . We will show that this framework creates distance functions which are significantly more effective than popularly used functions such as the Euclidean metric .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O


Why collective inference improves relational classification Procedures for collective inference make simultaneous statistical judgments about the same variables for a set of related data instances . For example , collective inference could be used to simultaneously classify a set of hyperlinked documents or infer the legitimacy of a set of related financial transactions . Several recent studies indicate that collective inference can significantly reduce classification error when compared with traditional inference techniques . We investigate the underlying mechanisms for this error reduction by reviewing past work on collective inference and characterizing different types of statistical models used for making inference in relational data . We show important differences among these models , and we characterize the necessary and sufficient conditions for reduced classification error based on experiments with real and simulated data .

O B I O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O

learning
probabilistic relational models
relational learning


Alpha seeding for support vector machines

classification
implementation
training speed-ups


Large human communication networks : patterns and a utility-driven generator Given a real , and weighted person-to-person network which changes over time , what can we say about the cliques that it contains ? Do the incidents of communication , or weights on the edges of a clique follow any pattern ? Real , and in-person social networks have many more triangles than chance would dictate . As it turns out , there are many more cliques than one would expect , in surprising patterns . In this paper , we study massive real-world social networks formed by direct contacts among people through various personal communication services , such as Phone-Call , SMS , IM etc. . The contributions are the following : ( a ) we discover surprising patterns with the cliques , ( b ) we report power-laws of the weights on the edges of cliques , ( c ) our real networks follow these patterns such that we can trust them to spot outliers and finally , ( d ) we propose the first utility-driven graph generator for weighted time-evolving networks , which match the observed patterns . Our study focused on three large datasets , each of which is a different type of communication service , with over one million records , and spans several months of activity .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The IOC algorithm : efficient many-class non-parametric classification for high-dimensional data This paper is about a variant of k nearest neighbor classification on large many-class high dimensional datasets . K nearest neighbor remains a popular classification technique , especially in areas such as computer vision , drug activity prediction and astrophysics . Furthermore , many more modern classifiers , such as kernel-based Bayes classifiers or the prediction phase of SVMs , require computational regimes similar to k-NN . We believe that tractable k-NN algorithms therefore continue to be important . This paper relies on the insight that even with many classes , the task of finding the majority class among the k nearest neighbors of a query need not require us to explicitly find those k nearest neighbors . This insight was previously used in ( Liu et al. , 2003 ) in two algorithms called KNS2 and KNS3 which dealt with fast classification in the case of two classes . In this paper we show how a different approach , IOC ( standing for the International Olympic Committee ) can apply to the case of n classes where n ) 2 . IOC assumes a slightly different processing of the datapoints in the neighborhood of the query . This allows it to search a set of metric trees , one for each class . During the searches it is possible to quickly prune away classes that can not possibly be the majority . We give experimental results on datasets of up to 5.8 x 105 records and 1.5 x 103 attributes , frequently showing an order of magnitude acceleration compared with each of ( i ) conventional linear scan , ( ii ) a well-known independent SR-tree implementation of conventional k-NN and ( iii ) a highly optimized conventional k-NN metric tree search .

O O O O O O O B O O O O O O O O O O B I I B O O O O O O O B I I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O

classification
k nearest neighbor
metric tree


Statistical entity-topic models The primary purpose of news articles is to convey information about who , what , when and where . But learning and summarizing these relationships for collections of thousands to millions of articles is difficult . While statistical topic models have been highly successful at topically summarizing huge collections of text documents , they do not explicitly address the textual interactions between who\/where , i.e. named entities ( persons , organizations , locations ) and what , i.e. the topics . We present new graphical models that directly learn the relationship between topics discussed in news articles and entities mentioned in each article . We show how these entity-topic models , through a better understanding of the entity-topic relationships , are better at making predictions about entities .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

entity recognition
miscellaneous
probabilistic algorithms
text modeling
topic modeling


Primal sparse Max-margin Markov networks Max-margin Markov networks ( M3N ) have shown great promise in structured prediction and relational learning . Due to the KKT conditions , the M3N enjoys dual sparsity . However , the existing M3N formulation does not enjoy primal sparsity , which is a desirable property for selecting significant features and reducing the risk of over-fitting . In this paper , we present an l1-norm regularized max-margin Markov network ( l1-M3N ) , which enjoys dual and primal sparsity simultaneously . To learn an l1-M3N , we present three methods including projected sub-gradient , cutting-plane , and a novel EM-style algorithm , which is based on an equivalence between l1-M3N and an adaptive M3N . We perform extensive empirical studies on both synthetic and real data sets . Our experimental results show that : ( 1 ) l1-M3N can effectively select significant features ; ( 2 ) l1-M3N can perform as well as the pseudo-primal sparse Laplace M3N in prediction accuracy , while consistently outperforms other competing methods that enjoy either primal or dual sparsity ; and ( 3 ) the EM-algorithm is more robust than the other two in pre-diction accuracy and time efficiency .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O


Mining and summarizing customer reviews Merchants selling products on the Web often ask their customers to review the products that they have purchased and the associated services . As e-commerce is becoming more and more popular , the number of customer reviews that a product receives grows rapidly . For a popular product , the number of reviews can be in hundreds or even thousands . This makes it difficult for a potential customer to read them to make an informed decision on whether to purchase the product . It also makes it difficult for the manufacturer of the product to keep track and to manage customer opinions . For the manufacturer , there are additional difficulties because many merchant sites may sell the same product and the manufacturer normally produces many kinds of products . In this research , we aim to mine and to summarize all the customer reviews of a product . This summarization task is different from traditional text summarization because we only mine the features of the product on which the customers have expressed their opinions and whether the opinions are positive or negative . We do not summarize the reviews by selecting a subset or rewrite some of the original sentences from the reviews to capture the main points as in the classic text summarization . Our task is performed in three steps : ( 1 ) mining product features that have been commented on by customers ; ( 2 ) identifying opinion sentences in each review and deciding whether each opinion sentence is positive or negative ; ( 3 ) summarizing the results . This paper proposes several novel techniques to perform these tasks . Our experimental results using reviews of a number of products sold online demonstrate the effectiveness of the techniques .

O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O


A Bayesian network framework for reject inference Most learning methods assume that the training set is drawn randomly from the population to which the learned model is to be applied . However in many applications this assumption is invalid . For example , lending institutions create models of who is likely to repay a loan from training sets consisting of people in their records to whom loans were given in the past ; however , the institution approved loan applications previously based on who was thought unlikely to default . Learning from only approved loans yields an incorrect model because the training set is a biased sample of the general population of applicants . The issue of including rejected samples in the learning process , or alternatively using rejected samples to adjust a model learned from accepted samples only , is called reject inference . The main contribution of this paper is a systematic analysis of different cases that arise in reject inference , with explanations of which cases arise in various real-world situations . We use Bayesian networks to formalize each case as a set of conditional independence relationships and identify eight cases , including the familiar missing completely at random ( MCAR ) , missing at random ( MAR ) , and missing not at random ( MNAR ) cases . For each case we present an overview of available learning algorithms . These algorithms have been published in separate fields of research , including epidemiology , econometrics , clinical trial evaluation , sociology , and credit scoring ; our second major contribution is to describe these algorithms in a common framework .

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

expectation-maximization
heckman estimator
propensity scores
sample selection bias


Finding partial orders from unordered 0-1 data In applications such as paleontology and medical genetics the 0-1 data has an underlying unknown order ( the ages of the fossil sites , the locations of markers in the genome ) . The order might be total or partial : for example , two sites in different parts of the globe might be ecologically incomparable , or the ordering of certain markers might be different in different subgroups of the data . We consider the following problem . Given a table over a set of 0-1 variables , find a partial order for the rows minimizing a score function and being as specific as possible . The score function can be , e.g. , the number of changes from 1 to 0 in a column ( for paleontology ) or the likelihood of the marker sequence ( for genomic data ) . Our solution for this task first constructs small totally ordered fragments of the partial order , then finds good orientations for the fragments , and finally uses a simple and efficient heuristic method for finding a partial order that corresponds well with the collection of fragments . We describe the method , discuss its properties , and give empirical results on paleontological data demonstrating the usefulness of the method . In the application the use of the method highlighted some previously unknown properties of the data and pointed out probable errors in the data .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

consecutive ones property
hidden ordering


Parallel computation of high dimensional robust correlation and covariance matrices The computation of covariance and correlation matrices are critical to many data mining applications and processes . Unfortunately the classical covariance and correlation matrices are very sensitive to outliers . Robust methods , such as QC and the Maronna method , have been proposed . However , existing algorithms for QC only give acceptable performance when the dimensionality of the matrix is in the hundreds ; and the Maronna method is rarely used in practice because of its high computational cost . In this paper , we develop parallel algorithms for both QC and the Maronna method . We evaluate these parallel algorithms using a real data set of the gene expression of over 6,000 genes , giving rise to a matrix of over 18 million entries . In our experimental evaluation , we explore scalability in dimensionality and in the number of processors . We also compare the parallel behaviors of the two methods . After thorough experimentation , we conclude that for many data mining applications , both QC and Maronna are viable options . Less robust , but faster , QC is the recommended choice for small parallel platforms . On the other hand , the Maronna method is the recommended choice when a high degree of robustness is required , or when the parallel platform features a high number of processors .

B O O O O B B O B O O O O B O B O O O O O O O O O O O O O O B O B O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O

correlation covariance maronna parallel robust


Clustering based large margin classification : a scalable approach using SOCP formulation This paper presents a novel Second Order Cone Programming ( SOCP ) formulation for large scale binary classification tasks . Assuming that the class conditional densities are mixture distributions , where each component of the mixture has a spherical covariance , the second order statistics of the components can be estimated efficiently using clustering algorithms like BIRCH . For each cluster , the second order moments are used to derive a second order cone constraint via a Chebyshev-Cantelli inequality . This constraint ensures that any data point in the cluster is classified correctly with a high probability . This leads to a large margin SOCP formulation whose size depends on the number of clusters rather than the number of training data points . Hence , the proposed formulation scales well for large datasets when compared to the state-of-the-art classifiers , Support Vector Machines ( SVMs ) . Experiments on real world and synthetic datasets show that the proposed algorithm outperforms SVM solvers in terms of training time and achieves similar accuracies .

O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Deformable Markov model templates for time-series pattern matching

deformable templates
hidden markov models
segmental markov models
time series


Unifying dependent clustering and disparate clustering for non-homogeneous data Modern data mining settings involve a combination of attribute-valued descriptors over entities as well as specified relationships between these entities . We present an approach to cluster such non-homogeneous datasets by using the relationships to impose either dependent clustering or disparate clustering constraints . Unlike prior work that views constraints as boolean criteria , we present a formulation that allows constraints to be satisfied or violated in a smooth manner . This enables us to achieve dependent clustering and disparate clustering using the same optimization framework by merely maximizing versus minimizing the objective function . We present results on both synthetic data as well as several real-world datasets .

O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O

contingency tables
learning
multi-criteria optimization.
relational clustering


Web object indexing using domain knowledge A web object is defined to represent any meaningful object embedded in web pages ( e.g. images , music ) or pointed to by hyperlinks ( e.g. downloadable files ) . In many cases , users would like to search for information of a certain ` object ' , rather than a web page containing the query terms . To facilitate web object searching and organizing , in this paper , we propose a novel approach to web object indexing , by discovering its inherent structure information with existed domain knowledge . In our approach , first , Layered LSI spaces are built for a better representation of the hierarchically structured domain knowledge , in order to emphasize the specific semantics and term space in each layer of the domain knowledge . Meanwhile , the web object representation is constructed by hyperlink analysis , and further pruned to remove the noises . Then an optimal matching between the web object and the domain knowledge is performed , in order to pick out the structure attributes of the web object from the knowledge . Finally , the obtained structure attributes are used to re-organize and index the web objects . Our approach also indicates a new promising way to use trust-worthy Deep Web knowledge to help organize dispersive information of Surface Web .

B I B O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

confidence propagation
information retrieval
latent semantic indexing
link analysis
music indexing


TreeDT : gene mapping by tree disequilibrium test We introduce and evaluate TreeDT , a novel gene mapping method which is based on discovering and assessing tree-like patterns in genetic marker data . Gene mapping aims at discovering a statistical connection from a particular disease or trait to a narrow region in the genome . In a typical case-control setting , data consists of genetic markers typed for a set of disease-associated chromosomes and a set of control chromosomes . A computer scientist would view this data as a set of strings . TreeDT extracts , essentially in the form of substrings and prefix trees , information about the historical recombinations in the population . This information is used to locate fragments potentially inherited from a common diseased founder , and to map the disease gene into the most likely such fragment . The method measures for each chromosomal location the disequilibrium of the prefix tree of marker strings starting from the location , to assess the distribution of disease-associated chromosomes . We evaluate experimentally the performance of TreeDT on realistic , simulated data sets , and comparisons to state of the art methods ( TDT , HPM ) show that TreeDT is very competitive .

O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Discovering similar patterns in time series


Collaborative crawling : mining user experiences for topical resource discovery The rapid growth of the world wide web had made the problem of topic specific resource discovery an important one in recent years . In this problem , it is desired to find web pages which satisfy a predicate specified by the user . Such a predicate could be a keyword query , a topical query , or some arbitrary contraint . Several techniques such as focussed crawling and intelligent crawling have recently been proposed for topic specific resource discovery . All these crawlers are linkage based , since they use the hyperlink behavior in order to perform resource discovery . Recent studies have shown that the topical correlations in hyperlinks are quite noisy and may not always show the consistency necessary for a reliable resource discovery process . In this paper , we will approach the problem of resource discovery from an entirely different perspective ; we will mine the significant browsing patterns of world wide web users in order to model the likelihood of web pages belonging to a specified predicate . This user behavior can be mined from the freely available traces of large public domain proxies on the world wide web . We refer to this technique as collaborative crawling because it mines the collective user experiences in order to find topical resources . Such a strategy is extremely effective because the topical consistency in world wide web browsing patterns turns out to very reliable . In addition , the user-centered crawling system can be combined with linkage based systems to create an overall system which works more effectively than a system based purely on either user behavior or hyperlinks .

O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Practical guide to controlled experiments on the web : listen to your customers not to the hippo The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments , also called randomized experiments ( single factor or factorial designs ) , A\/B tests ( and their generalizations ) , split tests , Control\/Treatment tests , and parallel flights . Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior . We provide a practical guide to conducting online experiments , where end-users can help guide the development of features . Our experience indicates that significant learning and return-on-investment ( ROI ) are seen when development teams listen to their customers , not to the Highest Paid Person 's Opinion ( HiPPO ) . We provide several examples of controlled experiments with surprising results . We review the important ingredients of running controlled experiments , and discuss their limitations ( both technical and organizational ) . We focus on several areas that are critical to experimentation , including statistical power , sample size , and techniques for variance reduction . We describe common architectures for experimentation systems and analyze their advantages and disadvantages . We evaluate randomization and hashing techniques , which we show are not as simple in practice as is often assumed . Controlled experiments typically generate large amounts of data , which can be analyzed using data mining techniques to gain deeper understanding of the factors influencing the outcome of interest , leading to new hypotheses and creating a virtuous cycle of improvements . Organizations that embrace controlled experiments with clear evaluation criteria can evolve their systems with automated optimizations and real-time analyses . Based on our extensive practical experience with multiple systems and organizations , we share key lessons that will help practitioners in running trustworthy controlled experiments .

O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O


Robust space transformations for distance-based operations For many KDD operations , such as nearest neighbor search , distance-based clustering , and outlier detection , there is an underlying & kgr ;-D data space in which each tuple\/object is represented as a point in the space . In the presence of differing scales , variability , correlation , and\/or outliers , we may get unintuitive results if an inappropriate space is used . The fundamental question that this paper addresses is : `` What then is an appropriate space ? '' We propose using a robust space transformation called the Donoho-Stahel estimator . In the first half of the paper , we show the key properties of the estimator . Of particular importance to KDD applications involving databases is the stability property , which says that in spite of frequent updates , the estimator does not : ( a ) change much , ( b ) lose its usefulness , or ( c ) require re-computation . In the second half , we focus on the computation of the estimator for high-dimensional databases . We develop randomized algorithms and evaluate how well they perform empirically . The novel algorithm we develop called the Hybrid-random algorithm is , in most cases , at least an order of magnitude faster than the Fixed-angle and Subsampling algorithms .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

computation of transforms
data mining
distance-based operations
robust estimators
robust statistics


A rank sum test method for informative gene discovery Finding informative genes from microarray data is an important research problem in bioinformatics research and applications . Most of the existing methods rank features according to their discriminative capability and then find a subset of discriminative genes ( usually top k genes ) . In particular , t-statistic criterion and its variants have been adopted extensively . This kind of methods rely on the statistics principle of t-test , which requires that the data follows a normal distribution . However , according to our investigation , the normality condition often can not be met in real data sets . To avoid the assumption of the normality condition , in this paper , we propose a rank sum test method for informative gene discovery . The method uses a rank-sum statistic as the ranking criterion . Moreover , we propose using the significance level threshold , instead of the number of informative genes , as the parameter . The significance level threshold as a parameter carries the quality specification in statistics . We follow the Pitman efficiency theory to show that the rank sum method is more accurate and more robust than the t-statistic method in theory . To verify the effectiveness of the rank sum method , we use support vector machine ( SVM ) to construct classifiers based on the identified informative genes on two well known data sets , namely colon data and leukemia data . The prediction accuracy reaches 96.2 % on the colon data and 100 % on the leukemia data . The results are clearly better than those from the previous feature ranking methods . By experiments , we also verify that using significance level threshold is more effective than directly specifying an arbitrary k.

O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

informative genes
rank sum test
ranking criterion
support vector machine


Bypass rates : reducing query abandonment using negative inferences We introduce a new approach to analyzing click logs by examining both the documents that are clicked and those that are bypassed-documents returned higher in the ordering of the search results but skipped by the user . This approach complements the popular click-through rate analysis , and helps to draw negative inferences in the click logs . We formulate a natural objective that finds sets of results that are unlikely to be collectively bypassed by a typical user . This is closely related to the problem of reducing query abandonment . We analyze a greedy approach to optimizing this objective , and establish theoretical guarantees of its performance . We evaluate our approach on a large set of queries , and demonstrate that it compares favorably to the maximal marginal relevance approach on a number of metrics including mean average precision and mean reciprocal rank .

B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O

bypass rates
query abandonment
relevance


On interactive visualization of high-dimensional data using the hyperbolic plane We propose a novel projection based visualization method for high-dimensional datasets by combining concepts from MDS and the geometry of the hyperbolic spaces . Our approach Hyperbolic Multi-Dimensional Scaling ( H-MDS ) extends earlier work ( 7 ) using hyperbolic spaces for visualization of tree structures data ( `` hyperbolic tree browser '' ) . By borrowing concepts from multi-dimensional scaling we map proximity data directly into the 2-dimensional hyperbolic space ( H2 ) . This removes the restriction to `` quasihierarchical '' , graph-based data -- limiting previous work . Since a suitable distance function can convert all kinds of data to proximity ( or distance-based ) data this type of data can be considered the most general . We used the circular Poincaré model of the H2 which allows effective human-computer interaction : by moving the `` focus '' via mouse the user can navigate in the data without loosing the `` context '' . In H2 the `` fish-eye '' behavior originates not simply by a non-linear view transformation but rather by extraordinary , non-Euclidean properties of the H2 . Especially , the exponential growth of length and area of the underlying space makes the H2 a prime target for mapping hierarchical and ( now also ) high-dimensional data . We present several high-dimensional mapping examples including synthetic and real world data and a successful application for unstructured text . By analyzing and integrating multiple film critiques from news : rec . art . movies . reviews and the internet movie database , each movie becomes placed within the H2 . Here the idea is , that related films share more words in their reviews than unrelated . Their semantic proximity leads to a closer arrangement . The result is a kind of high-level content structured display allowing the user to explore the `` space of movies '' .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

focus+context
h-mds
hyperbolic multi-dimensional scaling
infoviz
interaction styles
semantic browsing
text mining
visualizing high-dimensional data


Mining intrusion detection alarms for actionable knowledge In response to attacks against enterprise networks , administrators increasingly deploy intrusion detection systems . These systems monitor hosts , networks , and other resources for signs of security violations . The use of intrusion detection has given rise to another difficult problem , namely the handling of a generally large number of alarms . In this paper , we mine historical alarms to learn how future alarms can be handled more efficiently . First , we investigate episode rules with respect to their suitability in this approach . We report the difficulties encountered and the unexpected insights gained . In addition , we introduce a new conceptual clustering technique , and use it in extensive experiments with real-world data to show that intrusion detection alarms can be handled efficiently by using previously mined knowledge .

O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O

conceptual clustering
episode rules
intrusion detection


Classification features for attack detection in collaborative recommender systems Collaborative recommender systems are highly vulnerable to attack . Attackers can use automated means to inject a large number of biased profiles into such a system , resulting in recommendations that favor or disfavor given items . Since collaborative recommender systems must be open to user input , it is difficult to design a system that can not be so attacked . Researchers studying robust recommendation have therefore begun to identify types of attacks and study mechanisms for recognizing and defeating them . In this paper , we propose and study different attributes derived from user profiles for their utility in attack detection . We show that a machine learning classification approach that includes attributes derived from attack models is more successful than more generalized detection algorithms previously studied .

O O O B I O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O

attack detection
learning
recommender systems

collaborative filtering
information search and retrieval
robustness
systems and software


Cross-language information retrieval using PARAFAC2 A standard approach to cross-language information retrieval ( CLIR ) uses Latent Semantic Analysis ( LSA ) in conjunction with a multilingual parallel aligned corpus . This approach has been shown to be successful in identifying similar documents across languages - or more precisely , retrieving the most similar document in one language to a query in another language . However , the approach has severe drawbacks when applied to a related task , that of clustering documents `` language-independently '' , so that documents about similar topics end up closest to one another in the semantic space regardless of their language . The problem is that documents are generally more similar to other documents in the same language than they are to documents in a different language , but on the same topic . As a result , when using multilingual LSA , documents will in practice cluster by language , not by topic . We propose a novel application of PARAFAC2 ( which is a variant of PARAFAC , a multi-way generalization of the singular value decomposition ( SVD ) ) to overcome this problem . Instead of forming a single multilingual term-by-document matrix which , under LSA , is subjected to SVD , we form an irregular three-way array , each slice of which is a separate term-by-document matrix for a single language in the parallel corpus . The goal is to compute an SVD for each language such that V ( the matrix of right singular vectors ) is the same across all languages . Effectively , PARAFAC2 imposes the constraint , not present in standard LSA , that the `` concepts '' in all documents in the parallel corpus are the same regardless of language . Intuitively , this constraint makes sense , since the whole purpose of using a parallel corpus is that exactly the same concepts are expressed in the translations . We tested this approach by comparing the performance of PARAFAC2 with standard LSA in solving a particular CLIR problem . From our results , we conclude that PARAFAC2 offers a very promising alternative to LSA not only for multilingual document clustering , but also for solving other problems in cross-language information retrieval .

O B I O B O O O O O B I O O O O B I I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B I O

information retrieval
latent semantic analysis
multilingual
parafac2


A framework for simultaneous co-clustering and learning from complex data For difficult classification or regression problems , practitioners often segment the data into relatively homogenous groups and then build a model for each group . This two-step procedure usually results in simpler , more interpretable and actionable models without any lossin accuracy . We consider problems such as predicting customer behavior across products , where the independent variables can be naturally partitioned into two groups . A pivoting operation can now result in the dependent variable showing up as entries in a `` customer by product '' data matrix . We present a model-based co-clustering ( meta ) - algorithm that interleaves clustering and construction of prediction models to iteratively improve both cluster assignment and fit of the models . This algorithm provably converges to a local minimum of a suitable cost function . The framework not only generalizes co-clustering and collaborative filtering to model-basedco-clustering , but can also be viewed as simultaneous co-segmentation and classification or regression , which is better than independently clustering the data first and then building models . Moreover , it applies to a wide range of bi-modal or multimodal data , and can be easily specialized to address classification and regression problems . We demonstrate the effectiveness of our approach on both these problems through experimentation on real and synthetic data .

O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O

classification
multimodal data
prediction models
regression


Semi-supervised time series classification The problem of time series classification has attracted great interest in the last decade . However current research assumes the existence of large amounts of labeled training data . In reality , such data may be very difficult or expensive to obtain . For example , it may require the time and expertise of cardiologists , space launch technicians , or other domain specialists . As in many other domains , there are often copious amounts of unlabeled data available . For example , the PhysioBank archive contains gigabytes of ECG data . In this work we propose a semi-supervised technique for building time series classifiers . While such algorithms are well known in text domains , we will show that special considerations must be made to make them both efficient and effective for the time series domain . We evaluate our work with a comprehensive set of experiments on diverse data sources including electrocardiograms , handwritten documents , and video datasets . The experimental results demonstrate that our approach requires only a handful of labeled examples to construct accurate classifiers .

O B I B O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


CloseGraph : mining closed frequent graph patterns Recent research on pattern discovery has progressed form mining frequent itemsets and sequences to mining structured patterns including trees , lattices , and graphs . As a general data structure , graph can model complicated relations among data with wide applications in bioinformatics , Web exploration , and etc. . However , mining large graph patterns in challenging due to the presence of an exponential number of frequent subgraphs . Instead of mining all the subgraphs , we propose to mine closed frequent graph patterns . A graph g is closed in a database if there exists no proper supergraph of g that has the same support as g. A closed graph pattern mining algorithm , CloseGraph , is developed by exploring several interesting pruning methods . Our performance study shows that CloseGraph not only dramatically reduces unnecessary subgraphs to be generated but also substantially increases the efficiency of mining , especially in the presence of large graph patterns .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

canonical label
closed pattern
database applications
graph representation


k-NN as an implementation of situation testing for discrimination discovery and prevention With the support of the legally-grounded methodology of situation testing , we tackle the problems of discrimination discovery and prevention from a dataset of historical decisions by adopting a variant of k-NN classification . A tuple is labeled as discriminated if we can observe a significant difference of treatment among its neighbors belonging to a protected-by-law group and its neighbors not belonging to it . Discrimination discovery boils down to extracting a classification model from the labeled tuples . Discrimination prevention is tackled by changing the decision value for tuples labeled as discriminated before training a classifier . The approach of this paper overcomes legal weaknesses and technical limitations of existing proposals .

O O O O O O O O B I I I O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


ArnetMiner : extraction and mining of academic social networks This paper addresses several key issues in the ArnetMiner system , which aims at extracting and mining academic social networks . Specifically , the system focuses on : 1 ) Extracting researcher profiles automatically from the Web ; 2 ) Integrating the publication data into the network from existing digital libraries ; 3 ) Modeling the entire academic network ; and 4 ) Providing search services for the academic network . So far , 448,470 researcher profiles have been extracted using a unified tagging approach . We integrate publications from online Web databases and propose a probabilistic framework to deal with the name ambiguity problem . Furthermore , we propose a unified modeling approach to simultaneously model topical aspects of papers , authors , and publication venues . Search services such as expertise search and people association search have been provided based on the modeling results . In this paper , we describe the architecture and main features of the system . We also present the empirical evaluation of the proposed methods .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

information extraction
information search and retrieval
name disambiguation
social network
topic modeling


Explicitly representing expected cost : an alternative to ROC representation


Wavelet synopsis for data streams : minimizing non-euclidean error We consider the wavelet synopsis construction problem for data streams where given n numbers we wish to estimate the data by constructing a synopsis , whose size , say B is much smaller than n. The B numbers are chosen to minimize a suitable error between the original data and the estimate derived from the synopsis . Several good one-pass wavelet construction streaming algorithms minimizing the l2 error exist . For other error measures , the problem is less understood . We provide the first one-pass small space streaming algorithms with provable error guarantees ( additive approximation ) for minimizing a variety of non-Euclidean error measures including all weighted lp ( including l ∞ ) and relative error lp metrics . In several previous works solutions ( for weighted l2 , l ∞ and maximum relative error ) where the B synopsis coefficients are restricted to be wavelet coefficients of the data were proposed . This restriction yields suboptimal solutions on even fairly simple examples . Other lines of research , such as probabilistic synopsis , imposed restrictions on how the synopsis was arrived at . To the best of our knowledge this paper is the first paper to address the general problem , without any restriction on how the synopsis is arrived at , as well as provide the first streaming algorithms with guaranteed performance for these classes of error measures .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

miscellaneous
streaming algorithm
wavelet synopses


Correlating synchronous and asynchronous data streams In a variety of modern mining applications , data are commonly viewed as infinite time ordered data streams rather as finite data sets stored on disk . This view challenges fundamental assumptions commonly made in the context of several data mining algorithms . In this paper , we study the problem of identifying correlations between multiple data streams . In particular , we propose algorithms capable of capturing correlations between multiple continuous data streams in a highly efficient and accurate manner . Our algorithms and techniques are applicable in the case of both synchronous and asynchronous data streaming environments . We capture correlations between multiple streams using the well known technique of Singular Value Decomposition ( SVD ) . Correlations between data items , and the SVD technique in particular , have been repeatedly utilized in an off-line ( non stream ) data mining problems , for example forecasting , approximate query answering , and data reduction . We propose a methodology based on a combination of dimensionality reduction and sampling to make the SVD technique suitable for a data stream context . Our techniques are approximate , trading accuracy with performance , and we analytically quantify this tradeoff . We present a through experimental evaluation , using both real and synthetic data sets , from a prototype implementation of our technique , investigating the impact of various parameters in the accuracy of the overall computation . Our results indicate , that correlations between multiple data streams can be identified very efficiently and accurately . The algorithms proposed herein , are presented as generic tools , with a multitude of applications on data stream mining problems .

O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data streams
singular value decomposition


Unsupervised transfer classification : application to text categorization We study the problem of building the classification model for a target class in the absence of any labeled training example for that class . To address this difficult learning problem , we extend the idea of transfer learning by assuming that the following side information is available : ( i ) a collection of labeled examples belonging to other classes in the problem domain , called the auxiliary classes ; ( ii ) the class information including the prior of the target class and the correlation between the target class and the auxiliary classes . Our goal is to construct the classification model for the target class by leveraging the above data and information . We refer to this learning problem as unsupervised transfer classification . Our framework is based on the generalized maximum entropy model that is effective in transferring the label information of the auxiliary classes to the target class . A theoretical analysis shows that under certain assumption , the classification model obtained by the proposed approach converges to the optimal model when it is learned from the labeled examples for the target class . Empirical study on text categorization over four different data sets verifies the effectiveness of the proposed approach .

B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O

generalized maximum entropy model
text categorization
unsupervised transfer classification


Hypergraph spectral learning for multi-label classification A hypergraph is a generalization of the traditional graph in which the edges are arbitrary non-empty subsets of the vertex set . It has been applied successfully to capture high-order relations in various domains . In this paper , we propose a hypergraph spectral learning formulation for multi-label classification , where a hypergraph is constructed to exploit the correlation information among different labels . We show that the proposed formulation leads to an eigenvalue problem , which may be computationally expensive especially for large-scale problems . To reduce the computational cost , we propose an approximate formulation , which is shown to be equivalent to a least squares problem under a mild condition . Based on the approximate formulation , efficient algorithms for solving least squares problems can be applied to scale the formulation to very large data sets . In addition , existing regularization techniques for least squares can be incorporated into the model for improved generalization performance . We have conducted experiments using large-scale benchmark data sets , and experimental results show that the proposed hypergraph spectral learning formulation is effective in capturing the high-order relations in multi-label problems . Results also indicate that the approximate formulation is much more efficient than the original one , while keeping competitive classification performance .

B B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

hypergraph
least squares
regularization
spectral learning

canonical correlation analysis
efficiency
multi-label classification


Domain-constrained semi-supervised mining of tracking models in sensor networks Accurate localization of mobile objects is a major research problem in sensor networks and an important data mining application . Specifically , the localization problem is to determine the location of a client device accurately given the radio signal strength values received at the client device from multiple beacon sensors or access points . Conventional data mining and machine learning methods can be applied to solve this problem . However , all of them require large amounts of labeled training data , which can be quite expensive . In this paper , we propose a probabilistic semi supervised learning approach to reduce the calibration effort and increase the tracking accuracy . Our method is based on semi-supervised conditional random fields which can enhance the learned model from a small set of training data with abundant unlabeled data effectively . To make our method more efficient , we exploit a Generalized EM algorithm coupled with domain constraints . We validate our method through extensive experiments in a real sensor network using Crossbow MICA2 sensors . The results demonstrate the advantages of methods compared to other state-of-the-art object-tracking algorithms .

O O O O B O O B I O B O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

calibration
em
learning
localization
sensor networks
tracking


Unsupervised feature selection for principal components analysis Principal Components Analysis ( PCA ) is the predominant linear dimensionality reduction technique , and has been widely applied on datasets in all scientific domains . We consider , both theoretically and empirically , the topic of unsupervised feature selection for PCA , by leveraging algorithms for the so-called Column Subset Selection Problem ( CSSP ) . In words , the CSSP seeks the `` best '' subset of exactly k columns from an m x n data matrix A , and has been extensively studied in the Numerical Linear Algebra community . We present a novel two-stage algorithm for the CSSP . From a theoretical perspective , for small to moderate values of k , this algorithm significantly improves upon the best previously-existing results ( 24 , 12 ) for the CSSP . From an empirical perspective , we evaluate this algorithm as an unsupervised feature selection strategy in three application domains of modern statistical data analysis : finance , document-term data , and genetics . We pay particular attention to how this algorithm may be used to select representative or landmark features from an object-feature matrix in an unsupervised manner . In all three application domains , we are able to identify k landmark features , i.e. , columns of the data matrix , that capture nearly the same amount of information as does the subspace that is spanned by the top k `` eigenfeatures . ''

O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

numerical linear algebra
pca
subset selection


Angle-based outlier detection in high-dimensional data Detecting outliers in a large set of data objects is a major data mining task aiming at finding different mechanisms responsible for different groups of objects in a data set . All existing approaches , however , are based on an assessment of distances ( sometimes indirectly by assuming certain distributions ) in the full-dimensional Euclidean data space . In high-dimensional data , these approaches are bound to deteriorate due to the notorious `` curse of dimensionality '' . In this paper , we propose a novel approach named ABOD ( Angle-Based Outlier Detection ) and some variants assessing the variance in the angles between the difference vectors of a point to the other points . This way , the effects of the `` curse of dimensionality '' are alleviated compared to purely distance-based approaches . A main advantage of our new approach is that our method does not rely on any parameter selection influencing the quality of the achieved ranking . In a thorough experimental evaluation , we compare ABOD to the well-established distance-based method LOF for various artificial and a real world data set and show ABOD to perform especially well on high-dimensional data .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Relational Markov models and their application to adaptive web navigation Relational Markov models ( RMMs ) are a generalization of Markov models where states can be of different types , with each type described by a different set of variables . The domain of each variable can be hierarchically structured , and shrinkage is carried out over the cross product of these hierarchies . RMMs make effective learning possible in domains with very large and heterogeneous state spaces , given only sparse data . We apply them to modeling the behavior of web site users , improving prediction in our PROTEUS architecture for personalizing web sites . We present experiments on an e-commerce and an academic web site showing that RMMs are substantially more accurate than alternative methods , and make good predictions even when applied to previously-unvisited parts of the site .

O B I O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

personalization
relational probabilistic models
web mining


Xproj : a framework for projected structural clustering of xml documents XML has become a popular method of data representation both on the web and in databases in recent years . One of the reasons for the popularity of XML has been its ability to encode structural information about data records . However , this structural characteristic of data sets also makes it a challenging problem for a variety of data mining problems . One such problem is that of clustering , in which the structural aspects of the data result in a high implicit dimensionality of the data representation . As a result , it becomes more difficult to cluster the data in a meaningful way . In this paper , we propose an effective clustering algorithm for XML data which uses substructures of the documents in order to gain insights about the important underlying structures . We propose new ways of using multiple sub-structuralinformation in XML documents to evaluate the quality of intermediate cluster solutions , and guide the algorithms to a final solution which reflects the true structural behavior in individual partitions . We test the algorithm on a variety of real and synthetic data sets .

O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O