Global distance-based segmentation of trajectories This work introduces distance-based criteria for segmentation of object trajectories . Segmentation leads to simplification of the original objects into smaller , less complex primitives that are better suited for storage and retrieval purposes . Previous work on trajectory segmentation attacked the problem locally , segmenting separately each trajectory of the database . Therefore , they did not directly optimize the inter-object separability , which is necessary for mining operations such as searching , clustering , and classification on large databases . In this paper we analyze the trajectory segmentation problem from a global perspective , utilizing data aware distance-based optimization techniques , which optimize pairwise distance estimates hence leading to more efficient object pruning . We first derive exact solutions of the distance-based formulation . Due to the intractable complexity of the exact solution , we present anapproximate , greedy solution that exploits forward searching of locally optimal solutions . Since the greedy solution also imposes a prohibitive computational cost , we also put forward more light weight variance-based segmentation techniques , which intelligently `` relax '' the pairwise distance only in the areas that affect the least the mining operation .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Augmenting the generalized hough transform to enable the mining of petroglyphs Rock art is an archaeological term for human-made markings on stone . It is believed that there are millions of petroglyphs in North America alone , and the study of this valued cultural resource has implications even beyond anthropology and history . Surprisingly , although image processing , information retrieval and data mining have had large impacts on many human endeavors , they have had essentially zero impact on the study of rock art . In this work we identify the reasons for this , and introduce a novel distance measure and algorithms which allow efficient and effective data mining of large collections of rock art .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Next frontier This talk is about the next frontier in knowledge discovery and data mining .

knowledge representation formalisms and methods


Fastanova : an efficient algorithm for genome-wide association study Studying the association between quantitative phenotype ( such as height or weight ) and single nucleotide polymorphisms ( SNPs ) is an important problem in biology . To understand underlying mechanisms of complex phenotypes , it is often necessary to consider joint genetic effects across multiple SNPs . ANOVA ( analysis of variance ) test is routinely used in association study . Important findings from studying gene-gene ( SNP-pair ) interactions are appearing in the literature . However , the number of SNPs can be up to millions . Evaluating joint effects of SNPs is a challenging task even for SNP-pairs . Moreover , with large number of SNPs correlated , permutation procedure is preferred over simple Bonferroni correction for properly controlling family-wise error rate and retaining mapping power , which dramatically increases the computational cost of association study . In this paper , we study the problem of finding SNP-pairs that have significant associations with a given quantitative phenotype . We propose an efficient algorithm , FastANOVA , for performing ANOVA tests on SNP-pairs in a batch mode , which also supports large permutation test . We derive an upper bound of SNP-pair ANOVA test , which can be expressed as the sum of two terms . The first term is based on single-SNP ANOVA test . The second term is based on the SNPs and independent of any phenotype permutation . Furthermore , SNP-pairs can be organized into groups , each of which shares a common upper bound . This allows for maximum reuse of intermediate computation , efficient upper bound estimation , and effective SNP-pair pruning . Consequently , FastANOVA only needs to perform the ANOVA test on a small number of candidate SNP-pairs without the risk of missing any significant ones . Extensive experiments demonstrate that FastANOVA is orders of magnitude faster than the brute-force implementation of ANOVA tests on all SNP pairs .

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Cold start link prediction In the traditional link prediction problem , a snapshot of a social network is used as a starting point to predict , by means of graph-theoretic measures , the links that are likely to appear in the future . In this paper , we introduce cold start link prediction as the problem of predicting the structure of a social network when the network itself is totally missing while some other information regarding the nodes is available . We propose a two-phase method based on the bootstrap probabilistic graph . The first phase generates an implicit social network under the form of a probabilistic graph . The second phase applies probabilistic graph-based measures to produce the final prediction . We assess our method empirically over a large data collection obtained from Flickr , using interest groups as the initial information . The experiments confirm the effectiveness of our approach .

O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An approach to spacecraft anomaly detection problem using kernel feature space Development of advanced anomaly detection and failure diagnosis technologies for spacecraft is a quite significant issue in the space industry , because the space environment is harsh , distant and uncertain . While several modern approaches based on qualitative reasoning , expert systems , and probabilistic reasoning have been developed recently for this purpose , any of them has a common difficulty in obtaining accurate and complete a priori knowledge on the space systems from human experts . A reasonable alternative to this conventional anomaly detection method is to reuse a vast amount of telemetry data which is multi-dimensional time-series continuously produced from a number of system components in the spacecraft . This paper proposes a novel `` knowledge-free '' anomaly detection method for spacecraft based on Kernel Feature Space and directional distribution , which constructs a system behavior model from the past normal telemetry data from a set of telemetry data in normal operation and monitors the current system status by checking incoming data with the model . In this method , we regard anomaly phenomena as unexpected changes of causal associations in the spacecraft system , and hypothesize that the significant causal associations inside the system will appear in the form of principal component directions in a high-dimensional non-linear feature space which is constructed by a kernel function and a set of data . We have confirmed the effectiveness of the proposed anomaly detection method by applying it to the telemetry data obtained from a simulator of an orbital transfer vehicle designed to make a rendezvous maneuver with the International Space Station .

O O O B B I O O B I I O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

anomaly detection
kernel feature space
spacecraft

learning
principal component analysis
time series data
von mises fisher distribution


On detecting differences between groups Understanding the differences between contrasting groups is a fundamental task in data analysis . This realization has led to the development of a new special purpose data mining technique , contrast-set mining . We undertook a study with a retail collaborator to compare contrast-set mining with existing rule-discovery techniques . To our surprise we observed that straightforward application of an existing commercial rule-discovery system , Magnum Opus , could successfully perform the contrast-set-mining task . This led to the realization that contrast-set mining is a special case of the more general rule-discovery task . We present the results of our study together with a proof of this conclusion .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

contrast-set discovery
information search and retrieval
learning
retailing
rule discovery


Coherent closed quasi-clique discovery from large dense graph databases Frequent coherent subgraphs can provide valuable knowledge about the underlying internal structure of a graph database , and mining frequently occurring coherent subgraphs from large dense graph databases has been witnessed several applications and received considerable attention in the graph mining community recently . In this paper , we study how to efficiently mine the complete set of coherent closed quasi-cliques from large dense graph databases , which is an especially challenging task due to the downward-closure property no longer holds . By fully exploring some properties of quasi-cliques , we propose several novel optimization techniques , which can prune the unpromising and redundant sub-search spaces effectively . Meanwhile , we devise an efficient closure checking scheme to facilitate the discovery of only closed quasi-cliques . We also develop a coherent closed quasi-clique mining algorithm , ( B ) Cocain ( \/ B ) 1 Thorough performance study shows that Cocain is very efficient and scalable for large dense graph databases .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Cancer genomics Throughout life , the cells in every individual accumulate many changes in the DNA inherited from his or her parents . Certain combinations of changes lead to cancer . During the last decade , the cost of DNA sequencing has been dropping by a factor of 10 every two years , making it now possible to read most of the three billion base genome from a patient 's cancer tumor , and to try to determine all of the thousands of DNA changes in it . Under the auspices of NCI 's Cancer Genome Atlas Project , 10,000 tumors will be sequenced in this manner in the next few years . Soon cancer genome sequencing will be a widespread clinical practice , and millions of tumors will be sequenced . A massive computational problem looms in interpreting these data . First , because we can only read short pieces of DNA , we have the enormous problem of assembling a coherent and reliable representation of the tumor genome from massive amounts of incomplete and error-prone evidence . This is the first challenge . Second , every human genome is unique from birth , and every tumor a unique variant . There is no single route to cancer . We must learn to read the varied signatures of cancer within the tumor genome and associate these with optimal treatments . Already there are hundreds of molecularly targeted treatments for cancer available , each known to be more or less effective depending on specific genetic variants . However , targeting a single gene with one treatment rarely works . The second challenge is to tackle the combinatorics of personalized , targeted , combination therapy in cancer .

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


Programming the K-means clustering algorithm in SQL Using SQL has not been considered an efficient and feasible way to implement data mining algorithms . Although this is true for many data mining , machine learning and statistical algorithms , this work shows it is feasible to get an efficient SQL implementation of the well-known K-means clustering algorithm that can work on top of a relational DBMS . The article emphasizes both correctness and performance . From a correctness point of view the article explains how to compute Euclidean distance , nearest-cluster queries and updating clustering results in SQL . From a performance point of view it is explained how to cluster large data sets defining and indexing tables to store and retrieve intermediate and final results , optimizing and avoiding joins , optimizing and simplifying clustering aggregations , and taking advantage of sufficient statistics . Experiments evaluate scalability with synthetic data sets varying size and dimensionality . The proposed K-means implementation can cluster large data sets and exhibits linear scalability .

O O O B O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Personal privacy vs population privacy : learning to attack anonymization Over the last decade great strides have been made in developing techniques to compute functions privately . In particular , Differential Privacy gives strong promises about conclusions that can be drawn about an individual . In contrast , various syntactic methods for providing privacy ( criteria such as k-anonymity and l-diversity ) have been criticized for still allowing private information of an individual to be inferred . In this paper , we consider the ability of an attacker to use data meeting privacy definitions to build an accurate classifier . We demonstrate that even under Differential Privacy , such classifiers can be used to infer `` private '' attributes accurately in realistic data . We compare this to similar approaches for inference-based attacks on other forms of anonymized data . We show how the efficacy of all these attacks can be measured on the same scale , based on the probability of successfully inferring a private attribute . We observe that the accuracy of inference of private attributes for differentially private data and $ l $ - diverse data can be quite similar .

O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Issues in evaluation of stream learning algorithms Learning from data streams is a research area of increasing importance . Nowadays , several stream learning algorithms have been developed . Most of them learn decision models that continuously evolve over time , run in resource-aware environments , detect and react to changes in the environment generating data . One important issue , not yet conveniently addressed , is the design of experimental work to evaluate and compare decision models that evolve over time . There are no golden standards for assessing performance in non-stationary environments . This paper proposes a general framework for assessing predictive stream learning algorithms . We defend the use of Predictive Sequential methods for error estimate - the prequential error . The prequential error allows us to monitor the evolution of the performance of models that evolve over time . Nevertheless , it is known to be a pessimistic estimator in comparison to holdout estimates . To obtain more reliable estimators we need some forgetting mechanism . Two viable alternatives are : sliding windows and fading factors . We observe that the prequential error converges to an holdout estimator when estimated over a sliding window or using fading factors . We present illustrative examples of the use of prequential error estimators , using fading factors , for the tasks of : i ) assessing performance of a learning algorithm ; ii ) comparing learning algorithms ; iii ) hypothesis testing using McNemar test ; and iv ) change detection using Page-Hinkley test . In these tasks , the prequential error estimated using fading factors provide reliable estimators . In comparison to sliding windows , fading factors are faster and memory-less , a requirement for streaming applications . This paper is a contribution to a discussion in the good-practices on performance assessment when learning dynamic models that evolve over time .

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Turning down the noise in the blogosphere In recent years , the blogosphere has experienced a substantial increase in the number of posts published daily , forcing users to cope with information overload . The task of guiding users through this flood of information has thus become critical . To address this issue , we present a principled approach for picking a set of posts that best covers the important stories in the blogosphere . We define a simple and elegant notion of coverage and formalize it as a submodular optimization problem , for which we can efficiently compute a near-optimal solution . In addition , since people have varied interests , the ideal coverage algorithm should incorporate user preferences in order to tailor the selected posts to individual tastes . We define the problem of learning a personalized coverage function by providing an appropriate user-interaction model and formalizing an online learning framework for this task . We then provide a no-regret algorithm which can quickly learn a user 's preferences from limited feedback . We evaluate our coverage and personalization algorithms extensively over real blog data . Results from a user study show that our simple coverage algorithm does as well as most popular blog aggregation sites , including Google Blog Search , Yahoo ! Buzz , and Digg . Furthermore , we demonstrate empirically that our algorithm can successfully adapt to user preferences . We believe that our technique , especially with personalization , can dramatically reduce information overload .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O


Community evolution in dynamic multi-mode networks A multi-mode network typically consists of multiple heterogeneous social actors among which various types of interactions could occur . Identifying communities in a multi-mode network can help understand the structural properties of the network , address the data shortage and unbalanced problems , and assist tasks like targeted marketing and finding influential actors within or between groups . In general , a network and the membership of groups often evolve gradually . In a dynamic multi-mode network , both actor membership and interactions can evolve , which poses a challenging problem of identifying community evolution . In this work , we try to address this issue by employing the temporal information to analyze a multi-mode network . A spectral framework and its scalability issue are carefully studied . Experiments on both synthetic data and real-world large scale networks demonstrate the efficacy of our algorithm and suggest its generality in solving problems with complex relationships .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dynamic heterogeneous network
dynamic network analysis
evolution
multi-mode networks


Growing decision trees on support-less association rules

database applications
decision support
decision tree


Extracting shared subspace for multi-label classification Multi-label problems arise in various domains such as multi-topic document categorization and protein function prediction . One natural way to deal with such problems is to construct a binary classifier for each label , resulting in a set of independent binary classification problems . Since the multiple labels share the same input space , and the semantics conveyed by different labels are usually correlated , it is essential to exploit the correlation information contained in different labels . In this paper , we consider a general framework for extracting shared structures in multi-label classification . In this framework , a common subspace is assumed to be shared among multiple labels . We show that the optimal solution to the proposed formulation can be obtained by solving a generalized eigenvalue problem , though the problem is non-convex . For high-dimensional problems , direct computation of the solution is expensive , and we develop an efficient algorithm for this case . One appealing feature of the proposed framework is that it includes several well-known algorithms as special cases , thus elucidating their intrinsic relationships . We have conducted extensive experiments on eleven multi-topic web page categorization tasks , and results demonstrate the effectiveness of the proposed formulation in comparison with several representative algorithms .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

least squares
multi-label classification


Similarity analysis on government regulations Government regulations are semi-structured text documents that are often voluminous , heavily cross-referenced between provisions and even ambiguous . Multiple sources of regulations lead to difficulties in both understanding and complying with all applicable codes . In this work , we propose a framework for regulation management and similarity analysis . An online repository for legal documents is created with the help of text mining tool , and users can access regulatory documents either through the natural hierarchy of provisions or from a taxonomy generated by knowledge engineers based on concepts . Our similarity analysis core identifies relevant provisions and brings them to the user 's attention , and this is performed by utilizing both the hierarchical and referential structures of regulations to provide a better comparison between provisions . Preliminary results show that our system reveals hidden similarities that are not apparent between provisions based on node content comparisons .

B I O O B O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

regulations
similarity analysis
text mining


Quantifying trends accurately despite classifier error and class imbalance This paper promotes a new task for supervised machine learning research : quantification - the pursuit of learning methods for accurately estimating the class distribution of a test set , with no concern for predictions on individual cases . A variant for cost quantification addresses the need to total up costs according to categories predicted by imperfect classifiers . These tasks cover a large and important family of applications that measure trends over time . The paper establishes a research methodology , and uses it to evaluate several proposed methods that involve selecting the classification threshold in a way that would spoil the accuracy of individual classifications . In empirical tests , Median Sweep methods show outstanding ability to estimate the class distribution , despite wide disparity in testing and training conditions . The paper addresses shifting class priors and costs , but not concept drift in general .

O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

classification
cost quantification
quantification


Fast collapsed gibbs sampling for latent dirichlet allocation In this paper we introduce a novel collapsed Gibbs sampling method for the widely used latent Dirichlet allocation ( LDA ) model . Our new method results in significant speedups on real world text corpora . Conventional Gibbs sampling schemes for LDA require O ( K ) operations per sample where K is the number of topics in the model . Our proposed method draws equivalent samples but requires on average significantly less then K operations per sample . On real-word corpora FastLDA can be as much as 8 times faster than the standard collapsed Gibbs sampler for LDA . No approximations are necessary , and we show that our fast sampling scheme produces exactly the same results as the standard ( but slower ) sampling scheme . Experiments on four real world data sets demonstrate speedups for a wide range of collection sizes . For the PubMed collection of over 8 million documents with a required computation time of 6 CPU months for LDA , our speedup of 5.7 can save 5 CPU months of computation .

O O O B O B I I O O O O O O O O O B O O O O O B I I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning sparse metrics via linear programming Calculation of object similarity , for example through a distance function , is a common part of data mining and machine learning algorithms . This calculation is crucial for efficiency since distances are usually evaluated a large number of times , the classical example being query-by-example ( find objects that are similar to a given query object ) . Moreover , the performance of these algorithms depends critically on choosing a good distance function . However , it is often the case that ( 1 ) the correct distance is unknown or chosen by hand , and ( 2 ) its calculation is computationally expensive ( e.g. , such as for large dimensional objects ) . In this paper , we propose a method for constructing relative-distance preserving low-dimensional mapping ( sparse mappings ) . This method allows learning unknown distance functions ( or approximating known functions ) with the additional property of reducing distance computation time . We present an algorithm that given examples of proximity comparisons among triples of objects ( object i is more like object j than object k ) , learns a distance function , in as few dimensions as possible , that preserves these distance relationships . The formulation is based on solving a linear programming optimization problem that finds an optimal mapping for the given dataset and distance relationships . Unlike other popular embedding algorithms , this method can easily generalize to new points , does not have local minima , and explicitly models computational efficiency by finding a mapping that is sparse , i.e. one that depends on a small subset of features or dimensions . Experimental evaluation shows that the proposed formulation compares favorably with a state-of-the art method in several publicly available datasets .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

convex optimization
dimensionality reduction
information search and retrieval
linear projections
metric learning
miscellaneous
relative distance constraints


Diagnosing memory leaks using graph mining on heap dumps Memory leaks are caused by software programs that prevent the reclamation of memory that is no longer in use . They can cause significant slowdowns , exhaustion of available storage space and , eventually , application crashes . Detecting memory leaks is challenging because real-world applications are built on multiple layers of software frameworks , making it difficult for a developer to know whether observed references to objects are legitimate or the cause of a leak . We present a graph mining solution to this problem wherein we analyze heap dumps to automatically identify subgraphs which could represent potential memory leak sources . Although heap dumps are commonly analyzed in existing heap profiling tools , our work is the first to apply a graph grammar mining solution to this problem . Unlike classical graph mining work , we show that it suffices to mine the dominator tree of the heap dump , which is significantly smaller than the underlying graph . Our approach identifies not just leaking candidates and their structure , but also provides aggregate information about the access path to the leaks . We demonstrate several synthetic as well as real-world examples of heap dumps for which our approach provides more insight into the problem than state-of-the-art tools such as Eclipse 's MAT .

O B I O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dominator tree
graph mining
heap profiling
memory leaks


Cross channel optimized marketing by reinforcement learning The issues of cross channel integration and customer life time value modeling are two of the most important topics surrounding customer relationship management ( CRM ) today . In the present paper , we describe and evaluate a novel solution that treats these two important issues in a unified framework of Markov Decision Processes ( MDP ) . In particular , we report on the results of a joint project between IBM Research and Saks Fifth Avenue to investigate the applicability of this technology to real world problems . The business problem we use as a testbed for our evaluation is that of optimizing direct mail campaign mailings for maximization of profits in the store channel . We identify a problem common to cross-channel CRM , which we call the Cross-Channel Challenge , due to the lack of explicit linking between the marketing actions taken in one channel and the customer responses obtained in another . We provide a solution for this problem based on old and new techniques in reinforcement learning . Our in-laboratory experimental evaluation using actual customer interaction data show that as much as 7 to 8 per cent increase in the store profits can be expected , by employing a mailing policy automatically generated by our methodology . These results confirm that our approach is valid in dealing with the cross channel CRM scenarios in the real world .

O O O O O O B O O O O O O O B I I I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O

cost sensitive learning
reinforcement learning
targeted marketing


Query , analysis , and visualization of hierarchically structured data using Polaris In the last several years , large OLAP databases have become common in a variety of applications such as corporate data warehouses and scientific computing . To support interactive analysis , many of these databases are augmented with hierarchical structures that provide meaningful levels of abstraction that can be leveraged by both the computer and analyst . This hierarchical structure generates many challenges and opportunities in the design of systems for the query , analysis , and visualization of these databases . In this paper , we present an interactive visual exploration tool that facilitates exploratory analysis of data warehouses with rich hierarchical structure , such as might be stored in data cubes . We base this tool on Polaris , a system for rapidly constructing table-based graphical displays of multidimensional databases . Polaris builds visualizations using an algebraic formalism derived from the interface and interpreted as a set of queries to a database . We extend the user interface , algebraic formalism , and generation of data queries in Polaris to expose and take advantage of hierarchical structure . In the resulting system , analysts can navigate through the hierarchical projections of a database , rapidly and incrementally generating visualizations for each projection .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Deriving quantitative models for correlation clusters Correlation clustering aims at grouping the data set into correlation clusters such that the objects in the same cluster exhibit a certain density and are all associated to a common arbitrarily oriented hyperplane of arbitrary dimensionality . Several algorithms for this task have been proposed recently . However , all algorithms only compute the partitioning of the data into clusters . This is only a first step in the pipeline of advanced data analysis and system modelling . The second ( post-clustering ) step of deriving a quantitative model for each correlation cluster has not been addressed so far . In this paper , we describe an original approach to handle this second step . We introduce a general method that can extract quantitative information on the linear dependencies within a correlation clustering . Our concepts are independent of the clustering model and can thus be applied as a post-processing step to any correlation clustering algorithm . Furthermore , we show how these quantitative models can be used to predict the probability distribution that an object is created by these models . Our broad experimental evaluation demonstrates the beneficial impact of our method on several applications of significant practical importance .

O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

cluster description
cluster model
correlation clustering
data mining


PaintingClass : interactive construction , visualization and exploration of decision trees Decision trees are commonly used for classification . We propose to use decision trees not just for classification but also for the wider purpose of knowledge discovery , because visualizing the decision tree can reveal much valuable information in the data . We introduce PaintingClass , a system for interactive construction , visualization and exploration of decision trees . PaintingClass provides an intuitive layout and convenient navigation of the decision tree . PaintingClass also provides the user the means to interactively construct the decision tree . Each node in the decision tree is displayed as a visual projection of the data . Through actual examples and comparison with other classification methods , we show that the user can effectively use PaintingClass to construct a decision tree and explore the decision tree to gain additional knowledge .

O O O O O O O O O B I B I O O O O B O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O

information visualization
interactive visualization
visual data mining


Exploiting duality in summarization with deterministic guarantees Summarization is an important task in data mining . A major challenge over the past years has been the efficient construction of fixed-space synopses that provide a deterministic quality guarantee , often expressed in terms of a maximum-error metric . Histograms and several hierarchical techniques have been proposed for this problem . However , their time and\/or space complexities remain impractically high and depend not only on the data set size n , but also on the space budget B. These handicaps stem from a requirement to tabulate all allocations of synopsis space to different regions of the data . In this paper we develop an alternative methodology that dispels these deficiencies , thanks to a fruitful application of the solution to the dual problem : given a maximum allowed error , determine the minimum-space synopsis that achieves it . Compared to the state-of-the-art , our histogram construction algorithm reduces time complexity by ( at least ) a Blog2n over logε \* factor and our hierarchical synopsis algorithm reduces the complexity by ( at least ) a factor of log2B over logε \* + logn in time and B ( 1-log B over log n ) in space , where ε \* is the optimal error . These complexity advantages offer both a space-efficiency and a scalability that previous approaches lacked . We verify the benefits of our approach in practice by experimentation .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fast window correlations over uncooperative time series Data arriving in time order ( a data stream ) arises in fields including physics , finance , medicine , and music , to name a few . Often the data comes from sensors ( in physics and medicine for example ) whose data rates continue to improve dramatically as sensor technology improves . Further , the number of sensors is increasing , so correlating data between sensors becomes ever more critical in order to distill knowlege from the data . In many applications such as finance , recent correlations are of far more interest than long-term correlation , so correlation over sliding windows ( windowed correlation ) is the desired operation . Fast response is desirable in many applications ( e.g. , to aim a telescope at an activity of interest or to perform a stock trade ) . These three factors -- data size , windowed correlation , and fast response -- motivate this work . Previous work ( 10 , 14 ) showed how to compute Pearson correlation using Fast Fourier Transforms and Wavelet transforms , but such techniques do n't work for time series in which the energy is spread over many frequency components , thus resembling white noise . For such `` uncooperative '' time series , this paper shows how to combine several simple techniques -- sketches ( random projections ) , convolution , structured random vectors , grid structures , and combinatorial design -- to achieve high performance windowed Pearson correlation over a variety of data sets .

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O


Towards parameter-free data mining Most data mining algorithms require the setting of many input parameters . Two main dangers of working with parameter-laden algorithms are the following . First , incorrect settings may cause an algorithm to fail in finding the true patterns . Second , a perhaps more insidious problem is that the algorithm may report spurious patterns that do not really exist , or greatly overestimate the significance of the reported patterns . This is especially likely when the user fails to understand the role of parameters in the data mining process . Data mining algorithms should have as few parameters as possible , ideally none . A parameter-free algorithm would limit our ability to impose our prejudices , expectations , and presumptions on the problem at hand , and would let the data itself speak to us . In this work , we show that recent results in bioinformatics and computational theory hold great promise for a parameter-free data-mining paradigm . The results are motivated by observations in Kolmogorov complexity theory . However , as a practical matter , they can be implemented using any off-the-shelf compression algorithm with the addition of just a dozen or so lines of code . We will show that this approach is competitive or superior to the state-of-the-art approaches in anomaly\/interestingness detection , classification , and clustering with empirical tests on time series\/DNA\/text \/ video datasets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O

anomaly detection
parameter-free data mining


Single-shot detection of multiple categories of text using parametric mixture models In this paper , we address the problem of detecting multiple topics or categories of text where each text is not assumed to belong to one of a number of mutually exclusive categories . Conventionally , the binary classification approach has been employed , in which whether or not text belongs to a category is judged by the binary classifier for every category . In this paper , we propose a more sophisticated approach to simultaneously detect multiple categories of text using parametric mixture models ( PMMs ) , newly presented in this paper . PMMs are probabilistic generative models for text that has multiple categories . Our PMMs are essentially different from the conventional mixture of multinomial distributions in the sense that in the former several basis multinomial parameters are mixed in the parameter space , while in the latter several multinomial components are mixed . We derive efficient learning algorithms for PMMs within the framework of the maximum a posteriori estimate . We also empirically show that our method can outperform the conventional binary approach when applied to multitopic detection of World Wide Web pages , focusing on those from the `` yahoo.com '' domain .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A framework for community identification in dynamic social networks We propose frameworks and algorithms for identifying communities in social networks that change over time . Communities are intuitively characterized as `` unusually densely knit '' subsets of a social network . This notion becomes more problematic if the social interactions change over time . Aggregating social networks over time can radically misrepresent the existing and changing community structure . Instead , we propose an optimization-based approach for modeling dynamic community structure . We prove that finding the most explanatory community structure is NP-hard and APX-hard , and propose algorithms based on dynamic programming , exhaustive search , maximum matching , and greedy heuristics . We demonstrate empirically that the heuristics trace developments of community structure accurately for several synthetic and real-world examples .

O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

community identification
dynamic social networks


Document preprocessing for naive Bayes classification and clustering with mixture of multinomials Naive Bayes classifier has long been used for text categorization tasks . Its sibling from the unsupervised world , the probabilistic mixture of multinomial models , has likewise been successfully applied to text clustering problems . Despite the strong independence assumptions that these models make , their attractiveness come from low computational cost , relatively low memory consumption , ability to handle heterogeneous features and multiple classes , and often competitiveness with the top of the line models . Recently , there has been several attempts to alleviate the problems of Naive Bayes by performing heuristic feature transformations , such as IDF , normalization by the length of the documents and taking the logarithms of the counts . We justify the use of these techniques and apply them to two problems : classification of products in Yahoo ! Shopping and clustering the vectors of collocated terms in user queries to Yahoo ! Search . The experimental evaluation allows us to draw conclusions about the promise that these transformations carry with regard to alleviating the strong assumptions of the multinomial model .

O O O B I B O B O B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

classification
clustering
mixture of multinomials
naive bayes


Cross domain distribution adaptation via kernel mapping When labeled examples are limited and difficult to obtain , transfer learning employs knowledge from a source domain to improve learning accuracy in the target domain . However , the assumption made by existing approaches , that the marginal and conditional probabilities are directly related between source and target domains , has limited applicability in either the original space or its linear transformations . To solve this problem , we propose an adaptive kernel approach that maps the marginal distribution of target-domain and source-domain data into a common kernel space , and utilize a sample selection strategy to draw conditional probabilities between the two domains closer . We formally show that under the kernel-mapping space , the difference in distributions between the two domains is bounded ; and the prediction error of the proposed approach can also be bounded . Experimental results demonstrate that the proposed method outperforms both traditional inductive classifiers and the state-of-the-art boosting-based transfer algorithms on most domains , including text categorization and web page ratings . In particular , it can achieve around 10 % higher accuracy than other approaches for the text categorization problem . The source code and datasets are available from the authors .

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

domain transfer
ensemble
generalization bound


Computer aided detection via asymmetric cascade of sparse hyperplane classifiers This paper describes a novel classification method for computer aided detection ( CAD ) that identifies structures of interest from medical images . CAD problems are challenging largely due to the following three characteristics . Typical CAD training data sets are large and extremely unbalanced between positive and negative classes . When searching for descriptive features , researchers often deploy a large set of experimental features , which consequently introduces irrelevant and redundant features . Finally , a CAD system has to satisfy stringent real-time requirements . This work is distinguished by three key contributions . The first is a cascade classification approach which is able to tackle all the above difficulties in a unified framework by employing an asymmetric cascade of sparse classifiers each trained to achieve high detection sensitivity and satisfactory false positive rates . The second is the incorporation of feature computational costs in a linear program formulation that allows the feature selection process to take into account different evaluation costs of various features . The third is a boosting algorithm derived from column generation optimization to effectively solve the proposed cascade linear programs . We apply the proposed approach to the problem of detecting lung nodules from helical multi-slice CT images . Our approach demonstrates superior performance in comparison against support vector machines , linear discriminant analysis and cascade AdaBoost . Especially , the resulting detection system is significantly sped up with our approach .

B I I O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O

computer aided detection
support vector machines

cascading classification
mathematical programming
miscellaneous
sparse solutions


Inverted matrix : efficient discovery of frequent items in large datasets in the context of interactive mining Existing association rule mining algorithms suffer from many problems when mining massive transactional datasets . One major problem is the high memory dependency : either the gigantic data structure built is assumed to fit in main memory , or the recursive mining process is too voracious in memory resources . Another major impediment is the repetitive and interactive nature of any knowledge discovery process . To tune parameters , many runs of the same algorithms are necessary leading to the building of these huge data structures time and again . This paper proposes a new disk-based association rule mining algorithm called Inverted Matrix , which achieves its efficiency by applying three new ideas . First , transactional data is converted into a new database layout called Inverted Matrix that prevents multiple scanning of the database during the mining phase , in which finding frequent patterns could be achieved in less than a full scan with random access . Second , for each frequent item , a relatively small independent tree is built summarizing co-occurrences . Finally , a simple and non-recursive mining process reduces the memory requirements as minimum candidacy generation and counting is needed . Experimental studies reveal that our Inverted Matrix approach outperform FP-Tree especially in mining very large transactional databases with a very large number of unique items . Our random access disk-based approach is particularly advantageous in a repetitive and interactive setting .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

association rules
cofi-tree
database applications
frequent patterns mining


Transforming classifier scores into accurate multiclass probability estimates Class membership probability estimates are important for many applications of data mining in which classification outputs are combined with other sources of information for decision-making , such as example-dependent misclassification costs , the outputs of other classifiers , or domain knowledge . Previous calibration methods apply only to two-class problems . Here , we show how to obtain accurate probability estimates for multiclass problems by combining calibrated binary probability estimates . We also propose a new method for obtaining calibrated two-class probability estimates that can be applied to any classifier that produces a ranking of examples . Using naive Bayes and support vector machine classifiers , we give experimental results from a variety of two-class and multiclass domains , including direct marketing , text categorization and digit recognition .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning and making decisions when costs and probabilities are both unknown In many data mining domains , misclassification costs are different for different examples , in the same way that class membership probabilities are example-dependent . In these domains , both costs and probabilities are unknown for test examples , so both cost estimators and probability estimators must be learned . After discussing how to make optimal decisions given cost and probability estimates , we present decision tree and naive Bayesian learning methods for obtaining well-calibrated probability estimates . We then explain how to obtain unbiased estimators for example-dependent costs , taking into account the difficulty that in general , probabilities and costs are not independent random variables , and the training examples for which costs are known are not representative of all examples . The latter problem is called sample selection bias in econometrics . Our solution to it is based on Nobel prize-winning work due to the economist James Heckman . We show that the methods we propose perform better than MetaCost and all other known methods , in a comprehensive experimental comparison that uses the well-known , large , and challenging dataset from the KDD '98 data mining contest .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


CFI-Stream : mining closed frequent itemsets in data streams Mining frequent closed itemsets provides complete and condensed information for non-redundant association rules generation . Extensive studies have been done on mining frequent closed itemsets , but they are mainly intended for traditional transaction databases and thus do not take data stream characteristics into consideration . In this paper , we propose a novel approach for mining closed frequent itemsets over data streams . It computes and maintains closed itemsets online and incrementally , and can output the current closed frequent itemsets in real time based on users ' specified thresholds . Experimental results show that our proposed method is both time and space efficient , has good scalability as the number of transactions processed increases and adapts very rapidly to the change in data streams .

O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O B I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

association rules
data stream
frequent closed itemsets


A parallel learning algorithm for text classification Text classification is the process of classifying documents into predefined categories based on their content . Existing supervised learning algorithms to automatically classify text need sufficient labeled documents to learn accurately . Applying the Expectation-Maximization ( EM ) algorithm to this problem is an alternative approach that utilizes a large pool of unlabeled documents to augment the available labeled documents . Unfortunately , the time needed to learn with these large unlabeled documents is too high . This paper introduces a novel parallel learning algorithm for text classification task . The parallel algorithm is based on the combination of the EM algorithm and the naive Bayes classifier . Our goal is to improve the computational time in learning and classifying process . We studied the performance of our parallel algorithm on a large Linux PC cluster called PIRUN Cluster . We report both timing and accuracy results . These results indicate that the proposed parallel algorithm is capable of handling large document collections .

O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Approximating a collection of frequent sets One of the most well-studied problems in data mining is computing the collection of frequent item sets in large transactional databases . One obstacle for the applicability of frequent-set mining is that the size of the output collection can be far too large to be carefully examined and understood by the users . Even restricting the output to the border of the frequent item-set collection does not help much in alleviating the problem . In this paper we address the issue of overwhelmingly large output size by introducing and studying the following problem : What are the k sets that best approximate a collection of frequent item sets ? Our measure of approximating a collection of sets by k sets is defined to be the size of the collection covered by the the k sets , i.e. , the part of the collection that is included in one of the k sets . We also specify a bound on the number of extra sets that are allowed to be covered . We examine different problem variants for which we demonstrate the hardness of the corresponding problems and we provide simple polynomial-time approximation algorithms . We give empirical evidence showing that the approximation methods work well in practice .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

foundations of data mining
mining frequent itemsets
nonnumerical algorithms and problems


LungCAD : a clinically approved , machine learning system for lung cancer detection We present LungCAD , a computer aided diagnosis ( CAD ) system that employs a classification algorithm for detecting solid pulmonary nodules from CT thorax studies . We briefly describe some of the machine learning techniques developed to overcome the real world challenges in this medical domain . The most significant hurdle in transitioning from a machine learning research prototype that performs well on an in-house dataset into a clinically deployable system , is the requirement that the CAD system be tested in a clinical trial . We describe the clinical trial in which LungCAD was tested : a large scale multi-reader , multi-case ( MRMC ) retrospective observational study to evaluate the effect of CAD in clinical practice for detecting solid pulmonary nodules from CT thorax studies . The clinical trial demonstrates that every radiologist that participated in the trial had a significantly greater accuracy with LungCAD , both for detecting nodules and identifying potentially actionable nodules ; this , along with other findings from the trial , has resulted in FDA approval for LungCAD in late 2006 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

computer aided detection
lung cancer prognosis
miscellaneous


Empirical Bayesian data mining for discovering patterns in post-marketing drug safety Because of practical limits in characterizing the safety profiles of therapeutic products prior to marketing , manufacturers and regulatory agencies perform post-marketing surveillance based on the collection of adverse reaction reports ( `` pharmacovigilance '' ) . The resulting databases , while rich in real-world information , are notoriously difficult to analyze using traditional techniques . Each report may involve multiple medicines , symptoms , and demographic factors , and there is no easily linked information on drug exposure in the reporting population . KDD techniques , such as association finding , are well-matched to the problem , but are difficult for medical staff to apply and interpret . To deploy KDD effectively for pharmacovigilance , Lincoln Technologies and GlaxoSmithKline collaborated to create a webbased safety data mining web environment . The analytical core is a high-performance implementation of the MGPS ( Multi-Item Gamma Poisson Shrinker ) algorithm described previously by DuMouchel and Pregibon , with several significant extensions and enhancements . The environment offers an interface for specifying data mining runs , a batch execution facility , tabular and graphical methods for exploring associations , and drilldown to case details . Substantial work was involved in preparing the raw adverse event data for mining , including harmonization of drug names and removal of duplicate reports . The environment can be used to explore both drug-event and multi-way associations ( interactions , syndromes ) . It has been used to study age\/gender effects , to predict the safety profiles of proposed combination drugs , and to separate contributions of individual drugs to safety problems in polytherapy situations .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

association rules
empirical bayes methods
post-marketing surveillance


BLOSOM : a framework for mining arbitrary boolean expressions We introduce a novel framework , called BLOSOM , for mining ( frequent ) boolean expressions over binary-valued datasets . We organize the space of boolean expressions into four categories : pure conjunctions , pure disjunctions , conjunction of disjunctions , and disjunction of conjunctions . We focus on mining the simplest expressions the minimal generators for each class . We also propose a closure operator for each class that yields closed boolean expressions . BLOSOM efficiently mines frequent boolean expressions by utilizing a number of methodical pruning techniques . Experiments showcase the behavior of BLOSOM , and an application study on a real dataset is also given .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

boolean expression
closed itemsets
minimal generator


Modeling relationships at multiple scales to improve accuracy of large recommender systems The collaborative filtering approach to recommender systems predicts user preferences for products or services by learning past user-item relationships . In this work , we propose novel algorithms for predicting user ratings of items by integrating complementary models that focus on patterns at different scales . At a local scale , we use a neighborhood-based technique that infers ratings from observed ratings by similar users or of similar items . Unlike previous local approaches , our method is based on a formal model that accounts for interactions within the neighborhood , leading to improved estimation quality . At a higher , regional , scale , we use SVD-like matrix factorization for recovering the major structural patterns in the user-item rating matrix . Unlike previous approaches that require imputations in order to fill in the unknown matrix entries , our new iterative algorithm avoids imputation . Because the models involve estimation of millions , or even billions , of parameters , shrinkage of estimated values to account for sampling variability proves crucial to prevent overfitting . Both the local and the regional approaches , and in particular their combination through a unifying model , compare favorably with other approaches and deliver substantially better results than the commercial Netflix Cinematch recommender system on a large publicly available data set .

O O O O O O O O O O B I O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

collaborative filtering
recommender systems


Anonymizing transaction databases for publication This paper considers the problem of publishing `` transaction data '' for research purposes . Each transaction is an arbitrary set of items chosen from a large universe . Detailed transaction data provides an electronic image of one 's life . This has two implications . One , transaction data are excellent candidates for data mining research . Two , use of transaction data would raise serious concerns over individual privacy . Therefore , before transaction data is released for data mining , it must be made anonymous so that data subjects can not be re-identified . The challenge is that transaction data has no structure and can be extremely high dimensional . Traditional anonymization methods lose too much information on such data . To date , there has been no satisfactory privacy notion and solution proposed for anonymizing transaction data . This paper proposes one way to address this issue .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

anonymity
data publishing
transaction database


A framework for ontology-driven subspace clustering Traditional clustering is a descriptive task that seeks to identify homogeneous groups of objects based on the values of their attributes . While domain knowledge is always the best way to justify clustering , few clustering algorithms have ever take domain knowledge into consideration . In this paper , the domain knowledge is represented by hierarchical ontology . We develop a framework by directly incorporating domain knowledge into clustering process , yielding a set of clusters with strong ontology implication . During the clustering process , ontology information is utilized to efficiently prune the exponential search space of the subspace clustering algorithms . Meanwhile , the algorithm generates automatical interpretation of the clustering result by mapping the natural hierarchical organized subspace clusters with significant categorical enrichment onto the ontology hierarchy . Our experiments on a set of gene expression data using gene ontology demonstrate that our pruning technique driven by ontology significantly improve the clustering performance with minimal degradation of the cluster quality . Meanwhile , many hierarchical organizations of gene clusters corresponding to a sub-hierarchies in gene ontology were also successfully captured .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O


DynaMMo : mining and summarization of coevolving sequences with missing values Given multiple time sequences with missing values , we propose DynaMMo which summarizes , compresses , and finds latent variables . The idea is to discover hidden variables and learn their dynamics , making our algorithm able to function even when there are missing values . We performed experiments on both real and synthetic datasets spanning several megabytes , including motion capture sequences and chlorine levels in drinking water . We show that our proposed DynaMMo method ( a ) can successfully learn the latent variables and their evolution ; ( b ) can provide high compression for little loss of reconstruction accuracy ; ( c ) can extract compact but powerful features for segmentation , interpretation , and forecasting ; ( d ) has complexity linear on the duration of sequences .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

bayesian network
expectation maximization
missing value
time series


Streaming feature selection using alpha-investing In Streaming Feature Selection ( SFS ) , new features are sequentially considered for addition to a predictive model . When the space of potential features is large , SFS offers many advantages over traditional feature selection methods , which assume that all features are known in advance . Features can be generated dynamically , focusing the search for new features on promising subspaces , and overfitting can be controlled by dynamically adjusting the threshold for adding features to the model . We describe α-investing , an adaptive complexity penalty method for SFS which dynamically adjusts the threshold on the error reduction required for adding a new feature . α-investing gives false discovery rate-style guarantees against overfitting . It differs from standard penalty methods such as AIC , BIC or RIC , which always drastically over - or under-fit in the limit of infinite numbers of non-predictive features . Empirical results show that SFS is competitive with much more compute-intensive feature selection methods such as stepwise regression , and allows feature selection on problems with over a million potential features .

O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O

classification
false discovery rate
multiple regression


Mining heterogeneous gene expression data with time lagged recurrent neural networks Heterogeneous types of gene expressions may provide a better insight into the biological role of gene interaction with the environment , disease development and drug effect at the molecular level . In this paper for both exploring and prediction purposes a Time Lagged Recurrent Neural Network with trajectory learning is proposed for identifying and classifying the gene functional patterns from the heterogeneous nonlinear time series microarray experiments . The proposed procedures identify gene functional patterns from the dynamics of a state-trajectory learned in the heterogeneous time series and the gradient information over time . Also , the trajectory learning with Back-propagation through time algorithm can recognize gene expression patterns vary over time . This may reveal much more information about the regulatory network underlying gene expressions . The analyzed data were extracted from spotted DNA microarrays in the budding yeast expression measurements , produced by Eisen et al. . The gene matrix contained 79 experiments over a variety of heterogeneous experiment conditions . The number of recognized gene patterns in our study ranged from two to ten and were divided into three cases . Optimal network architectures with different memory structures were selected based on Akaike and Bayesian information statistical criteria using two-way factorial design . The optimal model performance was compared to other popular gene classification algorithms such as Nearest Neighbor , Support Vector Machine , and Self-Organized Map . The reliability of the performance was verified with multiple iterated runs .

O B B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

backpropagation through time
gene expression
heterogeneous
trajectory learning

probabilistic algorithms
self-modifying machines
time lagged neural network


A general model for clustering binary data Clustering is the problem of identifying the distribution of patterns and intrinsic correlations in large data sets by partitioning the data points into similarity classes . This paper studies the problem of clustering binary data . This is the case for market basket datasets where the transactions contain items and for document datasets where the documents contain `` bag of words '' . The contribution of the paper is three-fold . First a general binary data clustering model is presented . The model treats the data and features equally , based on their symmetric association relations , and explicitly describes the data assignments as well as feature assignments . We characterize several variations with different optimization procedures for the general model . Second , we also establish the connections between our clustering model with other existing clustering methods . Third , we also discuss the problem for determining the number of clusters for binary clustering . Experimental results show the effectiveness of the proposed clustering model .

O B I O B B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O


Structured entity identification and document categorization : two tasks with one joint model Traditionally , research in identifying structured entities in documents has proceeded independently of document categorization research . In this paper , we observe that these two tasks have much to gain from each other . Apart from direct references to entities in a database , such as names of person entities , documents often also contain words that are correlated with discriminative entity attributes , such age-group and income-level of persons . This happens naturally in many enterprise domains such as CRM , Banking , etc. . Then , entity identification , which is typically vulnerable against noise and incompleteness in direct references to entities in documents , can benefit from document categorization with respect to such attributes . In return , entity identification enables documents to be categorized according to different label-sets arising from entity attributes without requiring any supervision . In this paper , we propose a probabilistic generative model for joint entity identification and document categorization . We show how the parameters of the model can be estimated using an EM algorithm in an unsupervised fashion . Using extensive experiments over real and semi-synthetic data , we demonstrate that the two tasks can benefit immensely from each other when performed jointly using the proposed model .

O B I O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

document categorization
entity identification
probabilistic generative model


Mining rank-correlated sets of numerical attributes We study the mining of interesting patterns in the presence of numerical attributes . Instead of the usual discretization methods , we propose the use of rank based measures to score the similarity of sets of numerical attributes . New support measures for numerical data are introduced , based on extensions of Kendall 's tau , and Spearman 's Footrule and rho . We show how these support measures are related . Furthermore , we introduce a novel type of pattern combining numerical and categorical attributes . We give efficient algorithms to find all frequent patterns for the proposed support measures , and evaluate their performance on real-life datasets .

O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O


Probabilistic query models for transaction data We investigate the application of Bayesian networks , Markov random fields , and mixture models to the problem of query answering for transaction data sets . We formulate two versions of the querying problem : the query selectivity estimation ( i.e. , finding exact counts for tuples in a data set ) and the query generalization problem ( i.e. , computing the probability that a tuple will occur in new data ) . We show that frequent itemsets are useful for reducing the original data to a compressed representation and introduce a method to store them using an ADTree data structure . In an extension of our earlier work on this topic we propose several new schemes for query answering based on the compressed representation that avoid direct scans of the data at query time . Experimental results on real-world transaction data sets provide insights into various tradeoffs involving the offline time for model-building , the online time for query-answering , the memory footprint of the compressed data , and the accuracy of the estimate provided to the query .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Integrating feature and instance selection for text classification Instance selection and feature selection are two orthogonal methods for reducing the amount and complexity of data . Feature selection aims at the reduction of redundant features in a dataset whereas instance selection aims at the reduction of the number of instances . So far , these two methods have mostly been considered in isolation . In this paper , we present a new algorithm , which we call FIS ( Feature and Instance Selection ) that targets both problems simultaneously in the context of text classificationOur experiments on the Reuters and 20-Newsgroups datasets show that FIS considerably reduces both the number of features and the number of instances . The accuracy of a range of classifiers including Naïve Bayes , TAN and LB considerably improves when using the FIS preprocessed datasets , matching and exceeding that of Support Vector Machines , which is currently considered to be one of the best text classification methods . In all cases the results are much better compared to Mutual Information based feature selection . The training and classification speed of all classifiers is also greatly improved .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Information theoretic regularization for semi-supervised boosting We present novel semi-supervised boosting algorithms that incrementally build linear combinations of weak classifiers through generic functional gradient descent using both labeled and unlabeled training data . Our approach is based on extending information regularization framework to boosting , bearing loss functions that combine log loss on labeled data with the information-theoretic measures to encode unlabeled data . Even though the information-theoretic regularization terms make the optimization non-convex , we propose simple sequential gradient descent optimization algorithms , and obtain impressively improved results on synthetic , benchmark and real world tasks over supervised boosting algorithms which use the labeled data alone and a state-of-the-art semi-supervised boosting algorithm .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

ensemble
learning
semi-supervised learning


Probabilistic latent semantic visualization : topic model for visualizing documents We propose a visualization method based on a topic model for discrete data such as documents . Unlike conventional visualization methods based on pairwise distances such as multi-dimensional scaling , we consider a mapping from the visualization space into the space of documents as a generative process of documents . In the model , both documents and topics are assumed to have latent coordinates in a two - or three-dimensional Euclidean space , or visualization space . The topic proportions of a document are determined by the distances between the document and the topics in the visualization space , and each word is drawn from one of the topics according to its topic proportions . A visualization , i.e. latent coordinates of documents , can be obtained by fitting the model to a given set of documents using the EM algorithm , resulting in documents with similar topics being embedded close together . We demonstrate the effectiveness of the proposed model by visualizing document and movie data sets , and quantitatively compare it with conventional visualization methods .

O O O B O B I O O O O O O B O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O


Connections between the lines : augmenting social networks with text Network data is ubiquitous , encoding collections of relationships between entities such as people , places , genes , or corporations . While many resources for networks of interesting entities are emerging , most of these can only annotate connections in a limited fashion . Although relationships between entities are rich , it is impractical to manually devise complete characterizations of these relationships for every pair of entities on large , real-world corpora . In this paper we present a novel probabilistic topic model to analyze text corpora and infer descriptions of its entities and of relationships between those entities . We develop variational methods for performing approximate inference on our model and demonstrate that our model can be practically deployed on large corpora such as Wikipedia . We show qualitatively and quantitatively that our model can construct and annotate graphs of relationships and make useful predictions .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graphical models
social network learning
statistical topic models


Learning the kernel matrix in discriminant analysis via quadratically constrained quadratic programming The kernel function plays a central role in kernel methods . In this paper , we consider the automated learning of the kernel matrix over a convex combination of pre-specified kernel matrices in Regularized Kernel Discriminant Analysis ( RKDA ) , which performs lineardiscriminant analysis in the feature space via the kernel trick . Previous studies have shown that this kernel learning problem can be formulated as a semidefinite program ( SDP ) , which is however computationally expensive , even with the recent advances in interior point methods . Based on the equivalence relationship between RKDA and least square problems in the binary-class case , we propose a Quadratically Constrained Quadratic Programming ( QCQP ) formulation for the kernel learning problem , which can be solved more efficiently than SDP . While most existing work on kernel learning deal with binary-class problems only , we show that our QCQP formulation can be extended naturally to the multi-class case . Experimental results on both binary-class and multi-class benchmarkdata sets show the efficacy of the proposed QCQP formulations .

O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Data mining in metric space : an empirical analysis of supervised learning performance criteria Many criteria can be used to evaluate the performance of supervised learning . Different criteria are appropriate in different settings , and it is not always clear which criteria to use . A further complication is that learning methods that perform well on one criterion may not perform well on other criteria . For example , SVMs and boosting are designed to optimize accuracy , whereas neural nets typically optimize squared error or cross entropy . We conducted an empirical study using a variety of learning methods ( SVMs , neural nets , k-nearest neighbor , bagged and boosted trees , and boosted stumps ) to compare nine boolean classification performance metrics : Accuracy , Lift , F-Score , Area under the ROC Curve , Average Precision , Precision\/Recall Break-Even Point , Squared Error , Cross Entropy , and Probability Calibration . Multidimensional scaling ( MDS ) shows that these metrics span a low dimensional manifold . The three metrics that are appropriate when predictions are interpreted as probabilities : squared error , cross entropy , and calibration , lay in one part of metric space far away from metrics that depend on the relative order of the predicted values : ROC area , average precision , break-even point , and lift . In between them fall two metrics that depend on comparing predictions to a threshold : accuracy and F-score . As expected , maximum margin methods such as SVMs and boosted trees have excellent performance on metrics like accuracy , but perform poorly on probability metrics such as squared error . What was not expected was that the margin methods have excellent performance on ordering metrics such as ROC area and average precision . We introduce a new metric , SAR , that combines squared error , accuracy , and ROC area into one metric . MDS and correlation analysis shows that SAR is centrally located and correlates well with other metrics , suggesting that it is a good general purpose metric to use when more specific criteria are not known .

O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O B O O O B O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O B O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O

cross entropy
lift
metrics
precision
roc
supervised learning


Center-piece subgraphs : problem definition and fast solutions Given Q nodes in a social network ( say , authorship network ) , how can we find the node\/author that is the center-piece , and has direct or indirect connections to all , or most of them ? For example , this node could be the common advisor , or someone who started the research area that the Q nodes belong to . Isomorphic scenarios appear in law enforcement ( find the master-mind criminal , connected to all current suspects ) , gene regulatory networks ( find the protein that participates in pathways with all or most of the given Q proteins ) , viral marketing and many more . Connection subgraphs is an important first step , handling the case of Q = 2 query nodes . Then , the connection subgraph algorithm finds the b intermediate nodes , that provide a good connection between the two original query nodes . Here we generalize the challenge in multiple dimensions : First , we allow more than two query nodes . Second , we allow a whole family of queries , ranging from ` OR ' to ` AND ' , with ` softAND ' in-between . Finally , we design and compare a fast approximation , and study the quality\/speed trade-off . We also present experiments on the DBLP dataset . The experiments confirm that our proposed method naturally deals with multi-source queries and that the resulting subgraphs agree with our intuition . Wall-clock timing results on the DBLP dataset show that our proposed approximation achieve good accuracy for about 6:1 speedup .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

center-piece subgraph
goodness score
kand