On the potential of domain literature for clustering and Bayesian network learning Thanks to its increasing availability , electronic literature can now be a major source of information when developing complex statistical models where data is scarce or contains much noise . This raises the question of how to integrate information from domain literature with statistical data . Because quantifying similarities or dependencies between variables is a basic building block in knowledge discovery , we consider here the following question . Which vector representations of text and which statistical scores of similarity or dependency support best the use of literature in statistical models ? For the text source , we assume to have annotations for the domain variables as short free-text descriptions and optionally to have a large literature repository from which we can further expand the annotations . For evaluation , we contrast the variables similarities or dependencies obtained from text using different annotation sources and vector representations with those obtained from measurement data or expert assessments . Specifically , we consider two learning problems : clustering and Bayesian network learning . Firstly , we report performance ( against an expert reference ) for clustering yeast genes from textual annotations . Secondly , we assess the agreement between text-based and data-based scores of variable dependencies when learning Bayesian network substructures for the task of modeling the joint distribution of clinical measurements of ovarian tumors .

O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

bayesian networks
data mining
text mining


Single-pass online learning : performance , voting schemes and online feature selection To learn concepts over massive data streams , it is essential to design inference and learning methods that operate in real time with limited memory . Online learning methods such as perceptron or Winnow are naturally suited to stream processing ; however , in practice multiple passes over the same training data are required to achieve accuracy comparable to state-of-the-art batch learners . In the current work we address the problem of training an on-line learner with a single passover the data . We evaluate several existing methods , and also propose a new modification of Margin Balanced Winnow , which has performance comparable to linear SVM . We also explore the effect of averaging , a.k.a. voting , on online learning . Finally , we describe how the new Modified Margin Balanced Winnow algorithm can be naturally adapted to perform feature selection . This scheme performs comparably to widely-used batch feature selection methods like information gain or Chi-square , with the advantage of being able to select features on-the-fly . Taken together , these techniques allow single-pass online learning to be competitive with batch techniques , and still maintain the advantages of on-line learning .

O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O


Towards scalable support vector machines using squashing

boosting
database applications
scalability


Frequent regular itemset mining Concise representations of frequent itemsets sacrifice readability and direct interpretability by a data analyst of the concise patterns extracted . In this paper , we introduce an extension of itemsets , called regular , with an immediate semantics and interpretability , and a conciseness comparable to closed itemsets . Regular itemsets allow for specifying that an item may or may not be present ; that any subset of an itemset may be present ; and that any non-empty subset of an itemset may be present . We devise a procedure , called RegularMine , for mining a set of regular itemsets that is a concise representation of frequent itemsets . The procedure computes a covering , in terms of regular itemsets , of the frequent itemsets in the class of equivalence of a closed one . We report experimental results on several standard dense and sparse datasets that validate the proposed approach .

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Cost-effective outbreak detection in networks Given a water distribution network , where should we place sensors toquickly detect contaminants ? Or , which blogs should we read to avoid missing important stories ? . These seemingly different problems share common structure : Outbreak detection can be modeled as selecting nodes ( sensor locations , blogs ) in a network , in order to detect the spreading of a virus or information asquickly as possible . We present a general methodology for near optimal sensor placement in these and related problems . We demonstrate that many realistic outbreak detection objectives ( e.g. , detection likelihood , population affected ) exhibit the property of `` submodularity '' . We exploit submodularity to develop an efficient algorithm that scales to large problems , achieving near optimal placements , while being 700 times faster than a simple greedy algorithm . We also derive online bounds on the quality of the placements obtained by any algorithm . Our algorithms and bounds also handle cases where nodes ( sensor locations , blogs ) have different costs . We evaluate our approach on several large real-world problems , including a model of a water distribution network from the EPA , andreal blog data . The obtained sensor placements are provably near optimal , providing a constant fraction of the optimal solution . We show that the approach scales , achieving speedups and savings in storage of several orders of magnitude . We also show how the approach leads to deeper insights in both applications , answering multicriteria trade-off , cost-sensitivity and generalization questions .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

graphs
information cascades
submodular functions
virus propagation


Probabilistic author-topic models for information discovery We propose a new unsupervised learning technique for extracting information from large text collections . We model documents as if they were generated by a two-stage stochastic process . Each author is represented by a probability distribution over topics , and each topic is represented as a probability distribution over words for that topic . The words in a multi-author paper are assumed to be the result of a mixture of each authors ' topic mixture . The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm . We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library , and learn a model with 300 topics . We discuss in detail the interpretation of the results discovered by the system including specific topic and author models , ranking of authors by topic and topics by author , significant trends in the computer science literature between 1990 and 2002 , parsing of abstracts by topics and authors and detection of unusual papers by specific authors . An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer .

O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

gibbs sampling
text modeling
unsupervised learning


Mining the network value of customers One of the major applications of data mining is in helping companies determine which potential customers to market to . If the expected profit from a customer is greater than the cost of marketing to her , the marketing action for that customer is executed . So far , work in this area has considered only the intrinsic value of the customer ( i. e , the expected profit from sales to her ) . We propose to model also the customer 's network value : the expected profit from sales to other customers she may influence to buy , the customers those may influence , and so on recursively . Instead of viewing a market as a set of independent entities , we view it as a social network and model it as a Markov random field . We show the advantages of this approach using a social network mined from a collaborative filtering database . Marketing that exploits the network value of customers -- also known as viral marketing -- can be extremely effective , but is still a black art . Our work can be viewed as a step towards providing a more solid foundation for it , taking advantage of the availability of large relevant databases .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dependency networks
direct marketing
markov random fields
social networks


Sleeved coclustering A coCluster of a m x n matrix X is a submatrix determined by a subset of the rows and a subset of the columns . The problem of finding coClusters with specific properties is of interest , in particular , in the analysis of microarray experiments . In that case the entries of the matrix X are the expression levels of $ m $ genes in each of $ n $ tissue samples . One goal of the analysis is to extract a subset of the samples and a subset of the genes , such that the expression levels of the chosen genes behave similarly across the subset of the samples , presumably reflecting an underlying regulatory mechanism governing the expression level of the genes . We propose to base the similarity of the genes in a coCluster on a simple biological model , in which the strength of the regulatory mechanism in sample j is Hj , and the response strength of gene i to the regulatory mechanism is Gi . In other words , every two genes participating in a good coCluster should have expression values in each of the participating samples , whose ratio is a constant depending only on the two genes . Noise in the expression levels of genes is taken into account by allowing a deviation from the model , measured by a relative error criterion . The sleeve-width of the coCluster reflects the extent to which entry i , j in the coCluster is allowed to deviate , relatively , from being expressed as the product GiHj . We present a polynomial-time Monte-Carlo algorithm which outputs a list of coClusters whose sleeve-widths do not exceed a prespecified value . Moreover , we prove that the list includes , with fixed probability , a coCluster which is near-optimal in its dimensions . Extensive experimentation with synthetic data shows that the algorithm performs well .

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

clustering
co-regulation
gene expression data


Mining the space of graph properties Existing data mining algorithms on graphs look for nodes satisfying specific properties , such as specific notions of structural similarity or specific measures of link-based importance . While such analyses for predetermined properties can be effective in well-understood domains , sometimes identifying an appropriate property for analysis can be a challenge , and focusing on a single property may neglect other important aspects of the data . In this paper , we develop a foundation for mining the properties themselves . We present a theoretical framework defining the space of graph properties , a variety of mining queries enabled by the framework , techniques to handle the enormous size of the query space , and an experimental system called F-Miner that demonstrates the utility and feasibility of property mining .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Scaling up dynamic time warping for datamining applications


Efficient incremental constrained clustering Clustering with constraints is an emerging area of data mining research . However , most work assumes that the constraints are given as one large batch . In this paper we explore the situation where the constraints are incrementally given . In this way the user after seeing a clustering can provide positive and negative feedback via constraints to critique a clustering solution . We consider the problem of efficiently updating a clustering to satisfy the new and old constraints rather than reclustering the entire data set . We show that the problem of incremental clustering under constraints is NP-hard in general , but identify several sufficient conditions which lead to efficiently solvable versions . These translate into a set of rules on the types of constraints thatcan be added and constraint set properties that must be maintained . We demonstrate that this approach is more efficient than re-clustering the entire data set and has several other advantages .

O O O B B O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O B O O O B O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Frequent term-based text clustering Text clustering methods can be used to structure large sets of text or hypertext documents . The well-known methods of text clustering , however , do not really address the special problems of text clustering : very high dimensionality of the data , very large size of the databases and understandability of the cluster description . In this paper , we introduce a novel approach which uses frequent item ( term ) sets for text clustering . Such frequent sets can be efficiently discovered using algorithms for association rule mining . To cluster based on frequent term sets , we measure the mutual overlap of frequent sets with respect to the sets of supporting documents . We present two algorithms for frequent term-based text clustering , FTC which creates flat clusterings and HFTC for hierarchical clustering . An experimental evaluation on classical text documents as well as on web documents demonstrates that the proposed algorithms obtain clusterings of comparable quality significantly more efficiently than state-of-the - art text clustering algorithms . Furthermore , our methods provide an understandable description of the discovered clusters by their frequent term sets .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fully automatic cross-associations Large , sparse binary matrices arise in numerous data mining applications , such as the analysis of market baskets , web graphs , social networks , co-citations , as well as information retrieval , collaborative filtering , sparse matrix reordering , etc. . Virtually all popular methods for the analysis of such matrices -- e.g. , k-means clustering , METIS graph partitioning , SVD\/PCA and frequent itemset mining -- require the user to specify various parameters , such as the number of clusters , number of principal components , number of partitions , and `` support . '' Choosing suitable values for such parameters is a challenging problem.Cross-association is a joint decomposition of a binary matrix into disjoint row and column groups such that the rectangular intersections of groups are homogeneous . Starting from first principles , we furnish a clear , information-theoretic criterion to choose a good cross-association as well as its parameters , namely , the number of row and column groups . We provide scalable algorithms to approach the optimal . Our algorithm is parameter-free , and requires no user intervention . In practice it scales linearly with the problem size , and is thus applicable to very large matrices . Finally , we present experiments on multiple synthetic and real-life datasets , where our method gives high-quality , intuitive results .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

content analysis and indexing
cross-association
mdl


Short term performance forecasting in enterprise systems We use data mining and machine learning techniques to predict upcoming periods of high utilization or poor performance in enterprise systems . The abundant data available and complexity of these systems defies human characterization or static models and makes the task suitable for data mining techniques . We formulate the problem as one of classification : given current and past information about the system 's behavior , can we forecast whether the system will meet its performance targets over the next hour ? Using real data gathered from several enterprise systems in Hewlett-Packard , we compare several approaches ranging from time series to Bayesian networks . Besides establishing the predictive power of these approaches our study analyzes three dimensions that are important for their application as a stand alone tool . First , it quantifies the gain in accuracy of multivariate prediction methods over simple statistical univariate methods . Second , it quantifies the variations in accuracy when using different classes of system and workload features . Third , it establishes that models induced using combined data from various systems generalize well and are applicable to new systems , enabling accurate predictions on systems with insufficient historical data . Together this analysis offers a promising outlook on the development of tools to automate assignment of resources to stabilize performance , ( e.g. , adding servers to a cluster ) and allow opportunistic job scheduling ( e.g. , backups or virus scans ) .

O O B I O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

enterprise systems
performance forecasting


Robust information-theoretic clustering How do we find a natural clustering of a real world point set , which contains an unknown number of clusters with different shapes , and which may be contaminated by noise ? Most clustering algorithms were designed with certain assumptions ( Gaussianity ) , they often require the user to give input parameters , and they are sensitive to noise . In this paper , we propose a robust framework for determining a natural clustering of a given data set , based on the minimum description length ( MDL ) principle . The proposed framework , Robust Information-theoretic Clustering ( RIC ) , is orthogonal to any known clustering algorithm : given a preliminary clustering , RIC purifies these clusters from noise , and adjusts the clusterings such that it simultaneously determines the most natural amount and shape ( subspace ) of the clusters . Our RIC method can be combined with any clustering technique ranging from K-means and K-medoids to advanced methods such as spectral clustering . In fact , RIC is even able to purify and improve an initial coarse clustering , even if we start with very simple methods such as grid-based space partitioning . Moreover , RIC scales well with the data set size . Extensive experiments on synthetic and real world data sets validate the proposed RIC framework .

O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data summarization
noise-robustness
parameter-free data mining


Distributed multivariate regression based on influential observations Large-scale data sets are sometimes logically and physically distributed in separate databases . The issues of mining these data sets are not just their sizes , but also the distributed nature . The complication is that communicating all the data to a central database would be too slow . To reduce communication costs , one could compress the data during transmission . Another method is random sampling . We propose an approach for distributed multivariate regression based on sampling and discuss its relationship with the compression method . The central idea is motivated by the observation that , although communication is limited , each individual site can still scan and process all the data it holds . Thus it is possible for the site to communicate only influential samples without seeing data in other sites . We exploit this observation and derive a method that provides tradeoff between communication cost and accuracy . Experimental results show that it is better than the compression method and random sampling .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

distributed data mining
learning curve
multivariate linear regression


Turning CARTwheels : an alternating algorithm for mining redescriptions We present an unusual algorithm involving classification trees -- CARTwheels -- where two trees are grown in opposite directions so that they are joined at their leaves . This approach finds application in a new data mining task we formulate , called redescription mining . A redescription is a shift-of-vocabulary , or a different way of communicating information about a given subset of data ; the goal of redescription mining is to find subsets of data that afford multiple descriptions . We highlight the importance of this problem in domains such as bioinformatics , which exhibit an underlying richness and diversity of data descriptors ( e.g. , genes can be studied in a variety of ways ) . CARTwheels exploits the duality between class partitions and path partitions in an induced classification tree to model and mine redescriptions . It helps integrate multiple forms of characterizing datasets , situates the knowledge gained from one dataset in the context of others , and harnesses high-level abstractions for uncovering cryptic and subtle features of data . Algorithm design decisions , implementation details , and experimental results are presented .

O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data mining in biological domains
learning


Joint latent topic models for text and citations In this work , we address the problem of joint modeling of text and citations in the topic modeling framework . We present two different models called the Pairwise-Link-LDA and the Link-PLSA-LDA models . The Pairwise-Link-LDA model combines the ideas of LDA ( 4 ) and Mixed Membership Block Stochastic Models ( 1 ) and allows modeling arbitrary link structure . However , the model is computationally expensive , since it involves modeling the presence or absence of a citation ( link ) between every pair of documents . The second model solves this problem by assuming that the link structure is a bipartite graph . As the name indicates , Link-PLSA-LDA model combines the LDA and PLSA models into a single graphical model . Our experiments on a subset of Citeseer data show that both these models are able to predict unseen data better than the baseline model of Erosheva and Lafferty ( 8 ) , by capturing the notion of topical similarity between the contents of the cited and citing documents . Our experiments on two different data sets on the link prediction task show that the Link-PLSA-LDA model performs the best on the citation prediction task , while also remaining highly scalable . In addition , we also present some interesting visualizations generated by each of the models .

O O B I O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

hyperlinks
influence
learning
variational inference


A GPU-tailored approach for training kernelized SVMs We present a method for efficiently training binary and multiclass kernelized SVMs on a Graphics Processing Unit ( GPU ) . Our methods apply to a broad range of kernels , including the popular Gaus - sian kernel , on datasets as large as the amount of available memory on the graphics card . Our approach is distinguished from earlier work in that it cleanly and efficiently handles sparse datasets through the use of a novel clustering technique . Our optimization algorithm is also specifically designed to take advantage of the graphics hardware . This leads to different algorithmic choices then those preferred in serial implementations . Our easy-to-use library is orders of magnitude faster then existing CPU libraries , and several times faster than prior GPU approaches .


Visualization of navigation patterns on a Web site using model-based clustering

data visualization
internet
model-based clustering
sequence clustering


Boosting with structure information in the functional space : an application to graph classification Boosting is a very successful classification algorithm that produces a linear combination of `` weak '' classifiers ( a.k.a. base learners ) to obtain high quality classification models . In this paper we propose a new boosting algorithm where base learners have structure relationships in the functional space . Though such relationships are generic , our work is particularly motivated by the emerging topic of pattern based classification for semi-structured data including graphs . Towards an efficient incorporation of the structure information , we have designed a general model where we use an undirected graph to capture the relationship of subgraph-based base learners . In our method , we combine both L1 norm and Laplacian based L2 norm penalty with Logit loss function of Logit Boost . In this approach , we enforce model sparsity and smoothness in the functional space spanned by the basis functions . We have derived efficient optimization algorithms based on coordinate decent for the new boosting formulation and theoretically prove that it exhibits a natural grouping effect for nearby spatial or overlapping features . Using comprehensive experimental study , we have demonstrated the effectiveness of the proposed learning methods .

B O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

feature selection
l1 regularization
semi-structured data


Efficient mining of weighted association rules ( WAR )


Discovering additive structure in black box functions Many automated learning procedures lack interpretability , operating effectively as a black box : providing a prediction tool but no explanation of the underlying dynamics that drive it . A common approach to interpretation is to plot the dependence of a learned function on one or two predictors . We present a method that seeks not to display the behavior of a function , but to evaluate the importance of non-additive interactions within any set of variables . Should the function be close to a sum of low dimensional components , these components can be viewed and even modeled parametrically . Alternatively , the work here provides an indication of where intrinsically high-dimensional behavior takes place . The calculations used in this paper correspond closely with the functional ANOVA decomposition ; a well-developed construction in Statistics . In particular , the proposed score of interaction importance measures the loss associated with the projection of the prediction function onto a space of additive models . The algorithm runs in linear time and we present displays of the output as a graphical model of the function for interpretation purposes .

O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O

additive models
functional anova
interpretation
learning

diagnostics
draphical models
feature selection
visualization


Using a knowledge cache for interactive discovery of association rules


k-TTP : a new privacy model for large-scale distributed environments Secure multiparty computation allows parties to jointly compute a function of their private inputs without revealing anything but the output . Theoretical results ( 2 ) provide a general construction of such protocols for any function . Protocols obtained in this way are , however , inefficient , and thus , practically speaking , useless when a large number of participants are involved . The contribution of this paper is to define a new privacy model -- k-privacy -- by means of an innovative , yet natural generalization of the accepted trusted third party model . This allows implementing cryptographically secure efficient primitives for real-world large-scale distributed systems . As an example for the usefulness of the proposed model , we employ k-privacy to introduce a technique for obtaining knowledge -- by way of an association-rule mining algorithm -- from large-scale Data Grids , while ensuring that the privacy is cryptographically secure .

O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O

association rule mining
distributed data mining
privacy-preserving data mining


Mining concept-drifting data streams using ensemble classifiers Recently , mining data streams with concept drifts for actionable insights has become an important and challenging task for a wide range of applications including credit card fraud protection , target marketing , network intrusion detection , etc. . Conventional knowledge discovery tools are facing two challenges , the overwhelming volume of the streaming data , and the concept drifts . In this paper , we propose a general framework for mining concept-drifting data streams using weighted ensemble classifiers . We train an ensemble of classification models , such as C4 .5 , RIPPER , naive Beyesian , etc. , from sequential chunks of the data stream . The classifiers in the ensemble are judiciously weighted based on their expected classification accuracy on the test data under the time-evolving environment . Thus , the ensemble approach improves both the efficiency in learning the model and the accuracy in performing classification . Our empirical study shows that the proposed methods have substantial advantage over single-classifier approaches in prediction accuracy , and the ensemble framework is effective for a variety of classification models .

O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

classifier
classifier ensemble
concept drift


Support feature machine for classification of abnormal brain activity In this study , a novel multidimensional time series classification technique , namely support feature machine ( SFM ) , is proposed . SFM is inspired by the optimization model of support vector machine and the nearest neighbor rule to incorporate both spatial and temporal of the multi-dimensional time series data . This paper also describes an application of SFM for detecting abnormal brain activity . Epilepsy is a case in point in this study . In epilepsy studies , electroencephalograms ( EEGs ) , acquired in multidimensional time series format , have been traditionally used as a gold-standard tool for capturing the electrical changes in the brain . From multi-dimensional EEG time series data , SFM was used to identify seizure pre-cursors and detect seizure susceptibility ( pre-seizure ) periods . The empirical results showed that SFM achieved over 80 % correct classification of per-seizure EEG on average in 10 patients using 5-fold cross validation . The proposed optimization model of SFM is very compact and scalable , and can be implemented as an online algorithm . The outcome of this study suggests that it is possible to construct a computerized algorithm used to detect seizure pre-cursors and warn of impending seizures through EEG classification .

O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

classification
epilepsy
nearest neighbor
optimization


Mining asynchronous periodic patterns in time series data

asynchronous periodic pattern
database applications
segment-based approach


An

association

analysis

approach

to

biclustering

The

discovery

of

biclusters

which

denote

groups

of

items

that

show

coherent

values

across

a

subset

of

all

the

transactions

in

a

data

set

is

an

important

type

of

analysis

performed

on

real-valued

data

sets

in

various

domains

such

as

biology

Several

algorithms

have

been

proposed

to

find

different

types

of

biclusters

in

such

data

sets

However

these

algorithms

are

unable

to

search

the

space

of

all

possible

biclusters

exhaustively

Pattern

mining

algorithms

in

association

analysis

also

essentially

produce

biclusters

as

their

result

since

the

patterns

consist

of

items

that

are

supported

by

a

subset

of

all

the

transactions

However

a

major

limitation

of

the

numerous

techniques

developed

in

association

analysis

is

that

they

are

only

able

to

analyze

data

sets

with

binary

and\/or

categorical

variables

and

their

application

to

real-valued

data

sets

often

involves

some

lossy

transformation

such

as

discretization

or

binarization

of

the

attributes

In

this

paper

we

propose

a

novel

association

analysis

framework

for

exhaustively

and

efficiently

mining

range

support

patterns

from

such

a

data

set

On

one

hand

this

framework

reduces

the

loss

of

information

incurred

by

the

binarization

and

discretization-based

approaches

and

on

the

other

it

enables

the

exhaustive

discovery

of

coherent

biclusters

We

compared

the

performance

of

our

framework

with

two

standard

biclustering

algorithms

through

the

evaluation

of

the

similarity

of

the

cellular

functions

of

the

genes

constituting

the

patterns\/biclusters

derived

by

these

algorithms

from

microarray

data

These

experiments

show

that

the

real-valued

patterns

discovered

by

our

framework

are

better

enriched

by

small

biologically

interesting

functional

classes

Also

through

specific

examples

we

demonstrate

the

ability

of

the

RAP

framework

to

discover

functionally

enriched

patterns

that

are

not

found

by

the

commonly

used

biclustering

algorithm

ISA

The

source

code

and

data

sets

used

in

this

paper

as

well

as

the

supplementary

material

are

available

at

http:\/\/www.cs.umn.edu\/vk\/gaurav\/rap

O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O

association analysis
biclustering
microarray data
range support


Evolutionary spectral clustering by incorporating temporal smoothness Evolutionary clustering is an emerging research area essential to important applications such as clustering dynamic Web and blog contents and clustering data streams . In evolutionary clustering , a good clustering result should fit the current data well , while simultaneously not deviate too dramatically from the recent history . To fulfill this dual purpose , a measure of temporal smoothness is integrated in the overall measure of clustering quality . In this paper , we propose two frameworks that incorporate temporal smoothness in evolutionary spectral clustering . For both frameworks , we start with intuitions gained from the well-known k-means clustering problem , and then propose and solve corresponding cost functions for the evolutionary spectral clustering problems . Our solutions to the evolutionary spectral clustering problems provide more stable and consistent clustering results that are less sensitive to short-term noises while at the same time are adaptive to long-term cluster drifts . Furthermore , we demonstrate that our methods provide the optimal solutions to the relaxed versions of the corresponding evolutionary k-means clustering problems . Performance experiments over a number of real and synthetic data sets illustrate our evolutionary spectral clustering methods provide more robust clustering results that are not sensitive to noise and can adapt to data drifts .

B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O

evolutionary spectral clustering
temporal smoothness

mining data streams
preserving cluster membership
treserving cluster quality


Distributed cooperative mining for information consortia We consider the situation where a number of agents are distributed and each of them collects a data sequence generated according to an unknown probability distribution . Here each of the distributions is specified by common parameters and individual parameters e.g. , a normal distribution with an identical mean and a different variance . Here we introduce a notion of an information consortium , which is a framework where the agents can not show raw data to one another , but they like to enjoy significant information gain for estimating the respective distributions . Such an information consortium has recently received much interest in a broad range of areas including financial risk management , ubiquitous network mining , etc. . In this paper we are concerned with the following three issues : 1 ) how to design a collaborative strategy for agents to estimate the respective distributions in the information consortium , 2 ) characterizing when each agent has a benefit in terms of information gain for estimating its distribution or information loss for predicting future data , and 3 ) charracterizing how much benefit each agent obtains . In this paper we yield a statistical formulation of information consortia and solve all of the above three problems for a general form of probability distributions . Specifically we propose a basic strategy for cooperative estimation and derive a necessary and sufficient condition for each agent to have a significant benefit .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O

collective mining
database applications
distributed mining
statistical model


Rotation invariant distance measures for trajectories For the discovery of similar patterns in 1D time-series , it is very typical to perform a normalization of the data ( for example a transformation so that the data follow a zero mean and unit standard deviation ) . Such transformations can reveal latent patterns and are very commonly used in datamining applications . However , when dealing with multidimensional time-series , which appear naturally in applications such as video-tracking , motion-capture etc , similar motion patterns can also be expressed at different orientations . It is therefore imperative to provide support for additional transformations , such as rotation . In this work , we transform the positional information of moving data , into a space that is translation , scale and rotation invariant . Our distance measure in the new space is able to detect elastic matches and can be efficiently lower bounded , thus being computationally tractable . The proposed methods are easy to implement , fast to compute and can have many applications for real world problems , in areas such as handwriting recognition and posture estimation in motion-capture data . Finally , we empirically demonstrate the accuracy and the efficiency of the technique , using real and synthetic handwriting data .

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Experimental design for solicitation campaigns Data mining techniques are routinely used by fundraisers to select those prospects from a large pool of candidates who are most likely to make a financial contribution . These techniques often rely on statistical models based on trial performance data . This trial performance data is typically obtained by soliciting a smaller sample of the possible prospect pool . Collecting this trial data involves a cost ; therefore the fundraiser is interested in keeping the trial size small while still collecting enough data to build a reliable statistical model that will be used to evaluate the remainder of the prospects . We describe an experimental design approach to optimally choose the trial prospects from an existing large pool of prospects . Prospects are clustered to render the problem practically tractable . We modify the standard D-optimality algorithm to prevent repeated selection of the same prospect cluster , since each prospect can only be solicited at most once . We assess the benefits of this approach on the KDD-98 data set by comparing the performance of the model based on the optimal trial data set with that of a model based on a randomly selected trial data set of equal size .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data collection
probability and statistics
solicitation campaign


Scalable look-ahead linear regression trees Most decision tree algorithms base their splitting decisions on a piecewise constant model . Often these splitting algorithms are extrapolated to trees with non-constant models at the leaf nodes . The motivation behind Look-ahead Linear Regression Trees ( LLRT ) is that out of all the methods proposed to date , there has been no scalable approach to exhaustively evaluate all possible models in the leaf nodes in order to obtain an optimal split . Using several optimizations , LLRT is able to generate and evaluate thousands of linear regression models per second . This allows for a near-exhaustive evaluation of all possible splits in a node , based on the quality of fit of linear regression models in the resulting branches . We decompose the calculation of the Residual Sum of Squares in such a way that a large part of it is pre-computed . The resulting method is highly scalable . We observe it to obtain high predictive accuracy for problems with strong mutual dependencies between attributes . We report on experiments with two simulated and seven real data sets .

O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

linear regression tree
model tree
predictive model
scalable algorithms


Using ghost edges for classification in sparsely labeled networks We address the problem of classification in partially labeled networks ( a.k.a. within-network classification ) where observed class labels are sparse . Techniques for statistical relational learning have been shown to perform well on network classification tasks by exploiting dependencies between class labels of neighboring nodes . However , relational classifiers can fail when unlabeled nodes have too few labeled neighbors to support learning ( during training phase ) and\/or inference ( during testing phase ) . This situation arises in real-world problems when observed labels are sparse . In this paper , we propose a novel approach to within-network classification that combines aspects of statistical relational learning and semi-supervised learning to improve classification performance in sparse networks . Our approach works by adding `` ghost edges '' to a network , which enable the flow of information from labeled to unlabeled nodes . Through experiments on real-world data sets , we demonstrate that our approach performs well across a range of conditions where existing approaches , such as collective classification and semi-supervised learning , fail . On all tasks , our approach improves area under the ROC curve ( AUC ) by up to 15 points over existing approaches . Furthermore , we demonstrate that our approach runs in time proportional to L • E , where L is the number of labeled nodes and E is the number of edges .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

collective classification
statistical relational learning


A streaming ensemble algorithm ( SEA ) for large-scale classification Ensemble methods have recently garnered a great deal of attention in the machine learning community . Techniques such as Boosting and Bagging have proven to be highly effective but require repeated resampling of the training data , making them inappropriate in a data mining context . The methods presented in this paper take advantage of plentiful data , building separate classifiers on sequential chunks of training points . These classifiers are combined into a fixed-size ensemble using a heuristic replacement strategy . The result is a fast algorithm for large-scale or streaming data that classifies as well as a single decision tree built on all the data , requires approximately constant memory , and adjusts quickly to concept drift .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O

database applications
ensemble classification


Handling concept drifts in incremental learning with support vector machines


Real-time ranking with concept drift using expert advice In many practical applications , one is interested in generating a ranked list of items using information mined from continuous streams of data . For example , in the context of computer networks , one might want to generate lists of nodes ranked according to their susceptibility to attack . In addition , real-world data streams often exhibit concept drift , making the learning task even more challenging . We present an online learning approach to ranking with concept drift , using weighted majority techniques . By continuously modeling different snapshots of the data and tuning our measure of belief in these models over time , we capture changes in the underlying concept and adapt our predictions accordingly . We measure the performance of our algorithm on real electricity data as well as asynthetic data stream , and demonstrate that our approach to ranking from stream data outperforms previously known batch-learning methods and other online methods that do not account for concept drift .

O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O B O O O O O O O O O B O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O

concept drift
data streams
learning
ranking


New ensemble methods for evolving data streams Advanced analysis of data streams is quickly becoming a key area of data mining research as the number of applications demanding such processing increases . Online mining when such data streams evolve over time , that is when concepts drift or change completely , is becoming one of the core issues . When tackling non-stationary concepts , ensembles of classifiers have several advantages over single classifier methods : they are easy to scale and parallelize , they can adapt to change quickly by pruning under-performing parts of the ensemble , and they therefore usually also generate more accurate concept descriptions . This paper proposes a new experimental data stream framework for studying concept drift , and two new variants of Bagging : ADWIN Bagging and Adaptive-Size Hoeffding Tree ( ASHT ) Bagging . Using the new experimental framework , an evaluation study on synthetic and real-world datasets comprising up to ten million examples shows that the new ensemble methods perform very well compared to several known methods .

O B I O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O

concept drift
data streams
ensemble methods


Event summarization for system management In system management applications , an overwhelming amount of data are generated and collected in the form of temporal events . While mining temporal event data to discover interesting and frequent patterns has obtained rapidly increasing research efforts , users of the applications are overwhelmed by the mining results . The extracted patterns are generally of large volume and hard to interpret , they may be of no emphasis , intricate and meaningless to non-experts , even to domain experts . While traditional research efforts focus on finding interesting patterns , in this paper , we take a novel approach called event summarization towards the understanding of the seemingly chaotic temporal data . Event summarization aims at providing a concise interpretation of the seemingly chaotic data , so that domain experts may take actions upon the summarized models . Event summarization decomposes the temporal information into many independent subsets and finds well fitted models to describe each subset .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O


Gaining insights into support vector machine pattern classifiers using projection-based tour methods This paper discusses visual methods that can be used to understand and interpret the results of classification using support vector machines ( SVM ) on data with continuous real-valued variables . SVM induction algorithms build pattern classifiers by identifying a maximal margin separating hyperplane from training examples in high dimensional pattern spaces or spaces induced by suitable nonlinear kernel transformations over pattern spaces . SVM have been demonstrated to be quite effective in a number of practical pattern classification tasks . Since the separating hyperplane is defined in terms of more than two variables it is necessary to use visual techniques that can navigate the viewer through high-dimensional spaces . We demonstrate the use of projection-based tour methods to gain useful insights into SVM classifiers with linear kernels on 8-dimensional data .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

dynamic graphics
machine leaning
multivariate data
tours
visualization


Mining IC test data to optimize VLSI testing

belief networks
decision support
decision theory
em algorithm
real-time control
unsupervised learning


Transforming data to satisfy privacy constraints Data on individuals and entities are being collected widely . These data can contain information that explicitly identifies the individual ( e.g. , social security number ) . Data can also contain other kinds of personal information ( e.g. , date of birth , zip code , gender ) that are potentially identifying when linked with other available data sets . Data are often shared for business or legal reasons . This paper addresses the important issue of preserving the anonymity of the individuals or entities during the data dissemination process . We explore preserving the anonymity by the use of generalizations and suppressions on the potentially identifying portions of the data . We extend earlier works in this area along various dimensions . First , satisfying privacy constraints is considered in conjunction with the usage for the data being disseminated . This allows us to optimize the process of preserving privacy for the specified usage . In particular , we investigate the privacy transformation in the context of data mining applications like building classification and regression models . Second , our work improves on previous approaches by allowing more flexible generalizations for the data . Lastly , this is combined with a more thorough exploration of the solution space using the genetic algorithm framework . These extensions allow us to transform the data so that they are more useful for their intended purpose while satisfying the privacy constraints .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data transformation
generalization
predictive modeling
suppression


Using retrieval measures to assess similarity in mining dynamic web clickstreams While scalable data mining methods are expected to cope with massive Web data , coping with evolving trends in noisy data in a continuous fashion , and without any unnecessary stoppages and reconfigurations is still an open challenge . This dynamic and single pass setting can be cast within the framework of mining evolving data streams . In this paper , we explore the task of mining mass user profiles by discovering evolving Web session clusters in a single pass with a recently proposed scalable immune based clustering approach ( TECNO-STREAMS ) , and study the effect of the choice of different similarity measures on the mining process and on the interpretation of the mined patterns . We propose a simple similarity measure that has the advantage of explicitly coupling the precision and coverage criteria to the early learning stages , and furthermore requiring that the affinity of the data to the learned profiles or summaries be defined by the minimum of their coverage or precision , hence requiring that the learned profiles are simultaneously precise and complete , with no compromises . In our experiments , we study the task of mining evolving user profiles from Web clickstream data ( web usage mining ) in a single pass , and under different trend sequencing scenarios , showing that compared oto the cosine similarity measure , the proposed similarity measure explicitly based on precision and coverage allows the discovery of more correct profiles at the same precision or recall quality levels .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

artificial immune systems
personalization
stream data mining
web mining


Suggesting friends using the implicit social graph Although users of online communication tools rarely categorize their contacts into groups such as `` family '' , `` co-workers '' , or `` jogging buddies '' , they nonetheless implicitly cluster contacts , by virtue of their interactions with them , forming implicit groups . In this paper , we describe the implicit social graph which is formed by users ' interactions with contacts and groups of contacts , and which is distinct from explicit social graphs in which users explicitly add other individuals as their `` friends '' . We introduce an interaction-based metric for estimating a user 's affinity to his contacts and groups . We then describe a novel friend suggestion algorithm that uses a user 's implicit social graph to generate a friend group , given a small seed set of contacts which the user has already labeled as friends . We show experimental results that demonstrate the importance of both implicit group relationships and interaction-based affinity ranking in suggesting friends . Finally , we discuss two applications of the Friend Suggest algorithm that have been released as Gmail Labs features .

O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

clustering
contact group clustering
group and organization interfaces
tie strength


Visualization support for a user-centered KDD process Viewing knowledge discovery as a user-centered process that requires an effective collaboration between the user and the discovery system , our work aims to support an active role of the user in that process by developing synergistic visualization tools integrated in our discovery system D2MS . These tools provide an ability of visualizing the entire process of knowledge discovery in order to help the user with data preprocessing , selecting mining algorithms and parameters , evaluating and comparing discovered models , and taking control of the whole discover process . Our case-studies with two medical datasets on meningitis and stomach cancer show that , with visualization tools in D2MS , the user gains better insight in each step of the knowledge discovery process as well the relationship between data and discovered knowledge .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O

data and knowledge visualization
model selection
representations
the user's active role


Mining , indexing , and querying historical spatiotemporal data In many applications that track and analyze spatiotemporal data , movements obey periodic patterns ; the objects follow the same routes ( approximately ) over regular time intervals . For example , people wake up at the same time and follow more or less the same route to their work everyday . The discovery of hidden periodic patterns in spatiotemporal data , apart from unveiling important information to the data analyst , can facilitate data management substantially . Based on this observation , we propose a framework that analyzes , manages , and queries object movements that follow such patterns . We define the spatiotemporal periodic pattern mining problem and propose an effective and fast mining algorithm for retrieving maximal periodic patterns . We also devise a novel , specialized index structure that can benefit from the discovered patterns to support more efficient execution of spatiotemporal queries . We evaluate our methods experimentally using datasets with object trajectories that exhibit periodicity .

O O B O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O

indexing
pattern mining
spatiotemporal data
trajectories


Entity discovery and assignment for opinion mining applications Opinion mining became an important topic of study in recent years due to its wide range of applications . There are also many companies offering opinion mining services . One problem that has not been studied so far is the assignment of entities that have been talked about in each sentence . Let us use forum discussions about products as an example to make the problem concrete . In a typical discussion post , the author may give opinions on multiple products and also compare them . The issue is how to detect what products have been talked about in each sentence . If the sentence contains the product names , they need to be identified . We call this problem entity discovery . If the product names are not explicitly mentioned in the sentence but are implied due to the use of pronouns and language conventions , we need to infer the products . We call this problem entity assignment . These problems are important because without knowing what products each sentence talks about the opinion mined from the sentence is of little use . In this paper , we study these problems and propose two effective methods to solve the problems . Entity discovery is based on pattern discovery and entity assignment is based on mining of comparative sentences . Experimental results using a large number of forum posts demonstrate the effectiveness of the technique . Our system has also been successfully tested in a commercial setting .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Detecting outliers using transduction and statistical testing Outlier detection can uncover malicious behavior in fields like intrusion detection and fraud analysis . Although there has been a significant amount of work in outlier detection , most of the algorithms proposed in the literature are based on a particular definition of outliers ( e.g. , density-based ) , and use ad-hoc thresholds to detect them . In this paper we present a novel technique to detect outliers with respect to an existing clustering model . However , the test can also be successfully utilized to recognize outliers when the clustering information is not available . Our method is based on Transductive Confidence Machines , which have been previously proposed as a mechanism to provide individual confidence measures on classification decisions . The test uses hypothesis testing to prove or disprove whether a point is fit to be in each of the clusters of the model . We experimentally demonstrate that the test is highly robust , and produces very few misdiagnosed points , even when no clustering information is available . Furthermore , our experiments demonstrate the robustness of our method under the circumstances of data contaminated by outliers . We finally show that our technique can be successfully applied to identify outliers in a noisy data set for which no information is available ( e.g. , ground truth , clustering structure , etc. ) . As such our proposed methodology is capable of bootstrapping from a noisy data set a clean one that can be used to identify future outliers .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Mining risk patterns in medical data In this paper , we discuss a problem of finding risk patterns in medical data . We define risk patterns by a statistical metric , relative risk , which has been widely used in epidemiological research . We characterise the problem of mining risk patterns as an optimal rule discovery problem . We study an anti-monotone property for mining optimal risk pattern sets and present an algorithm to make use of the property in risk pattern discovery . The method has been applied to a real world data set to find patterns associated with an allergic event for ACE inhibitors . The algorithm has generated some useful results for medical researchers .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

medical application
optimal risk pattern set


Measuring and extracting proximity in networks Measuring distance or some other form of proximity between objects is a standard data mining tool . Connection subgraphs were recently proposed as a way to demonstrate proximity between nodes in networks . We propose a new way of measuring and extracting proximity in networks called `` cycle free effective conductance '' ( CFEC ) . Our proximity measure can handle more than two endpoints , directed edges , is statistically well-behaved , and produces an effectiveness score for the computed subgraphs . We provide an efficien talgorithm . Also , we report experimental results and show examples for three large network data sets : a telecommunications calling graph , the IMDB actors graph , and an academic co-authorship network .

O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

connection subgraph
cycle-free escape probability
escape probability
proximity subgraph
random walks


Discovering significant OPSM subspace clusters in massive gene expression data Order-preserving submatrixes ( OPSMs ) have been accepted as a biologically meaningful subspace cluster model , capturing the general tendency of gene expressions across a subset of conditions . In an OPSM , the expression levels of all genes induce the same linear ordering of the conditions . OPSM mining is reducible to a special case of the sequential pattern mining problem , in which a pattern and its supporting sequences uniquely specify an OPSM cluster . Those small twig clusters , specified by long patterns with naturally low support , incur explosive computational costs and would be completely pruned off by most existing methods for massive datasets containing thousands of conditions and hundreds of thousands of genes , which are common in today 's gene expression analysis . However , it is in particular interest of biologists to reveal such small groups of genes that are tightly coregulated under many conditions , and some pathways or processes might require only two genes to act in concert . In this paper , we introduce the KiWi mining framework for massive datasets , that exploits two parameters k and w to provide a biased testing on a bounded number of candidates , substantially reducing the search space and problem scale , targeting on highly promising seeds that lead to significant clusters and twig clusters . Extensive biological and computational evaluations on real datasets demonstrate that KiWi can effectively mine biologically meaningful OPSM subspace clusters with good efficiency and scalability .

O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O

order-preserving submatrix
subspace clustering
twig cluster


Hierarchical topic segmentation of websites In this paper , we consider the problem of identifying and segmenting topically cohesive regions in the URL tree of a large website . Each page of the website is assumed to have a topic label or a distribution on topic labels generated using a standard classifier . We develop a set of cost measures characterizing the benefit accrued by introducing a segmentation of the site based on the topic labels . We propose a general framework to use these measures for describing the quality of a segmentation ; we also provide an efficient algorithm to find the best segmentation in this framework . Extensive experiments on human-labeled data confirm the soundness of our framework and suggest that a judicious choice of cost measures allows the algorithm to perform surprisingly accurate topical segmentations .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

classification
facility location
gain ratio
information search and retrieval
kl-distance
tree partitioning
website hierarchy
website segmentation


A unifying framework for detecting outliers and change points from non-stationary time series data We are concerned with the issues of outlier detection and change point detection from a data stream . In the area of data mining , there have been increased interest in these issues since the former is related to fraud detection , rare event discovery , etc. , while the latter is related to event\/trend by change detection , activity monitoring , etc. . Specifically , it is important to consider the situation where the data source is non-stationary , since the nature of data source may change over time in real applications . Although in most previous work outlier detection and change point detection have not been related explicitly , this paper presents a unifying framework for dealing with both of them on the basis of the theory of on-line learning of non-stationary time series . In this framework a probabilistic model of the data source is incrementally learned using an on-line discounting learning algorithm , which can track the changing data source adaptively by forgetting the effect of past data gradually . Then the score for any given data is calculated to measure its deviation from the learned model , with a higher score indicating a high possibility of being an outlier . Further change points in a data stream are detected by applying this scoring method into a time series of moving averaged losses for prediction using the learned model . Specifically we develop an efficient algorithms for on-line discounting learning of auto-regression models from time series data , and demonstrate the validity of our framework through simulation and experimental applications to stock market data analysis .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Is there a grand challenge or X-prize for data mining ? This panel will discuss possible exciting and motivating Grand Challenge problems for Data Mining , focusing on bioinformatics , multimedia mining , link mining , text mining , and web mining .

O O O B I O O O O O O O O O O O O O O B I O O O O O O O B O B I O B I O B I O O B I O

bioinformatics
grand challenge
link mining
multimedia mining
text mining
web mining


Truth discovery with multiple conflicting information providers on the web The world-wide web has become the most important information source for most of us . Unfortunately , there is no guarantee for the correctness of information on the web . Moreover , different web sites often provide conflicting information on a subject , such as different specifications for the same product . In this paper we propose a new problem called Veracity , i.e. , conformity to truth , which studies how to find true facts from a large amount of conflicting information on many subjects that is provided by various web sites . We design a general framework for the Veracity problem , and invent an algorithm called TruthFinder , which utilizes the relationships between web sites and their information , i.e. , a web site is trustworthy if it provides many pieces of true information , and a piece of information is likely to be true if it is provided by many trustworthy web sites . Our experiments show that TruthFinder successfully finds true facts among conflicting information , and identifies trustworthy web sites better than the popular search engines .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Discriminative topic modeling based on manifold learning Topic modeling has been popularly used for data analysis in various domains including text documents . Previous topic models , such as probabilistic Latent Semantic Analysis ( pLSA ) and Latent Dirichlet Allocation ( LDA ) , have shown impressive success in discovering low-rank hidden structures for modeling text documents . These models , however , do not take into account the manifold structure of data , which is generally informative for the non-linear dimensionality reduction mapping . More recent models , namely Laplacian PLSI ( LapPLSI ) and Locally-consistent Topic Model ( LTM ) , have incorporated the local manifold structure into topic models and have shown the resulting benefits . But these approaches fall short of the full discriminating power of manifold learning as they only enhance the proximity between the low-rank representations of neighboring pairs without any consideration for non-neighboring pairs . In this paper , we propose Discriminative Topic Model ( DTM ) that separates non-neighboring pairs from each other in addition to bringing neighboring pairs closer together , thereby preserving the global manifold structure as well as improving the local consistency . We also present a novel model fitting algorithm based on the generalized EM and the concept of Pareto improvement . As a result , DTM achieves higher classification performance in a semi-supervised setting by effectively exposing the manifold structure of data . We provide empirical evidence on text corpora to demonstrate the success of DTM in terms of classification accuracy and robustness to parameters compared to state-of-the-art techniques .

O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

document classification
general
semi-supervised learning


Graphs over time : densification laws , shrinking diameters and possible explanations How do real graphs evolve over time ? What are `` normal '' growth patterns in social , technological , and information networks ? Many studies have discovered patterns in static graphs , identifying properties in a single snapshot of a large network , or in a very small number of snapshots ; these include heavy tails for in - and out-degree distributions , communities , small-world phenomena , and others . However , given the lack of information about network evolution over long periods , it has been hard to convert these findings into statements about trends over time . Here we study a wide range of real graphs , and we observe some surprising phenomena . First , most of these graphs densify over time , with the number of edges growing super-linearly in the number of nodes . Second , the average distance between nodes often shrinks over time , in contrast to the conventional wisdom that such distance parameters should increase slowly as a function of the number of nodes ( like O ( log n ) or O ( log ( log n ) ) . Existing graph generation models do not exhibit these types of behavior , even at a qualitative level . We provide a new graph generator , based on a `` forest fire '' spreading process , that has a simple , intuitive justification , requires very few parameters ( like the `` flammability '' of nodes ) , and produces graphs exhibiting the full range of properties observed both in prior work and in the present study .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

densification power laws
graph generators
graph mining
heavy-tailed distributions
small-world phenomena


Algorithms for time series knowledge mining Temporal patterns composed of symbolic intervals are commonly formulated with Allen 's interval relations originating in temporal reasoning . This representation has severe disadvantages for knowledge discovery . The Time Series Knowledge Representation ( TSKR ) is a new hierarchical language for interval patterns expressing the temporal concepts of coincidence and partial order . We present effective and efficient mining algorithms for such patterns based on itemset techniques . A novel form of search space pruning effectively reduces the size of the mining result to ease interpretation and speed up the algorithms . On a real data set a concise set of TSKR patterns can explain the underlying temporal phenomena , whereas the patterns found with Allen 's relations are far more numerous yet only explain fragments of the data .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

interval patterns
knowledge discovery
time series


Using randomized response techniques for privacy-preserving data mining Privacy is an important issue in data mining and knowledge discovery . In this paper , we propose to use the randomized response techniques to conduct the data mining computation . Specially , we present a method to build decision tree classifiers from the disguised data . We conduct experiments to compare the accuracy of our decision tree with the one built from the original undisguised data . Our results show that although the data are disguised , our method can still achieve fairly high accuracy . We also show how the parameter used in the randomized response techniques affects the accuracy of the results .

O O O O O O B I B O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Hardening soft information sources


Logical-shapelets : an expressive primitive for time series classification Time series shapelets are small , local patterns in a time series that are highly predictive of a class and are thus very useful features for building classifiers and for certain visualization and summarization tasks . While shapelets were introduced only recently , they have already seen significant adoption and extension in the community . Despite their immense potential as a data mining primitive , there are two important limitations of shapelets . First , their expressiveness is limited to simple binary presence\/absence questions . Second , even though shapelets are computed offline , the time taken to compute them is significant . In this work , we address the latter problem by introducing a novel algorithm that finds shapelets in less time than current methods by an order of magnitude . Our algorithm is based on intelligent caching and reuse of computations , and the admissible pruning of the search space . Because our algorithm is so fast , it creates an opportunity to consider more expressive shapelet queries . In particular , we show for the first time an augmented shapelet representation that distinguishes the data based on conjunctions or disjunctions of shapelets . We call our novel representation Logical-Shapelets . We demonstrate the efficiency of our approach on the classic benchmark datasets used for these problems , and show several case studies where logical shapelets significantly outperform the original shapelet representation and other time series classification techniques . We demonstrate the utility of our ideas in domains as diverse as gesture recognition , robotics , and biometrics .

O O O O O O B I B B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O


Efficient algorithms for constructing decision trees with constraints

classification
decision support
decision tree


Accurate decision trees for mining high-speed data streams In this paper we study the problem of constructing accurate decision tree models from data streams . Data streams are incremental tasks that require incremental , online , and any-time learning algorithms . One of the most successful algorithms for mining data streams is VFDT . In this paper we extend the VFDT system in two directions : the ability to deal with continuous data and the use of more powerful classification techniques at tree leaves . The proposed system , VFDTc , can incorporate and classify new information online , with a single scan of the data , in time constant per example . The most relevant property of our system is the ability to obtain a performance similar to a standard decision tree algorithm even for medium size datasets . This is relevant due to the any-time property . We study the behavior of VFDTc in different problems and demonstrate its utility in large and medium data sets . Under a bias-variance analysis we observe that VFDTc in comparison to C4 .5 is able to reduce the variance component .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

decision support
disk-based algorithms
hoeffding bounds
incremental learning
subsampling


Spectral domain-transfer learning Traditional spectral classification has been proved to be effective in dealing with both labeled and unlabeled data when these data are from the same domain . In many real world applications , however , we wish to make use of the labeled data from one domain ( called in-domain ) to classify the unlabeled data in a different domain ( out-of-domain ) . This problem often happens when obtaining labeled data in one domain is difficult while there are plenty of labeled data from a related but different domain . In general , this is a transfer learning problem where we wish to classify the unlabeled data through the labeled data even though these data are not from the same domain . In this paper , we formulate this domain-transfer learning problem under a novel spectral classification framework , where the objective function is introduced to seek consistency between the in-domain supervision and the out-of-domain intrinsic structure . Through optimization of the cost function , the label information from the in-domain data is effectively transferred to help classify the unlabeled data from the out-of-domain . We conduct extensive experiments to evaluate our method and show that our algorithm achieves significant improvements on classification performance over many state-of-the-art algorithms .

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning optimal ranking with tensor factorization for tag recommendation Tag recommendation is the task of predicting a personalized list of tags for a user given an item . This is important for many websites with tagging capabilities like last . fm or delicious . In this paper , we propose a method for tag recommendation based on tensor factorization ( TF ) . In contrast to other TF methods like higher order singular value decomposition ( HOSVD ) , our method RTF ( ` ranking with tensor factorization ' ) directly optimizes the factorization model for the best personalized ranking . RTF handles missing values and learns from pairwise ranking constraints . Our optimization criterion for TF is motivated by a detailed analysis of the problem and of interpretation schemes for the observed data in tagging systems . In all , RTF directly optimizes for the actual problem using a correct interpretation of the data . We provide a gradient descent algorithm to solve our optimization problem . We also provide an improved learning and prediction method with runtime complexity analysis for RTF . The prediction runtime of RTF is independent of the number of observations and only depends on the factorization dimensions . Besides the theoretical analysis , we empirically show that our method outperforms other state-of-the-art tag recommendation methods like FolkRank , PageRank and HOSVD both in quality and prediction runtime .

O O B O B I O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O

ranking
tag recommendation
tensor factorization


An integrated framework on mining logs files for computing system management Traditional approaches to system management have been largely based on domain experts through a knowledge acquisition process that translates domain knowledge into operating rules and policies . This has been well known and experienced as a cumbersome , labor intensive , and error prone process . In addition , this process is difficult to keep up with the rapidly changing environments . In this paper , we will describe our research efforts on establishing an integrated framework for mining system log files for automatic management . In particular , we apply text mining techniques to categorize messages in log files into common situations , improve categorization accuracy by considering the temporal characteristics of log messages , develop temporal mining techniques to discover the relationships between different events , and utilize visualization tools to evaluate and validate the interesting temporal patterns for system management .

O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O

applications
event relationship
learning
log categorization
temporal pattern


Information awareness : a prospective technical assessment Recent proposals to apply data mining systems to problems in law enforcement , national security , and fraud detection have attracted both media attention and technical critiques of their expected accuracy and impact on privacy . Unfortunately , the majority of technical critiques have been based on simplistic assumptions about data , classifiers , inference procedures , and the overall architecture of such systems . We consider these critiques in detail , and we construct a simulation model that more closely matches realistic systems . We show how both the accuracy and privacy impact of a hypothetical system could be substantially improved , and we discuss the necessary and sufficient conditions for this improvement to be achieved . This analysis is neither a defense nor a critique of any particular system concept . Rather , our model suggests alternative technical designs that could mitigate some concerns , but also raises more specific conditions that must be met for such systems to be both accurate and socially desirable .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

collective classification
database applications
iterative classification
ranking classifiers
relational data mining
social network analysis
technology assessment
tia