Swimming in Social Media

remove accidentally inserted words from your spelling dictionary for emacs

2010-05-15T17:29:00.001-06:00

somehow, I accidentally inserted an incorrect word to my spelling dictionary in emacs. Wow, it took me a while to figure out where my private dictionary is.

Below are my only configuration script for ispell in emacs:
(global-set-key (kbd "") 'ispell-word)
(global-set-key (kbd "") 'ispell)

So i thought I am using ispell and noticed the following documenation for ispell:

Whenever ispell is started the file `.ispell_words' is read from
your home directory (if it exists). This file contains a list of
words, one per line. The order of the words is not important, but
the case is. Ispell will consider all of the words good, and will
use them as possible near misses.

Unfortunately, I didn't find such a file in my home directory. It turned out that I was using aspell in ispell mode. What the hell!

The private dictionary for aspell is located at
~/.aspell.en.pws

Finally solved the problem.

how to make backspace working for firefox in Ubuntu?

2010-04-07T22:31:00.002-06:00

http://embraceubuntu.com/2006/12/21/fix-firefox-backspace-to-take-you-to-the-previous-page/

Fix Firefox Backspace to Take You to the Previous Page December 21, 2006

Posted by Carthik in bugsquashing, snippets, ubuntu.
trackback

In a surprising development that seems really strange and unnecessary, Firefox 2.0 won’t go to the previous page when I press the “backspace” button on the keyboard. I have grown used to this over the period I have used Firefox. The fact that I can’t use backspace the way I am used to has been annoying me no end. So I decided to dig a little deeper.
The feature was removed to fix a bug. The bug that was caused by fixing the previous bug, which is that the backspace does not behave like it should has been fixed too (Thank heavens!)
But then, until the bug fix propagates to a firefox build available on Ubuntu, one has to resort to a little scratching to fix the matter. Here’s how you resurrect the backspace button in Firefox 2.0 (current as of this date):
Type “about:config” in the address bar of Firefox and press Enter.
`Filter` for ‘browser.backspace_action’ and change its value to 0 (zero).

matlab sparse matrix format

2009-12-01T12:32:00.003-07:00

I have encountered several cases to handle matlab sparse matrix in mex c or c++.

Here, I use a simple example to explain how the sparse matrix is stored in matlab (Very creative, but not necessarily for my usage. Hence, it is always convenient to understand the structure)

The sparse matrix format in matlab involve several components:
nz: the number of non-zeros
m: the number of rows
n: the number of cols

jc: the index of columns.
ir: the index of rows;
pr: the nonzero entries stored in a double vector.

The tricky part is the relation between jc, ir and pr.

For example, consider a 7-by-3 sparse mxArray named Sparrow containing six nonzero elements, created by typing:

Sparrow = zeros(7,3);
Sparrow(2,1) = 1;
Sparrow(5,1) = 1;
Sparrow(3,2) = 1;
Sparrow(2,3) = 2;
Sparrow(5,3) = 1;
Sparrow(6,3) = 1;
Sparrow = sparse(Sparrow);

Then, the matrix looks like below:

>> full(Sparrow)

ans =

0 0 0
1 0 2
0 1 0
0 0 0
1 0 1
0 0 1
0 0 0

Then

The contents of the ir, jc, and pr arrays are listed in this table.

Subscript	ir	pr	jc	Comment
`(2,1)`	`1`	`1`	`0`	Column 1 contains two nonzero elements, with rows designated by `ir[0]` and `ir[1]`
`(5,1)`	`4`	`1`	`2`	Column 2 contains one nonzero element, with row designated by `ir[2]`
`(3,2)`	`2`	`1`	`3`	Column 3 contains three nonzero elements, with rows designated by `ir[3]`,`ir[4]`, and `ir[5]`
`(2,3)`	`1`	`2`	`6`	There are six nonzero elements in all.
`(5,3)`	`4`	`1`
`(6,3)`	`5`	`1`

If the jth column of the sparse mxArray has any nonzero elements:

jc[j] is the index in ir, pr, and pi (if it exists) of the first nonzero element in the jth column.
jc[j+1]-1 is the index of the last nonzero element in the jth column.
For the jth column of the sparse matrix, jc[j] is the total number of nonzero elements in all preceding columns.

The number of nonzero elements in the jth column of the sparse mxArray is:

jc[j+1] - jc[j];

Note that the size of jc is n+1.
the size of ir is nz, the same as pr.

Hence, to iterate over the spare matrix in c, you can use the following code:

 for (col=0; col < n; col++){  
   startIndex = jc[col];  
   endIndex = jc[col+1];  
   for (i=startIndex; i < endIndex; i++){
     row = ir[i];
     val = pr[i];
     ......

   }

}

ICDM Trip Soon

2009-11-30T18:47:00.003-07:00

This Sunday, I will be out of town for ICDM conference held in Miami, FL.

I will present my paper:

Uncovering Groups via Heterogeneous Interaction Analysis.

This work addresses the community detection problem when multiple different types of interactions are presented between the same set of users.

A more general case is that users registered at different social media sites. Can we somehow uncover the hidden community structure?

We show that using an integration based on structural features is more robust. For evaluation, I proposed a simple cross-dimension network validation scheme. Similar to cross validation. This could be used as a simple rule for evaluation in the future.

Of course, there are many interesting directions to pursue in the future. One important aspect is that some of the dimensions are noisy. Is it possible to identify them? Is this the same as tensor decomposition?

about collective behavior

2009-11-18T11:29:00.008-07:00

Recently, I just submitted a magzine article talking about collective behavior to IEEE Intelligent systems based on my recent work published this year:
Relational Learning via Latent Social Dimensions KDD-2009
Scalable Learning of Collective Behavior based on Sparse Social Dimensions CIKM-2009.

So what is collective behavior? I define collective behavior as behaviors when individuals are exposed in a social network environment.

I found that different areas have quite different definitions.
(1) . According to wiki (mostly written by sociologist),"The term "collective behavior" was first used by Robert E. Park, and employed definitively by Herbert Blumer, to refer to social processes and events which do not reflect existing social structure (laws, conventions, and institutions), but which emerge in a "spontaneous" way. "Collective behavior can be divided into four categories:

1. The Crowd

2. The public

3. The mass

4. The social movement

It is subtle to differentiate these four terms based on the name.

(2) On the other hand, in other field, such as artificial life

Swarm Intelligence in Data Mining

collective behavior refers to a group of agents which can be treated as an entity, which is commonly observed in bird flocks, ants and other animals. There are several principles:

homogeneity (all the agents follow the same behavior model), locality (influenced only by neighbors), collision avoidance (avoid with nearby flock mates), velocity matching and flock centering.

Two popular methods are introduced in the chapter: particle swarm intelligence and ant colony optimization. quite interesting ideas :)

(3) In data mining field, there is one paper talking about learning from collective behavior.

The setup is like oracle: suppose you have the luxury to observe the interaction and corresponding actions of a group of people and each person in the group follow a fixed policy, then how can we learn the policy and strategies so that when new situations arrive, we can predict the collective behavior? The authors provide some theoretical bounds about the learnability. Unfortunatelly, evertying is synthetic and it is really difficult for me to figure out a proper scenario such that their setup might be true.

The nice part of this work is that at least two kinds of policies are studied:

one is mimic your friend
the other is try to differentiate from your neighbors

while the first one is well known, the 2nd strategy, as I believe existing in the real world, do not studied well in data mining or SNA. One way of achieving this is connecting those nodes that are two hops away and seperate them from 1-hop away neighbors as suggested in using ghost edges for sparsely labeled networks But when everyting is mixed, more work needs to be done.

Another key difference of this work from my article is that my article talks about "spatial" prediction of collective behavior (given some observations, predict the others within the same network), while this one talks about "temporal" prediction.

I think it is a great idea to combine these two aspects together.

(4) Adaptive networks and behavior. Concerning social networks and collective behavior, two directions are converging. One study the dynamics of networks, the other study the dynamics on networks. In reality, there two factors are evolving simultaneously. Social networks can evolve, so are collective behavior. How to capture these two factors in the modeling?

This also relates to social influence and social selection and several papers are talking about this issue:

Modeling the co-evolution of network and behavior

Nonequilibrium phase transition in the coevolution of networks and opinions

Feedback effects between similarity and social influence in online communities

(5) Collective Attention investigate how a news or a resource attracts users' attentions. Does networks come into play? This can also be a further direction for investigation.

Anyway, I feel that this direction has many more issues to address. Also many challenges such as problem formulation, data collection, and evaluation. More in the near future :)
Just check my homepage

Change default paper size in tetex in Linux

2009-11-16T22:38:00.001-07:00

Run command in sudo mode:

texconfig

Tips in research

2009-11-16T17:16:00.001-07:00

writing is like a hour glass.

Starts big, cover every aspects of the neck, and present a big picture.

Use conference deadlines to make yourself efficient.

Start from bad writing then improve!

www abstract due today

2009-10-26T17:31:00.000-06:00

Today is the abstract due. I have uploaded the abstract.

As I'm leaving on 29th, I need to finish the paper in 3 days!!!

This time, we did some interesting work rather than conventional data mining (mostly focus on cross-validation result, not interesting enough).

Hopefully we can make it!

Travel Plans this year

2009-10-25T17:08:00.000-06:00

10/30/ - 11/07, Hongkong, for CIKM.
Presenting "scalable learning of collective behavior based on sparse social dimensions"

12/06 - 12/09, Miami, FL for ICDM.
Presenting "uncovering groups via heterogeneous interaction analysis"

An interesting reading

2009-10-22T18:46:00.000-06:00

I came accross this interesing paper by a statistician Leo Breiman talking about the difference of statistics and machine learning. Basically, there are two school: one takes the data model, trying to understand the nature process (which is widely studied by statistician), the other one takes an algorithmic model. They do not care the process in the black box, but just try to approximate the process using whatever effective methods such as neural network and decisions trees (this is the philosophy of machine learning guys).

Some of the discusssions are very incisive. Leo argued that the over emphasize of data model might lead to wrong conclusions. As there could be many different but comparable models to achieve the same performance. Taking a machine learning approach seems more practical.

I am more interested in reading a paper by a machine learning guy talking about statistics.

Especially, for a newbie to work on machine learning, should he go to the statistics department or computer science department?

refeshing by writing papers

2009-10-14T15:42:00.000-06:00

I'm not sure whether this is a good symptom for reseachers. I feel excited when I write papers. This has been confirmed many times.

starting job search

2009-10-07T15:58:00.000-06:00

After submitting all the journal papers relevant to my past work, I'm now ready to look for jobs.

Go Go Go!

Hope I can find an ideal job :)

US visa still pending...

2009-09-27T12:53:00.000-06:00

As I am planning to go to HK to attend CIKM'09, I went to Mexico to renew my US visa two weeks ago. As expected, I got checked. But fortunatelly, I "slipped" back into US.

Until today, I still didn't get any update from the consulate about the visa.

Hope I will receive the good news next week!

determine number of clusters?

2009-09-25T15:01:00.001-06:00

At present, many community detection methods have been proposed. "Unfortunately", most of these methods require users to specify the number of clusters, which seems insane at first glimpse.

Imaging a conversation like this:

Person: powerful machine, can you tell me something about the data set I have?
Machine: You honored! Sure, but please tell me how many clusters are there.
Person: I do not know the data set. That's why I am asking you!
Machine: Sorry, I cannot tell you anything unless you tell me something.
Person: ....... (stupid machine!)
Machine: ...... (this user is an idiot about data mining!)

So when an algorithm claims that it automatically determines the number of clusters, everyone just embraces it. For instance, modularity is claimed to find the optimal number of clusters. So are Bayesian guys by enforcing a Chinese restaurant process as a prior. These methods, as I understand, is like setting a threshold for the clustering process or setting an objective to optimize. It basically tells a user that under this objective function, this number of clusters seems reasonable. Instead of setting the number of clusters, these algorithm implicitly set a parameter or an biased objective function to optimize.

But do we really need to determine the number of clusters?

If just for data exploration, I think a hierarchical organization of data objects are more reasonable. But it seems that current state of hierarchical clustering is too far away from satisfactory. Most methods just give you a binary tree, which does not reflect any interesting structure. Another disadvantage is high sensitivity of the hierarchical structure to the algorithm implementation and data processing order.

Instead of begging for machines to determine the number of communities, organization clusters of multi-resolutions is more interesting and more consistent with users' request.

IJCAI post-conference report

2009-08-19T17:58:00.000-06:00

This year, thanks to the support from IJCAI conference, I can travel to Pasadena to attend the IJCAI conference. This is the first time I attended IJCAI (I attended AAAI in 2005). AI motivated me to pursue my Ph.D study in the first place. I was quite excited to attend this top conference on AI.

Overall, I like the invited talks and demos most. AI is now so separated into different subfields and each field currently (e.g., machine learning and data mining) rarely talks much about intelligence. So these high-quality invited talks in IJCAI really gave me a fresh view of AI from different perspectives. However, the papers in the conference, I have to say, are not quite interesting. What is even worse, it is difficult to grasp any idea if you are not working directly in the field the paper is talking about. While it seems IJCAI motivates to encourage collaboration and cross-discipline talks between AI researchers, the barrier is heavier than ever before. This makes me worry about the quality of the IJCAI conference. How to attract high-quality work to publish in IJCAI and break the boundary between different areas require a lot of efforts, or even a revolutionary reform. Maybe, it is more reasonable to host IJCAI as a symposium consists of high-quality talks and pioneer work, rather than yet-another conference to publish delta papers.

This conference provides two tours for APL and USC. I think these tours were great and such kind of activities should be kept in the following conferences as they provide a great opportunity to explore classical AI projects. It was a little sad that some attendants reserved the seats but did not show up. On the contrary, some other people who wish to join the tour could not go because they had no ticket. I hope the ticket-exchange at the registration desk would be allowed to accommodate more people in the next IJCAI.

Another is concerning the organization of IJCAI. The sessions were well organized. But the catering was really poor. I attended the student reception. The food there was not attractive at all. Some other attendants were also complaining about the food. Maybe, the conference organizer spent most of the fundings to provide travel scholarship for students. Then, I'll take my words back given such a bad economic situation.

I would like to thank IJCAI to provide me this great opportunity to make friends with other AI researchers, to meet those AI “stars”, and to re-cherish those AI dreams I used to have. Hope this conference would become better and inspire more AI communications and explorations.

what's your chance of acceptance if you submit late?

2009-08-05T14:09:00.003-06:00

Just came across this question:

The same paper, which one would have a higher chance of being accepted in a conference?

Submit early? normal? or submit late?

Basically, would a larger submission number gives you a higher chance of being accepted?

I believe so.

Actually, as a reviewer and author, I have seen many people rushing their paper for a conference especially in CS. So a large submission number automatically hint that "the paper is rushed". So somehow the reviewer would have a low expectation. Then a good quality paper would stands out in the local region and is likely to get accepted.

Of course, this is just a small trick and no evidence can be found.

A simple way to avoid this bias is to reshuffle the paper when assigning them to the reviwers. Such that the paper's reviewer number has no correlation with the quality. Then, that seems a more fair process.

Just my 2 cents.

A special issue of network analysis on Science

2009-07-27T14:53:00.001-06:00

http://www.sciencemag.org/cgi/content/short/325/5939/405

Quite interesting special issue.

"It is not enough to look at patterns; we need to study how they evolve and change."

Recent Update

2009-06-27T23:47:00.000-06:00

Finished conference papers for ICDM and CIKM. (Still a paper of understanding groups in progress, probably submitting to WSDM)

Need to come back to the journal papers and thesis proposal now.

recent conferences to go

2009-05-28T16:03:00.001-06:00

I'm working on papers to submit to CIKM or ICDE.

I just checked both conferences, one requires 10 pages, and the other requires 12 pages.

The ridiculous thing is that once you are accepted as short paper, you have to reduce to 2 pages for CIKM, and 4 pages for ICDE.

some notes on professional writing

2009-05-16T15:50:00.000-06:00

1. should use punctuations in equations.
2. Avoid starting a sentence with a mathematical expression, seperate symbols by puncutation marks or words.
3. "the" is inapproprieate when the object referred is not unique or does not exist
4. write "the kth term", not "the k-th term"
5. two types of ellipsis, vertically centered and ground level. The latter is used between a list of symbols or to indicate a product. A_1...A_n. i = 1, 2, ..., n
6. Several commonly used abbreviations, e.g., i.e., et al.,

7. avoid using the adj or advs. very, rather, quite, nice, interesting (all these are imprecise). So is "essentially"

what's the point of 30% new material?

2009-04-22T11:17:00.003-06:00

I decide to convert my KDD08 paper into a journal version. Unfortunately, it is said that I have to add at least 30% new materials.

The sad thing is that I have wrote too much for the conference paper, since I want to make the paper more solid and comprehensive.

But it turns out I have nothing to add for the journal paper(Of course, you can always add something new but the cost is way too high, almost another journal paper).

So that's what I learned from this class:
Don't write too much in one conference paper.

It's really funny and ridiculous.

Avoid being yesterday's sucker

2008-11-30T19:38:00.003-07:00

I have been a sucker for a long while. Thanks to this miserable time, I, right now, start to read more than ever before. Though my reading speed is still too low considering the English context. However, this do give me insights and a hilarous time.

My goal, avoid being yesterday's sucker and do something solid!

some interesting ICML papers

2008-06-18T11:31:00.001-06:00

Maybe too much.

The GroupLASSO for Generalized Linear Models: Uniqueness of Solutions and Efficient Algorithms

Volker Roth and Bernd Fischer

The GroupLASSO method for finding important explanatory factors suffers from the potential non-uniqueness of solutions and also from high computational costs. We formulate conditions for the uniqueness of GroupLASSO solutions which lead to an easily implementable test procedure. In addition to merely detecting ambiguities in solutions, this testing procedure identifies all potentially active groups. These results are used to derive an efficient algorithm that can deal with input dimensions in the millions and can approximate the solution path efficiently. The derived methods are applied to large-scale learning problems where they exhibit excellent performance. We show that the proposed testing procedure helps to avoid misinterpretations of GroupLASSO solutions.

[Full paper]

Dirichlet Component Analysis: Feature Extraction for Compositional Data

Hua-Yan Wang, Qiang Yang, Hong Qin, and Hongbin Zha

We consider feature extraction (dimensionality reduction) for compositional data, where the data vectors are constrained to be positive and constant-sum. In real-world problems, the data components (variables) usually have complicated "correlations" while their total number is huge. Such scenario demands feature extraction. That is, we shall de-correlate the components and reduce their dimensionality. Traditional techniques such as the Principle Component Analysis (PCA) are not suitable for these problems due to unique statistical properties and the need to satisfy the constraints in compositional data. This paper presents a novel approach to feature extraction for compositional data. Our method first identifies a family of dimensionality reduction projections that preserve all relevant constraints, and then finds the optimal projection that maximizes the estimated Dirichlet precision on projected data. It reduces the compositional data to a given lower dimensionality while the components in the lower-dimensional space are de-correlated as much as possible. We develop theoretical foundation of our approach, and validate its effectiveness on some synthetic and real-world datasets.

[Full paper]

paper ID: 145

Pairwise Constraint Propagation by Semidefinite Programming for Semi-Supervised Classification

Zhenguo Li, Jianzhuang Liu, and Xiaoou Tang

We consider the general problem of learning from pairwise constraints and unlabeled data. The pairwise constraints specify whether two objects belong to the same class or not, known as the must-link constraints and the cannot-link constraints. We propose to learn a mapping that is smooth over the data graph and maps the data onto a unit hypersphere, where two must-link objects are mapped to the same point while two cannot-link objects are mapped to be orthogonal. We show that such a mapping can be achieved by formulating a semidefinite programming problem, which is convex and can be solved globally. Our approach can effectively propagate pairwise constraints to the whole data set. It can be directly applied to multi-class classification and can handle data labels, pairwise constraints, or a mixture of them in a unified framework. Promising experimental results are presented for classification tasks on a variety of synthetic and real data sets.

[Full paper]

Localized Multiple Kernel Learning

Mehmet Gonen and Ethem Alpaydin

Recently, instead of selecting a single kernel, multiple kernel learning (MKL) has been proposed which uses a convex combination of kernels, where the weight of each kernel is optimized during training. However, MKL assigns the same weight to a kernel over the whole input space. In this paper, we develop a localized multiple kernel learning (LMKL) algorithm using a gating model for selecting the appropriate kernel function locally. The localizing gating model and the kernel-based classifier are coupled and their optimization is done in a joint manner. Empirical results on ten benchmark and two bioinformatics data sets validate the applicability of our approach. LMKL achieves statistically similar accuracy results compared with MKL by storing fewer support vectors. LMKL can also combine multiple copies of the same kernel function localized in different parts. For example, LMKL with multiple linear kernels gives better accuracy results than using a single linear kernel on bioinformatics data sets.

[Full paper]

Uncorrelated Multilinear Principal Component Analysis through Successive Variance Maximization

Haiping Lu, Konstantinos Plataniotis, and Anastasios Venetsanopoulos

Tensorial data are frequently encountered in various machine learning tasks today and dimensionality reduction is one of their most important applications. This paper extends the classical principal component analysis (PCA) to its multilinear version by proposing a novel dimensionality reduction algorithm for tensorial data, named as uncorrelated multilinear PCA (UMPCA). UMPCA seeks a tensor-to-vector projection that captures most of the variation in the original tensorial input while producing uncorrelated features through successive variance maximization. We evaluate the proposed algorithm on a second-order tensorial problem, face recognition, and the experimental results show its superiority, especially in low-dimensional spaces, through the comparison with three other PCA-based algorithms.

[Full paper]

A Dual Coordinate Descent Method for Large-scale Linear SVM

Cho-Jui Hsieh, Kai-Wei Chang, Chih-Jen Lin, S. Sathiya Keerthi, and S. Sundararajan

In many applications, data appear with a huge number of instances as well as features. Linear Support Vector Machines (SVM) is one of the most popular tools to deal with such large-scale sparse data. This paper presents a novel dual coordinate descent method for linear SVM with L1- and L2-loss functions. The proposed method is simple and reaches an epsilon-accurate solution in O(log (1/epsilon)) iterations. Experiments indicate that our method is much faster than state of the art solvers such as Pegasos, Tron, svmperf, and a recent primal coordinate descent implementation.

[Full paper]

Efficient MultiClass Maximum Margin Clustering

Bin Zhao, Fei Wang, and Changshui Zhang

This paper presents a cutting plane algorithm for multiclass maximum margin clustering (MMC). The proposed algorithm constructs a nested sequence of successively tighter relaxations of the original MMC problem, and each optimization problem in this sequence could be efficiently solved using the constrained concave-convex procedure (CCCP). Experimental evaluations on several real world datasets show that our algorithm converges much faster than existing MMC methods with guaranteed accuracy, and can thus handle much larger datasets efficiently.

[Full paper]

Nearest Hyperdisk Methods for High-Dimensional Classification

Hakan Cevikalp, Bill Triggs, and Robi Polikar

In high-dimensional classification problems it is infeasible to include enough training samples to cover the class regions densely. Irregularities in the resulting sparse sample distributions cause local classifiers such as Nearest Neighbors (NN) and kernel methods to have irregular decision boundaries. One solution is to "fill in the holes" by building a convex model of the region spanned by the training samples of each class and classifying examples based on their distances to these approximate models. Methods of this kind based on affine and convex hulls and bounding hyperspheres have already been studied. Here we propose a method based on the bounding hyperdisk of each class -- the intersection of the affine hull and the smallest bounding hypersphere of its training samples. We argue that in many cases hyperdisks are preferable to affine and convex hulls and hyperspheres: they bound the classes more tightly than affine hulls or hyperspheres while avoiding much of the sample overfitting and computational complexity that is inherent in high-dimensional convex hulls. We show that the hyperdisk method can be kernelized to provide nonlinear classifiers based on non-Euclidean distance metrics. Experiments on several classification problems show promising results.

[Full paper]

Local Likelihood Modeling of Temporal Text Streams

Guy Lebanon and Yang Zhao

Temporal text data is often generated by a time-changing process or distribution. Such a drift in the underlying distribution cannot be captured by stationary likelihood techniques. We consider the application of local likelihood methods to generative and conditional modeling of temporal document sequences. We examine the asymptotic bias and variance and present an experimental study using the RCV1 dataset containing a temporal sequence of Reuters news stories.

[Full paper]

Learning to Classify with Missing and Corrupted Features

Ofer Dekel and Ohad Shamir

After a classifier is trained using a machine learning algorithm and put to use in a real world system, it often faces noise which did not appear in the training data. Particularly, some subset of features may be missing or may become corrupted. We present two novel machine learning techniques that are robust to this type of classification-time noise. First, we solve an approximation to the learning problem using linear programming. We analyze the tightness of our approximation and prove statistical risk bounds for this approach. Second, we define the online-learning variant of our problem, address this variant using a modified Perceptron, and obtain a statistical learning algorithm using an online-to-batch technique. We conclude with a set of experiments that demonstrate the effectiveness of our algorithms.

[Full paper]

Fast Solvers and Efficient Implementations for Distance Metric Learning

Kilian Weinberger and Lawrence Saul

In this paper we study how to improve nearest neighbor classification by learning a Mahalanobis distance metric. We build on a recently proposed framework for distance metric learning known as large margin nearest neighbor (LMNN) classification. Within this framework, we focus specifically on the challenges in scalability and adaptability posed by large data sets. Our paper makes three contributions. First, we describe a highly efficient solver for the particular instance of semidefinite programming that arises in LMNN classification; our solver can handle problems with billions of large margin constraints in a few hours. Second, we show how to reduce both training and testing times using metric ball trees; the speedups from ball trees are further magnified by learning low dimensional representations of the input space. Third, we show how to learn different Mahalanobis distance metrics in different parts of the input space. For large data sets, these mixtures of locally adaptive metrics lead to even lower error rates.

[Full paper]

Random Classification Noise Defeats All Convex Potential Boosters

Philip M. Long and Rocco A. Servedio

A broad class of boosting algorithms can be interpreted as performing coordinate-wise gradient descent to minimize some potential function of the margins of a data set. This class includes AdaBoost, LogitBoost, and other widely used and well-studied boosters. In this paper we show that for a broad class of convex potential functions, any such boosting algorithm is highly susceptible to random classification noise. We do this by showing that for any such booster and any nonzero random classification noise rate R, there is a simple data set of examples which is efficiently learnable by such a booster if there is no noise, but which cannot be learned to accuracy better than 1/2 if there is random classification noise at rate R. This negative result is in contrast with known branching program based boosters which do not fall into the convex potential function framework and which can provably learn to high accuracy in the presence of random classification noise.

[Full paper]

Learning Diverse Rankings with Multi-Armed Bandits

Filip Radlinski, Robert Kleinberg, and Thorsten Joachims

Algorithms for learning to rank Web documents usually assume a document's relevance is independent of other documents. This leads to learned ranking functions that produce rankings with redundant results. In contrast, user studies have shown that diversity at high ranks is often preferred. We present two new learning algorithms that directly learn a diverse ranking of documents based on users' clicking behavior. We show that these algorithms minimize abandonment, or alternatively, maximize the probability that a relevant document is found in the top k positions of a ranking. We show that one of our algorithms asymptotically achieves the best possible payoff obtainable in polynomial time even as user's interests change. The other performs better empirically when user interests are static, and is still theoretically near-optimal in that case.

[Full paper]

SVM Optimization: Inverse Dependence on Training Set Size

Shai Shalev-Shwartz and Nathan Srebro

We discuss how the runtime of SVM optimization should decrease as the size of the training data increases. We present theoretical and empirical results demonstrating how a simple subgradient descent approach indeed displays such behavior, at least for linear kernels.

[Full paper]

Training Structural SVMs when Exact Inference is Intractable

Thomas Finley and Thorsten Joachims

While discriminative training (e.g., CRF, structural SVM) holds much promise for machine translation, image segmentation, and clustering, the complex inference these applications require make exact training intractable. This leads to a need for approximate training methods. Unfortunately, knowledge about how to perform efficient and effective approximate training is limited. Focusing on structural SVMs, we provide and explore algorithms for two different classes of approximate training algorithms, which we call undergenerating (e.g., greedy) and overgenerating (e.g., relaxations) algorithms. We provide a theoretical and empirical analysis of both types of approximate trained structural SVMs, focusing on fully connected pairwise Markov random fields. We find that models trained with overgenerating methods have theoretic advantages over undergenerating methods, are empirically robust relative to their undergenerating brethren, and relaxed trained models favor non-fractional predictions from relaxed predictors.

[Full paper]

Graph Transduction via Alternating Minimization

Jun Wang, Tony Jebara, and Shih-Fu Chang

Graph transduction methods label input data by learning a classification function that is regularized to exhibit smoothness along a graph over labeled and unlabeled samples. In practice, these algorithms are sensitive to the initial set of labels provided by the user. For instance, classification accuracy drops if the training set contains weak labels, if imbalances exist across label classes or if the labeled portion of the data is not chosen at random. This paper introduces a propagation algorithm that more reliably minimizes a cost function over both a function on the graph and a binary label matrix. The cost function generalizes prior work in graph transduction and also introduces node normalization terms for resilience to label imbalances. We demonstrate that global minimization of the function is intractable but instead provide an alternating minimization scheme that incrementally adjusts the function and the labels towards a reliable local minimum. Unlike prior methods, the resulting propagation of labels does not prematurely commit to an erroneous labeling and obtains more consistent labels. Experiments are shown for synthetic and real classification tasks including digit and text recognition. A substantial improvement in accuracy compared to state of the art semi-supervised methods is achieved. The advantage are even more dramatic when labeled instances are limited.

[Full paper]

Grassmann Discriminant Analysis: a Unifying View on Subspace-Based Learning

Jihun Hamm and Daniel Lee

In this paper we propose a discriminant learning framework for problems in which data consist of linear subspaces instead of vectors. By treating subspaces as basic elements, we can make learning algorithms adapt naturally to the problems with linear invariant structures. We propose a unifying view on the subspace-based learning method by formulating the problems on the Grassmann manifold, which is the set of fixed-dimensional subspaces of a Euclidean space. Previous methods on the problem typically adopt an inconsistent strategy: feature extraction is performed in the Euclidean space while non-Euclidean dissimilarity measures are used. In our approach, we treat each subspace as a point in the Grassmann space, and perform feature extraction and classification in the same space. We show feasibility of the approach by using the Grassmann kernel functions such as the Projection kernel and the Binet-Cauchy kernel. Experiments with real image databases show that the proposed method performs well compared with state-of-the-art algorithms.

[Full paper]

On-line Discovery of Temporal-Difference Networks

Takaki Makino and Toshihisa Takagi

We present an algorithm for on-line, incremental discovery of temporal-difference (TD) networks. The key contribution is the establishment of three criteria to expand a node in TD network: a node is expanded when the node is well-known, independent, and has a prediction error that requires further explanation. Since none of these criteria requires centralized calculation operations, they are easily computed in a parallel and distributed manner, and scalable for bigger problems compared to other discovery methods of predictive state representations. Through computer experiments, we demonstrate the empirical effectiveness of our algorithm.

[Full paper]

Confidence-Weighted Linear Classification

Mark Dredze, Koby Crammer, and Fernando Pereira

We introduce confidence-weighted linear classifiers, a new class of algorithms that maintain confidence information about classifier parameters. Learning in this framework updates parameters by estimating weights and increasing model confidence. We investigate a new online algorithm that maintains a Gaussian distribution over weight vectors, updating the mean and variance of the model with each instance. Empirical evaluation on a range of NLP tasks show that our algorithm improves over other state of the art online and batch methods, learns faster in the online setting, and lends itself to better classifier combination after parallel training.

[Full paper]

On the Chance Accuracies of Large Collections of Classifiers

Mark Palatucci and Andrew Carlson

We provide a theoretical analysis of the chance accuracies of large collections of classifiers. We show that on problems with small numbers of examples, some classifier can perform well by random chance, and we derive a theorem to explicitly calculate this accuracy. We use this theorem to provide a principled feature selection criteria for sparse, high-dimensional problems. We evaluate this method on both microarray and fMRI datasets and show that it performs very close to the optimal accuracy obtained from an oracle. We also show that on the fMRI dataset this technique chooses relevant features successfully while another state-of-the-art method, the False Discovery Rate (FDR), completely fails at standard significance levels.

[Full paper]

Efficient Projections onto the L1-Ball for Learning in High Dimensions

John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra

We describe efficient algorithms for projecting a vector onto the L1-ball. We present two methods for projection. The first performs exact projection in O(n) time, where n is the dimension of the space. The second works on vectors k of whose elements are perturbed outside the L1-ball, projecting in O(k log(n)) time. This setting is especially useful for online learning in sparse feature spaces such as text categorization applications. We demonstrate the merits and effectiveness of our algorithms in numerous batch and online learning tasks. We show that variants of stochastic gradient projection methods augmented with our efficient projection procedures outperform state-of-the-art optimization techniques such as interior point methods. We also show that in online settings gradient updates with L1 projections outperform the EG algorithm while obtaining models with high degrees of sparsity.

[Full paper]

Tailoring Density Estimation via Reproducing Kernel Moment Matching

Le Song, Xinhua Zhang, Alex Smola, Arthur Gretton, and Bernhard Schoelkopf

Moment matching is a popular means of parametric density estimation. We extend this technique to nonparametric estimation of mixture models. Our approach works by embedding distributions into a reproducing kernel Hilbert space, and performing moment matching in that space. This allows us to tailor density estimators to a function class of interest (i.e., for which we would like to compute expectations). We show our density estimation approach is useful in applications such as message compression in graphical models, and image classification and retrieval.

[Full paper]

Graph Kernels Between Point Clouds

Francis Bach

Point clouds are sets of points in two or three dimensions. Most kernel methods for learning on sets of points have not yet dealt with the specific geometrical invariances and practical constraints associated with point clouds in computer vision and graphics. In this paper, we present extensions of graph kernels for point clouds, which allow to use kernel methods for such objects as shapes, line drawings, or any three-dimensional point clouds. In order to design rich and numerically efficient kernels with as few free parameters as possible, we use kernels between covariance matrices and their factorizations on graphical models. We derive polynomial time dynamic programming recursions and present applications to recognition of handwritten digits and Chinese characters from few training examples.

[Full paper]

Learning Dissimilarities by Ranking: From SDP to QP

Hua Ouyang and Alexander Gray

We consider the problem of learning dissimilarities between points via formulations which preserve a specified ordering between points rather than the numerical values of the dissimilarities. Dissimilarity ranking (d-ranking) learns from instances like "A is more similar to B than C is to D" or "The distance between E and F is larger than that between G and H". Three formulations of d-ranking problems are presented and new algorithms are presented for two of them, one by semidefinite programming (SDP) and one by quadratic programming (QP). Among the novel capabilities of these approaches are out-of-sample prediction and scalability to large problems.

[Full paper]

The Skew Spectrum of Graphs

Risi Kondor and Karsten Borgwardt

The central issue in representing graph-structured data instances in learning algorithms is designing features which are invariant to permuting the numbering of the vertices. We present a new system of invariant graph features which we call the skew spectrum of graphs. The skew spectrum is based on mapping the adjacency matrix to a function on the symmetric group and computing bispectral invariants. The reduced form of the skew spectrum is computable in O(n³) time, and experiments show that on several benchmark datasets it can outperform state of the art graph kernels.

[Full paper]

Accurate Max-margin Training for Structured Output Spaces

Sunita Sarawagi and Rahul Gupta

Tsochantaridis et al 2005 proposed two formulations for maximum margin training of structured spaces: margin scaling and slack scaling. While margin scaling has been extensively used since it requires the same kind of MAP inference as normal structured prediction, slack scaling is believed to be more accurate and better-behaved. We present an efficient variational approximation to the slack scaling method that solves its inference bottleneck while retaining its accuracy advantage over margin scaling. We further argue that existing scaling approaches do not separate the true labeling comprehensively while generating violating constraints. We propose a new max-margin trainer PosLearn that generates violators to ensure separation at each position of a decomposable loss function. Empirical results on real datasets illustrate that PosLearn can reduce test error by up to 25%. Further, PosLearn violators can be generated more efficiently than slack violators; for many structured tasks the time required is just twice that of MAP inference.

[Full paper]

Optimized Cutting Plane Algorithm for Support Vector Machines

Vojtech Franc and Soeren Sonnenburg

We have developed a new Linear Support Vector Machine (SVM) training algorithm called OCAS. Its computational effort scales linearly with the sample size. In an extensive empirical evaluation OCAS significantly outperforms current state of the art SVM solvers, like SVMLight, SVMPerf and BMRM, achieving speedups of over 1,000 on some datasets over SVMLight and 20 over SVMPerf, while obtaining the same precise Support Vector solution. OCAS even in the early optimization steps shows often faster convergence than the so far in this domain prevailing approximative methods SGD and Pegasos. Effectively parallelizing OCAS we were able to train on a dataset of size 15 million examples (itself about 32GB in size) in just 671 seconds --- a competing string kernel SVM required 97,484 seconds to train on 10 million examples sub-sampled from this dataset.

[Full paper]

Self-taught Clustering

Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu

This paper focuses on a new clustering task, called self-taught clustering. Self-taught clustering is an instance of unsupervised transfer learning, which aims at clustering a small collection of target unlabeled data with the help of a large amount of auxiliary unlabeled data. The target and auxiliary data can be different in topic distribution. We show that even when the target data are not sufficient to allow effective learning of a high quality feature representation, it is possible to learn the useful features with the help of the auxiliary data on which the target data can be clustered effectively. We propose a co-clustering based self-taught clustering algorithm to tackle this problem, by clustering the target and auxiliary data simultaneously to allow the feature representation from the auxiliary data to influence the target data through a common set of features. Under the new data representation, clustering on the target data can be improved. Our experiments on image clustering show that our algorithm can greatly outperform several state-of-the-art clustering methods when utilizing irrelevant unlabeled auxiliary data.

[Full paper]

A Quasi-Newton Approach to Nonsmooth Convex Optimization

Jin Yu, S.V.N. Vishwanathan, Simon Guenter, and Nicol Schraudolph

We extend the well-known BFGS quasi-Newton method and its limited-memory variant (LBFGS) to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: The local quadratic model, the identification of a descent direction, and the Wolfe line search conditions. We apply the resulting sub(L)BFGS algorithm to L₂-regularized risk minimization with binary hinge loss, and its direction-finding component to L₁-regularized risk minimization with logistic loss. In both settings our generic algorithms perform comparable to or better than their counterparts in specialized state-of-the-art solvers.

[Full paper]

Predicting Diverse Subsets Using Structural SVMs

Yisong Yue and Thorsten Joachims

In many retrieval tasks, one important goal involves retrieving a diverse set of results (e.g., documents covering a wide range of topics for a search query). First of all, this reduces redundancy, effectively presenting more information with the presented results. Secondly, search queries are often ambiguous at some level. For example, the query “Jaguar” can refer to many different topics (such as the car or the feline). A set of documents with high topic diversity ensures that fewer users abandon the query because none of the results are relevant to them. Unlike existing approaches to learning retrieval functions, we present a method that explicitly trains to diversify results. In particular, we formulate the learning problem of predicting a diverse subset and derive a training algorithm based on structural SVMs.

[Full paper]

Improved Nystrom Low-Rank Approximation and Error Analysis

Kai Zhang, Ivor Tsang, and James Kwok

Low-rank matrix approximation is an effective tool in alleviating the memory and computational burdens of kernel methods and sampling, as the mainstream of such algorithms, has drawn considerable attention in both theory and practice. This paper presents detailed studies on the Nystrom sampling scheme and in particular, an error analysis that directly relates the Nystrom approximation quality with the encoding powers of the landmark points in summarizing the data. The resultant error bound suggests a simple and efficient sampling scheme, the k-means clustering algorithm, for Nystrom low-rank approximation. We compare it with state-of-the-art approaches that range from greedy schemes to probabilistic sampling. Our algorithm achieves significant performance gains in a number of supervised/unsupervised learning tasks including kernel PCA and least squares SVM.

[Full paper]

Expectation-Maximization for Sparse and Non-Negative PCA

Christian David Sigg and Joachim M. Buhmann

We study the problem of finding the dominant eigenvector of the sample covariance matrix, under additional constraints on its elements: a cardinality constraint limits the number of non-zero elements, and non-negativity forces the elements to have equal sign. This problem is known as sparse and non-negative principal component analysis (PCA), and has many applications including dimensionality reduction and feature selection. Based on expectation-maximization for probabilistic PCA, we present an algorithm for any combination of these constraints. Its complexity is at most quadratic in the number of dimensions of the data. We demonstrate significant improvements in performance and computational efficiency compared to the state-of-the-art, using large data sets from biology and computer vision.

[Full paper]

Training SVM with Indefinite Kernels

Jianhui Chen and Jieping Ye

Similarity matrices generated from many applications may not be positive semidefinite, and hence can't fit into the kernel machine framework. In this paper, we study the problem of training support vector machines with an indefinite kernel. We consider a regularized SVM formulation, in which the indefinite kernel matrix is treated as a noisy observation of some unknown positive semidefinite one (proxy kernel) and the support vectors and the proxy kernel can be computed simultaneously. We propose a semi-infinite quadratically constrained linear program formulation for the optimization, which can be solved iteratively to find a global optimum solution. We further propose to employ an additional pruning strategy, which significantly improves the efficiency of the algorithm, while retaining the convergence property of the algorithm. In addition, we show the close relationship between the proposed formulation and multiple kernel learning. Experiments on a collection of benchmark data sets demonstrate the efficiency and effectiveness of the proposed algorithm.

[Full paper]

An RKHS for Multi-View Learning and Manifold Co-Regularization

Vikas Sindhwani and David Rosenberg

Inspired by co-training, many multi-view semi-supervised kernel methods implement the following idea: find a function in each of multiple Reproducing Kernel Hilbert Spaces (RKHSs) such that (a) the chosen functions make similar predictions on unlabeled examples, and (b) the average prediction given by the chosen functions performs well on labeled examples. In this paper, we construct a single RKHS with a data-dependent “co-regularization” norm that reduces these approaches to standard supervised learning. The reproducing kernel for this RKHS can be explicitly derived and plugged into any kernel method, greatly extending the theoretical and algorithmic scope of co-regularization. In particular, with this development, the Rademacher complexity bound for co-regularization given in (Rosenberg & Bartlett, 2007) follows easily from well-known results. Furthermore, more refined bounds given by localized Rademacher complexity can also be easily applied. We propose a co-regularization based algorithmic alternative to manifold regularization (Belkin et al., 2006; Sindhwani et al., 2005a) that leads to major empirical improvements on semi-supervised tasks. Unlike the recently proposed transductive approach of (Yu et al., 2008), our RKHS formulation is truly semi-supervised and naturally extends to unseen test data.

[Full paper]

A Generalization of Haussler's Convolution Kernel - Mapping Kernel

Kilho Shin and Tetsuji Kuboyama

Haussler's convolution kernel provides a successful framework for engineering new positive semidefinite kernels, and has been applied to a wide range of data types and applications. In the framework, each data object represents a finite set of finer grained components. Then, Haussler's convolution kernel takes a pair of data objects as input, and returns the sum of the return values of the predetermined primitive kernel calculated for all the possible pairs of the components of the input data objects. Due to the definition, Haussler's convolution kernel is also known as the cross product kernel, and is positive semidefinite, if so is the primitive kernel. On the other hand, the mapping kernel that we introduce in this paper is a natural generalization of Haussler's convolution kernel, in that the input to the primitive kernel moves over a predetermined subset rather than the entire cross product. Although we have plural instances of the mapping kernel in the literature, their positive semidefiniteness was investigated in case-by-case manners, and worse yet, was sometimes incorrectly concluded. In fact, there exists a simple and easily checkable necessary and sufficient condition, which is generic in the sense that it enables us to investigate the positive semidefiniteness of an arbitrary instance of the mapping kernel. This is the first paper that presents and proves the validity of the condition. In addition, we introduce two important instances of the mapping kernel, which we refer to as the size-of-index-structure-distribution kernel and the edit-cost-distribution kernel. Both of them are naturally derived from well known (dis)similarity measurements in the literature (e.g. the maximum agreement tree, the edit distance), and are reasonably expected to improve the performance of the existing measures by evaluating their distributional features rather than their peak (maximum/minimum) features.

[Full paper]

Composite Kernel Learning

Marie Szafranski, Yves Grandvalet, and Alain Rakotomamonjy

The Support Vector Machine (SVM) is an acknowledged powerful tool for building classifiers, but it lacks flexibility, in the sense that the kernel is chosen prior to learning. Multiple Kernel Learning (MKL) enables to learn the kernel, from an ensemble of basis kernels, whose combination is optimized in the learning process. Here, we propose Composite Kernel Learning to address the situation where distinct components give rise to a group structure among kernels. Our formulation of the learning problem encompasses several setups, putting more or less emphasis on the group structure. We characterize the convexity of the learning problem, and provide a general wrapper algorithm for computing solutions. Finally, we illustrate the behavior of our method on multi-channel data where groups correspond to channels.

[Full paper]

Nonnegative Matrix Factorization via Rank-One Downdate

Michael Biggs, Ali Ghodsi, and Stephen Vavasis

Nonnegative matrix factorization (NMF) was popularized as a tool for data mining by Lee and Seung in 1999. NMF attempts to approximate a matrix with nonnegative entries by a product of two low-rank matrices, also with nonnegative entries. We propose an algorithm called rank-one downdate (R1D) for computing a NMF that is partly motivated by singular value decomposition. This algorithm computes the dominant singular values and vectors of adaptively determined submatrices of a matrix. On each iteration, R1D extracts a rank-one submatrix from the dataset according to an objective function. We establish a theoretical result that maximizing this objective function corresponds to correctly classifying articles in a nearly separable corpus. We also provide computational experiments showing the success of this method in identifying features in realistic datasets. The method is much faster than either LSI or other NMF routines.

[Full paper]

paper ID: 668

Closed-form Supervised Dimensionality Reduction with Generalized Linear Models

Irina Rish, Genady Grabarnilk, Guillermo Cecchi, Francisco Pereira, and Geoffrey J. Gordon

We propose a family of supervised dimensionality reduction (SDR) algorithms that combine feature extraction (dimensionality reduction) with learning a predictive model in a unified optimization framework, using data- and class-appropriate generalized linear models (GLMs), and handling both classification and regression problems. Our approach uses simple closed-form update rules and is provably convergent. Promising empirical results are demonstrated on a variety of high-dimensional datasets.

[Full paper]

Tibet Riot

2008-03-17T09:45:00.004-06:00

It seems that all the governments and media will do certain degree of censorship. The above figure is just an example. The left one is used in CNN to show that Chinese communist government is trying to overwhelm the riot. The right one shows the original picture. As can be seen, several Tibetan is trying to hurt some innocent people walking by the street.

DaLai only cares about the independence of Tibet but never morn for those innocent Han or Hui people who were killed during the riot. Ridiculous!

Here is one foreigner who is traveling in Lhasa and experienced the riot.
http://kadfly.blogspot.com/

and here is a video took by an Italian to show that those people were attacking an innocent motorcyclist.
http://rapidshare.de/files/38832674/MVI_0483.AVI.html

Also, for those who support "Tibet Independence", please check out the following videos. Tibet will always be part of China.

http://www.youtube.com/watch?v=x9QNKB34cJo

Using Samba in Ubuntu

2008-03-02T17:17:00.002-07:00

Really simple!

Right click the folder actually has an effect of sharing folders.

All you need is to add a smbuser by the following command:
smbpasswd -a user

and then restart the service by the following command:
sudo /etc/init.d/samba restart

Daily Linux (1) -- FreeNX, remote access, list files

2007-12-12T17:11:00.000-07:00

1. Here is a very useful instruction to install FreeNX server on Ubuntu Gusty,

and it works pretty cool! Much much faster than VNC.

2. To list files matches certain pattern in one directory, use the following command:

find . -name "*.c"

This basically find all the c-files in a directory.

dynamic topic models

2007-11-20T10:06:00.000-07:00

Recently, I'm pretty interested in topic models.
But this is very difficult to follow.

Instead, I'll read some material about Kalman filtering and Wavelet first.

Learning is endless...

NIPS 08 proceeding available

2007-11-19T17:49:00.001-07:00

Here are some interesting paper I'm planning to read or browse.
My biggest concern is how effective is the work. It seems currently most are beautiful with formulas, but can not even beat the simplest method.

It's always human who make the world so complicated.
My belief is "The World is Simple!"

Heterogeneous Component Analysis
Shigeyuki Oba, Motoaki Kawanabe, Klaus-Robert Müller, Shin Ishii

Neural characterization in partially observed populations of spiking neurons
Jonathan Pillow, Peter Latham

Probabilistic Matrix Factorization
Ruslan Salakhutdinov, Andriy Mnih

Hidden Common Cause Relations in Relational Learning
Ricardo Silva, Wei Chu, Zoubin Ghahramani

Hierarchical Penalization
Marie Szafranski, Yves Grandvalet, Pierre Morizet-Mahoudeaux

Learning with Transformation Invariant Kernels
Christian Walder

A Spectral Regularization Framework for Multi-Task Structure Learning
Andreas Argyriou, Charles A. Micchelli, Massimiliano Pontil, Yiming Ying

Supervised Topic Models
David Blei, Jon McAuliffe

Learning Bounds for Domain Adaptation
John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman

Multi-task Gaussian Process Prediction
Edwin Bonilla, Kian Ming Chai, Chris Williams

Automatic Generation of Social Tags for Music Recommendation
Douglas Eck, Paul Lamere, Thierry Bertin-Mahieux, Stephen Green

Kernel Measures of Conditional Dependence
Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, Bernhard Sch??lkopf

What do you see?

2007-11-05T20:13:00.001-07:00

A funny joke

2007-08-27T09:40:00.000-06:00

狼来了，经过山洞，见兔子面前放台笔记本电脑，噼噼噼的乱敲。狼说：“兔子！干嘛呢？” “写论文！” 狼想：“世道真变啦，兔子也能写论文？”。 “啥题目？” “论兔子比狼厉害！” “你有病吧？” “你进洞瞧瞧！我收集好多资料！” 狼进洞了，半天没出来。天近黄昏，兔子收拾起笔记本电脑，回山洞了。洞中一只狮子正在剔牙。见兔子进来，拍着胸脯狂笑：“早跟你说了，不在乎你写什么题目，什么内容，关键是，你导师是谁！

Research is a joke?

2007-08-22T22:33:00.000-06:00

Just read the chapter of Quasi-Newton Methods and came across this paragraph:
"The first quasi-Newton algorithm turned out to be one of the most creative ideas in nonlinear optimization. .... An interesting historical irony is that Davidon's paper was not accepted for publication; it remained as a technical report for more than thirty years until it appeared in the first issue of the SIAM Journal on Optimization in 1991."

It seems that high-quality work is more difficult to publish.

However, computer science still generates thousands of papers each year. How many paper will be remembered after 5 years?

Fast food is everywhere. This also includes "fast paper". Ridiculous!

SIGIR & KDD papers

2007-07-18T00:29:00.000-06:00

SIGIR'07

An InterActive Algorithm For Asking And Incorporating Feature Feedback into Support Vector Machines
Hema Raghavan, James Allan

Analyzing Feature Trajectories for Event Detection
QI HE, Kuiyu Chang, Ee-Peng Lim

New Event Detection Based on Indexing-tree and Named Entity
Kuo ZHANG, JuanZi LI, Gang WU

Session 12: Learning to Rank I (Wed 09:00-10:30)

Chair: Bruce Croft, Foyer Room

A Support Vector Method for Optimizing Average Precision
Yisong Yue, Thomas Finley, Filip Radlinski, Thorsten Joachims
Ranking with Multiple Hyperplanes
Tie-Yan Liu, Tao Qin, Wei Lai, Xu-Dong Zhang, De-Sheng Wang, Hang Li
A Regression Framework for Learning Ranking Functions Using Relative Relevance Judgments
Zhaohui Zheng, Hongyuan Zha, Keke Chen, Gordon Sun

Session 16: Learning to Rank II (Wed 14:30-16:30)

Chair: Keith van Rijsbergen, Grand Ballroom

FRank: A Ranking Method with Fidelity Loss
Ming-Feng Tsai, Tie-Yan Liu, Tao Qin, Hsin-Hsi Chen, Wei-Ying Ma
AdaRank: A Boosting Algorithm for Information Retrieval
Jun Xu, Hang Li
A Combined Component Approach for Finding Collection-Adapted Ranking Functions based on Genetic Programming
Humberto Almeida, Marcos Goncalves, Marco Cristo, Pavel Calado
Feature Selection for Ranking
Tie-Yan Liu, Xiubo Geng, Tao Qin

KDD'07

Active Exploration for Learning Rankings from Clickthrough Data - Filip Radlinski and Thorsten Joachims

Co-clustering based Classification for Out-of-domain Documents - Wenyuan Dai, Gui-Rong Xue, Qiang Yang, and Yong Yu

Enhancing Semi-Supervised Clustering: A Feature Projection Perspective - Wei Tang, Hui Xiong, Shi Zhong, and Jie Wu

Evolutionary Spectral Clustering by Incorporating Temporal Smoothness - Yun Chi, Xiaodan Song, Dengyong Zhou, Koji Hino, and Belle Tseng

Mining Statistically Important Equivalence Classes - Jinyan Li, Guimei Liu, and Limsoon Wong

Model-Shared Subspace Boosting for Multi-label Classification - Rong Yan, Jelena Tesic, and John Smith

Support Feature Machine for Classification of Abnormal Brain Activity - W. Art Chaovalitwongse, Ya-Ju Fan, and Rajesh Sachdeo

ICML 07 proceeding is online now

2007-05-27T01:13:00.000-06:00

http://oregonstate.edu/conferences/icml2007/paperlist.html

Here are some interesting papers:

Kernel Selection:

Kernel Selection for Semi-Supervised Kernel Machines [Abstract][Paper]

Learning Nonparametric Kernel Matrices from Pairwise Constraints

[Abstract][Paper]

More Efficiency in Multiple Kernel Learning [Abstract][Paper]

Multiclass Multiple Kernel Learning [Abstract][Paper]

MTL and Transfer Learning:

Uncovering Shared Structures in Multiclass Classification [Abstract][Paper]

Discriminative Learning for Differing Training and Test Distributions [Abstract][Paper]

Boosting for Transfer Learning [Abstract][Paper]

Learning a Meta-Level Prior for Feature Relevance from Multiple Related Tasks [Abstract][Paper]

Multi-Task Learning for Sequential Data via iHMMs and the Nested Dirichlet Process [Abstract][Paper]

Self-taught Learning: Transfer Learning from Unlabeled Data [Abstract][Paper]

The Matrix Stick-Breaking Process for Flexible Multi-Task Learning [Abstract][Paper]

Asymptotic Bayesian Generalization Error When Training and Test Distributions Are Different [Abstract][Paper]

Robust Multi-Task Learning with $t$-Processes [Abstract][Paper]

Relational Learning and structured prediction

Relational Clustering by Symmetric Convex Coding [Abstract][Paper]

Fast and Effective Kernels for Relational Learning from Texts [Abstract][Paper]

Exponentiated Gradient Algorithms for Log-Linear Structured Prediction [Abstract][Paper]

Ranking:

Learning Random Walks to Rank Nodes in Graphs [Abstract][Paper]

Feature Selection:

Supervised Feature Selection via Dependence Estimation [Abstract][Paper]

Feature Selection in Kernel Space [Abstract][Paper]

Minimum Reference Set Based Feature Selection for Small Sample Classifications [Abstract][Paper]

Spectral Feature Selection for Supervised and Unsupervised Learning [Abstract][Paper]

MSR internship offer

2007-05-21T10:02:00.000-06:00

I just received an offer from MSR for this summer intern. But I decided to reject it since it's a more application oriented intern. Maybe, staying in school can help produce more works. That's more hope and belief.

BTW: I just checked with Martha and found that we can use CPT twice. Anyhow, this is a good news for ASU student. I'm not sure whether or not my complaint makes the difference. Again, do things proactively can always makes future better.

Crafting papers in machine learning

2007-05-08T00:23:00.000-06:00

Pat is like a tutor.

http://lyonesse.stanford.edu/~langley/papers/craft.ml2k.ps

I just encountered Pat's interesting paper on how to write a machine learning paper. Not sure whether or not he is a little regretful about emphasizing the experimental evaluation too much in the paper. But I think most of the general guide lines are still reasonable.

linux trick

2007-05-04T02:04:00.000-06:00

I just found the rdesktop tool in linux is better to remote access windows system. Especially in the full-screen mode.

One trick to quit full screen mode:
Ctrl+Alt+Enter

Windows Backup & Norton Ghost

2007-05-01T00:13:00.000-06:00

After my system crashed, I have to reinstall a windows system on my laptop, which is horrible. It takes almost three days to finish all the installation and configuration process. Suffering from reinstall system, I decided to try to use Norton Ghost to backup the system to make my life easier. Unfortunately, GHOST requires a floppy drive to make sure your system is bootable when crashed. More serious, I installed a dual-boot system on my laptop (ubuntu7.04+windows xp pro (sp2)) with the first parition for windows being NTFS. So the MBR of windows is actually modified by linux, and ghost can not find the proper MBR infor for windows. To backup linux system is easy, just compress all the stuff into one tar file would be OK.

So I'm wondering whether or not I can backup the whole hard disk under linux system. But the problem is, even if I can backup the windows partition in linux, I cannot restore it as Ubuntu 7.04 does not support modification on NTFS. Haha, really frustrated with this backup research work.

It seems that linux backup is way too much easier than windows. But I'm still a newbie to linux. Maybe, after sometime, I can get rid of windows.

Feedback of ICML

2007-04-03T16:13:00.000-06:00

"I think the paper is techincally sound, but I think that it is quite narrow in
scope, and am not sure it will be of great general interest. There are only few
and unsurprising conclusions, and moreover the results seem to indicate that
there is an interacation between IN/OUT and the specific FS algorithms used.
Focussing so strongly on toy data, is also a considerable worry. I recommend
rejection. "

It seems that this year is not my "lucky" year. This is the same situation for my intern applications. Getting so many interviews(7 interviews), but none of them makes an offer.

Anyway, just keep going!

Several Future directions

2007-03-26T16:32:00.000-06:00

While reckon over the future directions to go, I think the following problems might be interesting concerning text categorization.

1. Large-number of categories, multi-label classification problem. Typically, a hierarchy is employed to dissect the problem, thus reduce to learning with structured output.

2. "Dirty" text categorization. Typical text categorization requires the features to be clean, such as newswire articles, paper abstract etc. However, current fashion extends to "dirty" texts, such as notes(spell errors), prescription(lots of abbreviations) customer telephone log (usually with noisy, contradicting facts). Another example is email spam filtering. Currently, most of spams consists of images rather than just text. However, existing OCR techniques can not extract the characters very correctly. Hence, the final words/terms obtained might not a "proper" feature. Hence, some techniques are required to transform a "image-derived" word into a word in the dictionary. Such kind of transformation can be done via some algorithm like shortest-path algorithm. However, when the spammer add noise in purpose in the text within images, this problem seems more complicated. Is it possible to automatically learn feature similarity? How to extract useful similarities measure between these noisy vectors? How to derive a useful kernel? So, this problem actually is related to feature extraction, kernel learning, and robust learning and uncertainty.

3. Event detection and concept drift. I believe such kind of directions has more promising effect. I think the difficulty lays mainly on the lack of benchmark data set. But with the development of Web 2.0, this kind of problem should gain some attention in the future.

4. Ambiguous label problem. I really doubt the existence of small sample in text classification. Seems labellings some documents requires very little human labor. Now, some websites already provides some schemes for users to tag some web pages and blog posts. How to effectively employ the tag info seems to be missing in current work. When I tried delicious, only some key-word matching are performed. How to organize the text into more sensible way?

5. "Universal" text classification. As so many benchmark data sets are online, can we any how use all of them. This might be related to transfer learning. At least, the benchmark data can serve to provide a common prior for the target classification task. But can we extract more? Human beings can immediately classify the documents given very few examples. Existing transfer learning (most actually are doing MTL), in nature, is doing dimensionality reduction. How to related the features of different domains? Is it possible to extract the "structural" information? Zhang Tong's work talks about that, but it actually focus more on semi-supervised learning.

6. Sentimental classification/author gender identification/ genre identification. Such kind of problems requires new feature extraction techniques.

Some other concerns:

Feature selection for text categorization? As far as I can see, I do not think this direction will provide more interesting result. It works, and efficient. It can be used as a preprocessing step to reduce the computation burden. But some complicated methods (such as kernel learning) can be used to do a better job.

Active Learning, a greedy method can works fine enough.

Clustering, not making any sense to me. But as for simple text, clustering might show some potential impact. I believe clustering should be "customized". Different user will ask different clustering results. It seems more interesting to do clustering given some prespecified parameters. Clustering of multi-labels under concept drift can also be explored.

Leonhard Euler

2007-03-22T22:56:00.000-06:00

"Leonhard Euler(1707-1783) A Swiss mathematician and physicist.
......
During the last 17 years of his life, he was almost totally blind, and yet he produced nearly half of his results during that period."

(Copied from Pattern Recognition and Machine Learning, Page 465)

I am wondering whether or not he has any chance to win the Fields Award:)

Where to Go?

2007-03-22T12:07:00.000-06:00

Recently, I seems to lose the enthusiasm of research. No mood to read, to discuss, to work on experiments. I know this is not correct way to go, but I just couldn't help become anxious. I would rather there's a problem let me to just jump in.

I don't know whether this is related to the frustration of recent progress. Too slow.

To be frank, I really doubt current development of machine learning field. Too fast. Each year, in the top level conference, there are lots of publications, but very few of them worth reading. Lots of them are just rubbish. Is this common in other fields?

I am wondering which direction should be a correct way to go. Searching in blind is really difficult:(

Human Intelligence and Current Artificial Intelligence

2007-03-02T13:36:00.000-07:00

I've take Pat Langley's cognitive system class this semester.

My feeling of intelligence is rather vague.

What's intelligence? If a system is complicated enough, is that intelligent? What should be a component of an intelligence? Like Deep Blue, is that intelligence. I guess most people won't vote for intelligence. Now, people are trying to develop a meta-game player, which can automatically train itself as long as the rules of the game are given. Is that intelligent? No. Human beings are actually open-minded. Yep, machines can outperform human beings in one field, but they can never be "artificial intelligence". That's why I feel AI would never come true in my life. But I agree with Pat that more focus should be on the nature of mind.

For a long time, I've been thinking that the brain and a machine's difference is mainly due to the hardware difference (brain's processing speed is low, memory is limited, but highly parallel). Like playing chess, planning, scheduling, while machines can do an exhaustive search, or deeper search, human beings try to prune a lot of branches in each step (I guess there should be some pattern recognition involved). It's more like beam-search, as I think. The problem for efficient processing is how to extract those useful patterns.

Until recently, Pat mentioned this paper:
The magic of 7 +/- 2
which tries to understand why human beings did a better job to remember 7 digits more easily than 4 or more digits(This is partly the reason why telephone digits are around 7).

Based on information theory, the more digits we have, the more information they contains, thus require more bits to represent the information. But for human beings, this is not the case. This phenomenon is really interesting, and justifies the study for human brains. I guess Pat didn't realize this example is so exciting to me:)

KDD submisssion is done

2007-03-01T10:59:00.000-07:00

Nitin and I coauthored one paper talking about finding influentials in blogosphere. It's an interesting problem. the method seems a little ad hoc. But we finally made it.

Through a long distance communication (that's the wonder of Web), Nitin is working in India while Dr.Liu and I were working in the united states. The final version seems much better. But whether or not this work can be accepted is really depending on our luck. Part of my concern the technical part is really subjective.

I don't know whether the reviewer is obsessed with cross validation.

Let's see.

No hope for this google interview

2007-02-13T13:51:00.000-07:00

Just done with the google interview. I believe this this the worst case I've ever done.
It's so bad. The interviewee is a Chinese, but we still talk in English.

First question ask me to write a SQL query. I haven't touch database for a long time. So after 10 mins, I still can't come up with a good answer.

Second question: how to implement a merge sort? write the pseudo code. Then, I ask him whether or not I can use recursion. He questioned: without recursion, how do you do that? (Stupid, why not just give the answer with recursion). What's the time complexity and space complexity. I know time complexity is O(nlgn) but space complexity is not sure. I finally figure out the correct solution should be O(n). I guess he already kicked me out of his candidate list based on my performance.

3rd question: 10G large array data, 2G RAM, how do you sort them? I know this question should be merge sort? But how many data should you load each time into the memory?
I answered you can split an array of size n into 2 parts, 3 parts or even more, and then do a combination. So for 2 parts, you need a working memory of (n/2), 3 parts, you need a working memory of (n/3). but what should be the optimal number? No idea.

Too nervous, cannot think any more. And he mentioned we were out of time several times.

4. it's about the project. How to solve their problem.

He asked me if I have any questions, but I already gave up. So just ask him some normal questions.

Really need to know the algorithm and structure book well. But too busy to prepare the interview. Really shame on myself.

How to collobrate with others?

2007-02-10T14:02:00.000-07:00

I just read John's blog and found it is really very helpful. Thus, I just copy and past his stuff here.

The full article can be found via the link below:
http://hunch.net/?p=251

2/10/2007
Best Practices for Collaboration
Filed under: Papers, Research — jl @ 1:51 pm

Many people, especially students, haven’t had an opportunity to collaborate with other researchers. Collaboration, especially with remote people can be tricky. Here are some observations of what has worked for me on collaborations involving a few people.

1. Travel and Discuss Almost all collaborations start with in-person discussion. This implies that travel is often necessary. We can hope that in the future we’ll have better systems for starting collaborations remotely (such as blogs), but we aren’t quite there yet.
2. Enable your collaborator. A collaboration can fall apart because one collaborator disables another. This sounds stupid (and it is), but it’s far easier than you might think.
1. Avoid Duplication. Discovering that you and a collaborator have been editing the same thing and now need to waste time reconciling changes is annoying. The best way to avoid this to be explicit about who has write permission to what. Most of the time, a write lock is held for the entire document, just to be sure.
2. Don’t keep the write lock unnecessarily. Some people are perfectionists so they have a real problem giving up the write lock on a draft until it is perfect. This prevents other collaborators from doing things. Releasing write lock (at least) when you sleep, is a good idea.
3. Send all necessary materials. Some people try to save space or bandwidth by not passing ‘.bib’ files or other auxiliary components. Forcing your collaborator to deal with the missing subdocument problem is disabling. Space and bandwidth are cheap while your collaborators time is precious. (Sending may be pass-by-reference rather than attach-to-message in most cases.)
4. Version Control. This doesn’t mean “use version control software”, although that’s fine. Instead, it means: have a version number for drafts passed back and forth. This means you can talk about “draft 3″ rather than “the draft that was passed last tuesday”. Coupled with “send all necessary materials”, this implies that you naturally backup previous work.
3. Be Generous. It’s common for people to feel insecure about what they have done or how much “credit” they should get.
1. Coauthor standing. When deciding who should have a chance to be a coauthor, the rule should be “anyone who has helped produce a result conditioned on previous work”. “Helped produce” is often interpreted too narrowly—a theoretician should be generous about crediting experimental results and vice-versa. Potential coauthors may decline (and senior ones often do so). Control over who is a coauthor is best (and most naturally) exercised by the choice of who you talk to.
2. Author ordering. Author ordering is the wrong thing to worry about, so don’t. The CS theory community has a substantial advantage here because they default to alpha-by-author ordering, as is understood by everyone.
3. Who presents. A good default for presentations at a conference is “student presents” (or suitable generalizations). This gives young people a real chance to get involved and learn how things are done. Senior collaborators already have plentiful alternative methods to present research at workshops or invited talks.
4. Communicate by default Not cc’ing a collaborator is a bad idea. Even if you have a very specific question for one collaborator and not another, it’s a good idea to cc everyone. In the worst case, this is a few-second annoyance for the other collaborator. In the best case, the exchange answers unasked questions. This also prevents “conversation shifts into subjects interesting to everyone, but oops! you weren’t cced” problem.

These practices are imperfectly followed even by me, but they are a good ideal to strive for.

Done with the ICML paper

2007-02-09T11:53:00.000-07:00

OK. Payam and I just finished the ICML paper. It's good. I believe it should be accepted unless we are really unlucky.

Anyway, it's a good experience to work with Payam. I believe our cooperation makes the task much easier and efficient.

Hope we can get good news soon.

Need to rush for my KDD paper now:)

How to test two data sets from the same distribution?

2007-02-05T21:18:00.000-07:00

Suppose I have two data sets, how can I tell that they are from two distributions? what's the difference of these two data sets?

This is actually very generic question emerged in machine learning.

Some intuitive ideas:
1. estimate the density of data samples for each data set. This method might be very weak. In reality, density estimation is a very difficult task compared with "simple classification". This approach is generally not application in reality.

2. Estimate the sufficient statistics of each data set. Like the mean, variance of each feature, the class conditional distribution. This can be interpreted as analogy in cognition. An analogy can be derived if the relationship between multiple symbols can be maintained. The problem is when can we conclude that the difference is large enough? It seems some hypothesis testing is required.

3. Transformation. A data set can be transformed into another data set. But how do you know the feature mapping? A more reasonable way is to enforce equivalence of sufficient statistics in a newly generated space.

4. Dimensionality reduction. Assume that two data set shared the same distribution on a fixed number of dimensions. By projecting the two data set into those dimensions, probably we can find some interesting patterns.

5. Learn the classification function. Use the decision function to measure the difference. this is quite related to transfer learning.

6. Any other ways?

OK. To endeavor this direction, where can we find the data set?

The approach of Kernel Maximum Mean discrepancy might be related.

最牛的博士论文！(转载)

2007-02-05T20:26:00.000-07:00

最牛的博士论文！教授们紧张到恍惚以为自己才是答辩人

　 1. 最牛博士论文就是在还没答辩之前已经发表在最好的期刊上，而且鉴于论文很长
，该期刊必须像小说一样连载。

　实例：张五常博士论文《佃农理论》，当年在JLE上连载四期。

2. 最牛博士论文答辩就是答辩人一直在挑战答辩委员会成员，直到问得这些教授们紧
张到恍惚以为自己才是答辩人。

　实例：萨缪尔森的博士论文答辩结束后，答辩委员会成员之一的熊彼特（上世纪最
伟大的经济学家之一）转过头去问另一位成员里昂剔夫（诺奖得主）：“瓦西里，我们
通过了么？”

3. 最牛投稿论文就是让编辑满世界都找不到一个能看懂这篇论文的匿名审稿人，最后
只能发表，根本不需要修改的。

实例：SIMS1971年发表在《数理统计年鉴》上的论文《无穷维参数空间中的分布滞后估
计》。SIMS写完这篇论文后没投经济学杂志，因为他显然知道没人看的懂。于是投给了
最牛B的数理统计杂志，结果编辑死活找不到审稿人，最后好不容易凑合拉来一个，审
稿报告是这么写的：“我真的不明白这篇论文在说什么，但是我检验了其中的几个定理
，好像是对的。所以我猜应该发表。”

4. 最牛B的论文没必要长篇大论，千把字足以。实例：德布罗意是个花花公子贵族，本
科是历史学专的，后来实在闲着无聊去读了5年博士，最后交的博士论文是一页纸，还
涉嫌“抄袭”。

答辩委员会气的都不想让他答辩。他的导师、著名物理学家朗之万感到很没面子，自己
学生毕业不了真是耻辱，于是他鼓动了爱因斯坦一起帮着求情：让这小子过了吧，他老
爸是法国内政部长，咱惹不起。那篇“垃圾”论文后来被薛定谔看到了，薛定谔看着这
页论文苦思冥想了1个月，发表了量子力学里最重要的理论之一的薛定谔方程，薛定谔
猫也成为最有趣的一只猫。

德布罗意因这篇论文说阐述的观点获得了诺贝尔物理学奖。薛定谔凭借德布罗意的这篇
论文对量子力学作出了杰出贡献，从一名普通而不得志的讲师一跃成为了一名伟大的科
学家并获得了诺贝尔物理学奖。可以说，一篇1页纸的博士论文成就2个诺贝尔物理学奖
可谓前无古人，估计也是后无来者。由此看来，最牛b的论文不必象张五常那样连载，
一页A4的纸足以。不过我想德布罗意要是在中国读博士就惨了，论文因为字数太少，根
本连答辩的资格都没有。

不得不说两句：德布罗意幼年即失去双亲，被他的哥哥莫里斯公爵（也是一名杰出的
物理学家）一手养大的，在他1924年的著名博士论文之前一年，德布罗意就已连续发表
三篇论文提出物质波的猜想，至于博士论文是几页纸，这个我还没考证过。

关于薛定谔：薛定谔多才多艺，会4种语言，出过诗集。另外他于1944年出版的《生
命是什么》，吸引了一大批物理学家转向生物学研究。其中包括后来双螺旋的发现者沃
森和克里克。所以说，这帮牛人并不一定像人们想象的那样传奇，也不能把其成功单纯
的归结为偶然的因素。正所谓：牛者恒牛。
--

science research is not proof

2007-01-31T15:28:00.000-07:00

Two days ago, we had a very nice discussion with Pat Langley.
Pat is more a cognitive scientist than a typical computer scientist, also very talkative:)

Basically, I question him why he recommend one paper saying that it took longer time for human beings to reason about difficult puzzles. The authors use a forward chaining scheme to build their model. Based on the similar performance of the model and human beings, it concludes that it took longer time for human beings to solve difficult puzzles.

OK. That paper is a little boring to me as the result is totally not surprising, or even too "obvious". Also, I really doubt human beings reasons like forward chaining or backward chaining. At the first sight, it seems the result of the paper is totally not convincing to me though the result is obvious.

So I asked him about the validality of this experiment set up. Then, I got his question, except this way, what can you do? Yep, I just post questions but forgot to think out of the box. What would I do if I am the researcher? You propose a model, then what you can do is to find "sufficient" evidence to support your claim. However, in most disciplines, this cannot be proved as mathematics. And even it's proved in someway, it might not work in reality. That's a common case in data mining, also in planning. The disadvantage of deduction is you have to trust your premise 100%, if one of your assumption is wrong, you mess up.

After reckon over that problem for a while, I have to sadly admit that's the only possible way or proper way to justify a new claim. So the problem is how to find "enough" evidence?

It seems that our science research is very weak. Or maybe that's exactly the process of doing research. You propose a model to solve a problem, explain some phenomenon. Then people use one counter example to disprove your model. (Disproof is always much easier than proof, but this seems not the case for refutation in logic proof). Then, a new model or theory is proposed.

You'll find that almost all science or engineering research are repeating the same cycle.

When going back to machine learning experiment, I think I've already put too much belief to 10-fold cross validation. But actually, most of tasks have no correct evaluation method. Like information retrieval, data analysis. these tasks requires a reasonable good answer but not an optimal solution.

Like the recent paper I coauthored with Payam for ICML, we found that two dramatically different feature selection evaluation methods turn out to be almost the same for comparing feature selection methods. There's actually no proof. But I believe it's an interesting research.

Science Research is pragmatic.

Just finished the journal paper

2007-01-25T14:51:00.000-07:00

I've just finished the journal paper for TKDD.
This paper really takes a while to fill in. I even got puke for rewriting it.

I guess I've looked through it at least 15 times. I really cannot bear to read it any more.

I am wondering, to do research, should I put more effort to polish a paper or to think about the idea?

I really hate spending so much time on writing the paper. i think it's not worthwhile.

The ridiculous thing about current stage of machine learning and data mining is that all these papers tend to foster UNNECESSARY formula to make paper more easy to accept.

That's not the goal for research. i would like more to motivate the problem well and present some simple, intuitive, sensible and working methods!!

Some experiment result

2007-01-21T22:08:00.000-07:00

I just finished some demo experiments. Originally, I wanted to find some toy example to show task selection's effect in transfer learning. Unfortunately, all the results I found are very disappointing.
Let me summarize the results a little bit:
(1) If the target task has very limited training data, transfer learning do help a lot compared with single task learning.
(2) The tasks selected make a very tiny difference (within 1% percent). Actually, it seems that combine all the tasks together is a very robust and reliable strategy for the data set I'm working on.
(3) Combine all the data together seems always better than transfer learning.

(1) is not surprising and has been approved by existing researchers.
(2) can not justify task selection.
(3) It seems that there's no difference between these tasks in the data set.

Maybe, one interesting problem is to determine whether the data extracted from multiple sources are actually the same. But I feel that's a more difficult problem than task selection.

Flexing Muscle, China Destroys Satellite in Test

2007-01-20T14:10:00.000-07:00

This is great news for Chinese!!

http://www.nytimes.com/2007/01/19/world/asia/19china.html?n=Top%2fNews%2fWorld%2fCountries%20and%20Territories%2fChina

Article Tools Sponsored By
By WILLIAM J. BROAD and DAVID E. SANGER
Published: January 19, 2007

China successfully carried out its first test of an antisatellite weapon last week, signaling its resolve to play a major role in military space activities and bringing expressions of concern from Washington and other capitals, the Bush administration said yesterday.

Only two nations — the Soviet Union and the United States — have previously destroyed spacecraft in antisatellite tests, most recently the United States in the mid-1980s.

Arms control experts called the test, in which the weapon destroyed an aging Chinese weather satellite, a troubling development that could foreshadow an antisatellite arms race. Alternatively, however, some experts speculated that it could precede a diplomatic effort by China to prod the Bush administration into negotiations on a weapons ban.

“This is the first real escalation in the weaponization of space that we’ve seen in 20 years,” said Jonathan McDowell, a Harvard astronomer who tracks rocket launchings and space activity. “It ends a long period of restraint.”

White House officials said the United States and other nations, which they did not identify, had “expressed our concern regarding this action to the Chinese.” Despite its protest, the Bush administration has long resisted a global treaty banning such tests because it says it needs freedom of action in space.

Jianhua Li, a spokesman at the Chinese Embassy in Washington, said that he had heard about the antisatellite story but that he had no statement or information.

At a time when China is modernizing its nuclear weapons, expanding the reach of its navy and sending astronauts into orbit for the first time, the test appears to mark a new sphere of technical and military competition. American officials complained yesterday that China had made no public or private announcements about its test, despite repeated requests by American officials for more openness about its actions.

The weather satellite hit by the weapon had circled the globe at an altitude of roughly 500 miles. In theory, the test means that China can now hit American spy satellites, which orbit closer to Earth. The satellites presumably in range of the Chinese missile include most of the imagery satellites used for basic military reconnaissance, which are essentially the eyes of the American intelligence community for military movements, potential nuclear tests and even some counterterrorism, and commercial satellites.

Experts said the weather satellite’s speeding remnants could pose a threat to other satellites for years or even decades.

In late August, President Bush authorized a new national space policy that ignored calls for a global prohibition on such tests. The policy said the United States would “preserve its rights, capabilities, and freedom of action in space” and “dissuade or deter others from either impeding those rights or developing capabilities intended to do so.” It declared the United States would “deny, if necessary, adversaries the use of space capabilities hostile to U.S. national interests.”

The Chinese test “could be a shot across the bow,” said Theresa Hitchens, director of the Center for Defense Information, a private group in Washington that tracks military programs. “For several years, the Russians and Chinese have been trying to push a treaty to ban space weapons. The concept of exhibiting a hard-power capability to bring somebody to the negotiating table is a classic cold war technique.”

Gary Samore, the director of studies at the Council on Foreign Relations, said in an interview: “I think it makes perfect sense for the Chinese to do this both for deterrence and to hedge their bets. It puts pressure on the U.S. to negotiate agreements not to weaponize space.”

Ms. Hitchens and other critics have accused the administration of conducting secret research on advanced antisatellite weapons using lasers, which are considered a far speedier and more powerful way of destroying satellites than the weapons of two decades ago.

The White House statement, issued by the National Security Council, said China’s “development and testing of such weapons is inconsistent with the spirit of cooperation that both countries aspire to in the civil space area.”

An administration official who had reviewed the intelligence about China’s test said the launching was detected by the United States in the early evening of Jan. 11, which would have been early morning on Jan. 12 in China. American satellites tracked the launching of the medium-range ballistic missile, and later space radars saw the debris.

The antisatellite test was first reported late Wednesday on the Web site of Aviation Week and Space Technology, an industry magazine. It said intelligence agencies had yet to “complete confirmation of the test.”

The test, the magazine said, appeared to employ a ground-based interceptor that used the sheer force of impact rather than an exploding warhead to shatter the satellite.

Dr. McDowell of Harvard said the satellite was known as Feng Yun, or “wind and cloud.” Launched in 1999, it was the third in a series. He said that it was a cube measuring 4.6 feet on each side, and that its solar panels extended about 28 feet. He added that it was due for retirement but that it still appeared to be electronically alive, making it an ideal target.

“If it stops working,” he said, “you know you have a successful hit.”

David C. Wright, a senior scientist at the Union of Concerned Scientists, a private group in Cambridge, Mass., said he calculated that the Chinese satellite had shattered into 800 fragments four inches wide or larger, and millions of smaller pieces.

The Soviet Union conducted roughly a dozen antisatellite tests from 1968 to 1982, Dr. McDowell said, adding that the Reagan administration carried out its experiments in 1985 and 1986.

The Bush administration has conducted research that critics say could produce a powerful ground-based laser weapon that would be used against enemy satellites.

The largely secret project, parts of which were made public through Air Force budget documents submitted to Congress last year, appears to be part of a wide-ranging administration effort to develop space weapons, both defensive and offensive.

The administration’s laser research is far more ambitious than a previous effort by the Clinton administration to develop an antisatellite laser, though the administration denies that it is an attempt to build a laser weapon.

The current research takes advantage of an optical technique that uses sensors, computers and flexible mirrors to counteract the atmospheric turbulence that seems to make stars twinkle. The weapon would essentially reverse that process, shooting focused beams of light upward with great clarity and force.

Michael Krepon, co-founder of the Henry L. Stimson Center, a group that studies national security, called the Chinese test very un-Chinese.

“There’s nothing subtle about this,” he said. “They’ve created a huge debris cloud that will last a quarter century or more. It’s at a higher elevation than the test we did in 1985, and for that one the last trackable debris took 17 years to clear out.”

Mr. Krepon added that the administration had long argued that the world needed no space-weapons treaty because no such arms existed and because the last tests were two decades ago. “It seems,” he said, “that argument is no longer operative.”

Mark Mazzetti contributed reporting.

Rejected by Google

2007-01-19T22:40:00.000-07:00

I thought I could reject Google, but unfortunately Google rejected me!! :(

We would like to thank you for your interest in Google. After carefully reviewing your experience and qualifications, we have determined that we do not have a 'Software Engineering Intern' position available which is a strong match at this time.

Thanks again for considering Google. We wish you well in your endeavors
and hope you might consider us again in the future.

Two TA work again!!!!

2007-01-18T09:57:00.000-07:00

This semester, I have to work as TA for both AI and Data Mining Class again. So sad...
I am wondering why I am always so unlucky?

Just returned the new version of journal paper to boss. Feels really tired to make any change to that.

Google Phone Interview (1st round)

2007-01-17T13:37:00.001-07:00

(1st round)

Why do you like google?
What's the difference between process and threads?
What's the difference between Java and C++?
Tell me the basic concepts of object-oriented programming?
How to implement mutiple inherience in Java? Why java use interface while C++ keeps feature of multiple inherience?

Have you ever involved into any team project? what did you do?
How do you handle the case that you have different opinions with manager?

Then discuss about my research topic.

How to find most common word in billions of documents?
1. If memory allowed (hashtable). what's the time complexity?
2. What if multiple machines? What's the bottleneck?
3. What if just one machine and hashtable can not be stored in the memory?

Then, I asked hime some general questions.
The last task, he asked me to send code to him in 30 mintues. The task is write a function to transform a string into an integer.

Google Phone Interview(2nd round)

2007-01-17T13:14:00.000-07:00

Just finished my interview with Google. This time is an engineer, asking lots of detailed questions.

He knows a lot about lisp, so we just discuss about the issues about lisp. like
what's the difference of lisp and other languages?
which kind of lisp complier do you use? emacs lisp vs. clisp?

Then, some questions about operating system?
What's the difference between process and thread? What kind of information does thread maintain? its own stack? heap?
How and when to do a context switch? How do you handle an time slice interrupt?
What are the possible pitfalls for multi-thread programming?

How compiler works?
Can regular expression resolve the problem of nested structures?
Tell me something about grammar?
Is type check done before or after parsing?
(I did pretty bad in this session, so he finally stopped)

Familiar with TCP/IP, RPC, network programming? (NO, skipped)

Are you familiar with B-tree, red-black tree? (No, so we switch to binary search tree)
What's the time complexity of insertion or query in a binary search tree? O(lg n)
Worst case? (O (n))
How to transform a unbalanced tree into balanced tree?
Are you familiar with TreeMap in Java?

How hash table works? What if two object have the same key value? Show me one example of hash function.
What is the innate structure of a hashtable? (I said array) How do you map a key value to an index?

Finally, one technical question:
Given a source word, a target word, and a dictionary, how to transform the source word into target word by changing only one letter in each step. The word you get in each step must be in the dictionary.

Then, I asked him about some projects details.

2nd round phone interview from google

2007-01-11T20:51:00.001-07:00

good news.

Desirable features of blog site

2007-01-03T13:04:00.000-07:00

I just initiated two blogs, one in google blogspot, the other one in windows live space. Compare these two blog site is kind of interesting.

Windows Live:
1. Better handling photos
2. Template seems to be more beautiful.

Google blogspot:

1. Support group blog which is absent in windows live space.

Both sucks:
1. Can not set the permission for a specific post;
2. Can not change the layout and template freely, like change the background of template.

I am wondering how could these two blog sites to be so successful. Is there any other good ones?

Redundant Features useful or not?

2007-01-02T23:16:00.000-07:00

I just browsed one interesting paper published in ICML'06:

nightmare at test time: robust learning by feature deletion

The motivation for the paper can be described as like buying stock.

Suppose you have several stocks with the same risk, and you have $1000. What would you do?

Of course, divide all the money evenly into these stocks should be more reliable than putting them into just one.

This is the same situation for feature selection. Suppose you select some relevant features from training data, but it could be wrong due to small samples or noise or any kind of noise.

In this process, you probably remove those redundant features as well. From this point of view, it seems more robust to keep those redundant features rather than remove them.

But from curse of dimensionality view, redundant features should be removed.

How to trade off redundancy and robustness?
I guess this is highly related to the definition of redundancy.

I'll comment on this issue more in future.

Some interesting search engines

2007-01-01T13:53:00.000-07:00

Some interesting search engines: Hakia, powerset, snap
there's an article in nytimes talking about these search engines.

Will these be one of the future google?

I used to be a google fan. But I just found that google maps sucks. I tried different maps for LA trip, and mapquest seems to do a much better job. Google map finally and always got me lost. To be sure, this is not the first time.

Travel to LA (23rd-27th)

2006-12-22T14:12:00.000-07:00

More will come after I come back:)

Data Mining dead for effective counterterrorism?

2006-12-19T15:49:00.000-07:00

Jeff Jonas has just posted one paper in his blog saying that data mining would fail to detect suspicious for counterterrorism. This kind of voice is really interesting to me.

In his article, he mentioned the limitations of existing data mining techniques:

1. typical data mining requires lots of data available to discover the pattern of terrorism. However, terrorism's frequency is so low that data mining would fail to detect it.

2. the cost of false positive. If the number of population is large, suppose 1 billion people under monitored, even with a very high accuracy(say 99.9%), the false positive would result in 1,000,000 suspicious, which would be too costly to perform furthur investigation and tracking.

After reading it, I just have several questions in mind:

Is inductive data mining dead for this case?
Is deductive analysis more suitble for this case?
How to detect novelty? What is an "anomly"?
Terrorists will change their stratgy for next attack. Is it possible to find it?

Instead of claiming the limitedness of data mining, I would cheer for the "dead" of statistical data mining. Machine learning, finally have to rethink about its correct direction.

Data mining to detect perpetrators and accomplices

2006-12-18T20:41:00.000-07:00

One article seems interesting. In this article, the researchers tried to use data mining technique to detect perpetrators and accomplices during online auction. Seems an interesting problem. But really difficult to obtain the data. I guess. Such kind of data always involves lots of private issues.

Carnegie Mellon Researchers Uncover Online Auction Fraud; Data Mining Software Fingers Both Perpetrators and Accomplices

PITTSBURGH, Dec. 5 (AScribe Newswire) -- Computer scientists at Carnegie Mellon University are using data mining techniques to identify perpetrators of fraud among online auction users as well as their otherwise unknown accomplices.

The new method analyzes publicly available histories of transactions posted by online auction sites such as eBay and identifies suspicious online behaviors and dubious associations among users.

Online auction sites are immensely popular. The largest, eBay, reported third quarter revenues of $1.449 billion, up 31 percent from the previous year, and registered 212 million users, up 26 percent. But the popularity of online auction sites also makes them a target for crooks. Internet auction fraud, such as failure to deliver goods after a sale, accounted for almost two-thirds of the 97,000 complaints referred to law enforcement agencies last year by the federal Internet Crime Complaint Center.

Perpetrators of these frauds have distinctive online behaviors that cause them to be readily purged from an online auction site, said Computer Science Professor Christos Faloutsos. The software developed by his research team - Network Detection via Propagation of Beliefs, or NetProbe - could prevent future frauds by identifying their accomplices, who can lurk on a site indefinitely and enable new generations of fraudsters.

In a test analysis of about one million transactions between almost 66,000 eBay users, NetProbe correctly detected 10 previously identified perpetrators, as well as more than a dozen probable fraudsters and several dozen apparent accomplices.

"To the best of our knowledge, this is the first work that uses a systematic approach to analyze and detect electronic auction frauds," said Faloutsos, who noted that NetProbe could eventually be useful for both law enforcement and security personnel of online sites.

The researchers have already adapted the software to provide a trustworthiness score for individual user IDs. Though not yet available to the public, the NetProbe score would complement user reputation scores that many auction sites already provide to help prevent fraud.

"We want to help people detect potential fraud before the fraud occurs," said research associate Duen Horng "Polo" Chau, who developed NetProbe with Faloutsos, undergraduate student Samuel Wang and graduate student Shashank Pandit.

Many auction sites try to avert fraud with so-called reputation systems. In eBay's case, buyers can report whether they had a positive, neutral or negative experience in a transaction, and that report is then translated into a feedback score for that seller.

Unfortunately, a crook can manipulate these feedback scores, obtaining a favorable score by engaging in a number of legitimate sales. But that is costly and time-consuming and, once the fraudster starts cheating buyers, that user identification is quickly red-flagged and shut down.

Perpetrating frauds may be sustainable, however, if a fraudster has accomplices or sets up separate user IDs to serve as accomplices. These accomplice accounts conduct legitimate transactions and maintain good reputations. They also have many transactions with the user IDs of fraudsters, using their good reputations to boost the fraudsters' feedback scores. Because accomplices don't perpetrate frauds, they usually escape notice and can keep working to establish new fraudster accounts, Faloutsos said.

But an unnatural pattern becomes evident when the transactions are plotted as a graph, with each user represented as a node, or dot, and transactions between individual users represented by lines connecting the nodes.

In the resulting graph, transactions between accomplices and fraudsters create a pattern that sticks out like "a guiding light," Chau said. Graph theorists call this pattern a "bipartite core" - members of one group have lots of transactions with members of a second group, but don't have transactions with members of their own group. One group, the accomplices, also deals with honest eBay users, but most of the transactions are with fraudster groups.

The researchers tested their method, in part, by accumulating transaction histories from eBay and demonstrating that they could detect the distinctive fraud patterns within these massive data sets. Chau reported on an analysis involving about 100 eBay users at a September data mining conference in Berlin. The team has since analyzed about a million transactions between almost 66,000 eBay users, and those as-yet unpublished findings have been submitted for presentation at an upcoming scientific conference.

"Crooks are extremely ingenious," Faloutsos warned, so identifying accomplices would not eliminate all online auction fraud. But eliminating accomplices would force crooks to resort to more sophisticated, complex schemes. "These schemes will require more effort and cost, so fraud would be increasingly unprofitable," he added.

Domain adaptation & learning from multiple sources

2006-12-18T10:47:00.000-07:00

It seems that different people interpret transfer learning in different terms:
Multitask learning, Domain Adaptation, learning from multiple sources, sample selection bias.

Here, I just tried to distinguish these terms and discuss about the difference.

Multitask learning has been studied a lot. Usually, the objective function is to learn a model for each task such that the overall performance is optimized. These models share some commonality.
A very strong limitation of this method is that
overall performance != performance on one specific task.

In my opinion, MTL actually solves the learning problem in an indirect way. There are lots of issues involved: how to find similar tasks, which tasks should I trust, how much should I trust for each task, what's the trade-off between data and tasks. All these problems requires additional knowledge from domain or data, which make it rather limited. Actually, MTL can only works if very few data is available for each task. If there's no training data for one task, MTL can not work.

Domain adaptation, is more like a transfer learning setting. Domain adaptation can be considered as involving two tasks: one is source domain(support task), one is target domain(target task). This term is coined(as I see) from NLP community. The goal is exactly the same as transfer learning, to improve the performance of target task.

Learning from multiple sources, can be a little tricky. There are actually two interpretations: one assume that all the sources are drawn from the same underlying distribution, but with various level noise or uncertainty(If some additional information like the bound of the noise can be obtained, then it's possible to take advantage of it); The other one assumes that different source have different underlying distribution. But they are related. Thus, the former is more like data selection, how to learn ONE model given some noisy data while the latter is exactly like Multitask learning (develope one model for each task(source)).

Finally, transfer learning can also be connected to sample selection bias as I mentioned in last post. How ever sample selection bias always deals with the case such that only one biased sample is available, how to obtain an unbiased model. This situation is more like domain adaptation. We can effectively apply the method in one field to the other.

Sample selection bias = Multitask learning?

2006-12-15T16:47:00.000-07:00

I just browsed one interesting paper:
Dirichlet enhanced spam filtering based on biased samples. which pose a question to me:

Is sample selection bias = multitask learning?

As in this paper, each user's spam filtering can be considered as one task.

However, there are two very "Strong" assumptions:

1. The sample selection bias of publicly available data and personalized data is still 0-1 consistent. That is P(x|public) !=0 then P(x|personal) !=0;

2. P(y|x, public) = P(y|x, personal).

The 1st assumption is still OK, since most sample selection bias adopt this one. (Otherwise, I think there's no way to inference something you have no chance to know).

The 2nd assumption is way too UNACCEPTABLE to me. Given a message, some users might treat it as a ham, some might treat it as a spam. How could this be possible to be equal?!
(Is this because it's easy to analyze?)

In my opinion, MTL can not be considered as a feature bias (in terms of sample selection bias), nor a label bias. A more general model should be a complete bias. That is, P(s=1|x,y) cannot be decomposed into any simpler form.

Actually, I am thinking whether or not it's possible to learn P(s=1|x,y) (as in the paper I mentioned) if biased and unbiased samples are both provided. OK. Let's start with the ideal case. Then a logistic regression learner can be adopted to estimate the bias. The trick here is treat (x, y) as input feature and s=1 or 0 as output.

In MTL, only very few labeled data are provided for the unbiased setting. Can we estimate the density reliably? The dilemma is that: we need "enough" data to improve the bias density estimation, but if we have "enough" data, we can already derive a very good model.

Any other possible way to enhance the estimation?

This paper adopt the Dirichlet process to improve the estimation. But why does it work? I couldn't figure it out.

Finally done with course work

2006-12-14T20:01:00.001-07:00

Finally finished all the finals and TA work.

Since this is the end of the semester, I would like to make a summary of the work I have done.

OK, this year, I took 2 courses, one about numerical linear algebra. I think this class is pretty good. I finally do not have scare away from those stupid matrix and SVD stuff. But unfortunately, I didn't spend much time on this. I think the projects on this class are pretty cool. Especially the Arnoldi process. I didn't expect such a large sparse matrix can be handled by only few iterations.

The SVD stuff, LU decomposition, QR decomposition(finally I got a chance to show off in this Blog:) I think I won't use these much in future research work. But combined with convex optimization, it should help.

Well... The 2nd course : distributed operating system. The instructor is a good talker and explains everything very well. Unfortunately, I am an eager learner(not kNN--"lazy learner"). He speaks so slow that I couldn't help getting asleep? Is this my fault?

Anyway, it's good to learn the concept about lock/unlock, synchronization, multi threading, PVM, messger system, shared memory, net messenger server, RPC, lock server, distributed shared memory, distributed mutual exclusive/snapshot. All the terms sound very cool... I did really bad in the final. But who cares? (PHD do not care about course work. That becomes my quote now. hehe...)

Finally, let's talk about the TA work. I am working as a TA for Subbarao Kambhampati.
To be frank, this is the toughest TA work I've ever have. 4 homeworks (the last one has around 50 questions, which really drove me crazy!!!!) and 5 projects. I was grading either the homework or the project nearly every weekend. So many stuff.

But from another perspective, I've never understand AI concept so clear. Rao connected agent design, search, planning, MDP, logic, Bayesian network, learning so well that I have to admit I learned much more by doing this TA work than taking a course. Actually, I don't think I learned anything really exciting when I took Dr. Liu's AI. Rao did a much better job. But I don't like his projects. No challenge, though lots of students spent way too much time on lisp. Anyway, I would like to thank to this experience. (I guess this is due to my gf. She changed me a lot!)

But...... I have to work as TA again next semester, both AI and data mining. I want to kill someone.....

Some interesting papers from NIPS 2006

2006-12-14T19:50:00.000-07:00

NIPS'2006 has just released their online proceedings.
http://books.nips.cc/nips19.html

Here are some interesting papers:

------------------------------------------------------------
Dirichlet-Enhanced Spam Filtering based on Biased Samples
Steffen Bickel, Tobias Scheffer [ps.gz][pdf][bibtex]

Denoising and Dimension Reduction in Feature Space
Mikio Braun, Joachim Buhmann, Klaus-Robert Müller [ps.gz][pdf][bibtex]

Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation
Gavin Cawley, Nicola Talbot, Mark Girolami [ps.gz][pdf][bibtex]

Modeling General and Specific Aspects of Documents with a Probabilistic Topic Model
Chaitanya Chemudugunta, Padhraic Smyth, Mark Steyvers [ps.gz][pdf][bibtex]

Learning from Multiple Sources
Koby Crammer, Michael Kearns, Jennifer Wortman [ps.gz][pdf][bibtex]

Optimal Single-Class Classification Strategies
Ran El-Yaniv, Mordechai Nisenson [ps.gz][pdf][bibtex]

Clustering Under Prior Knowledge with Application to Image Segmentation
Mario Figueiredo, Dong Seon Cheng, Vittorio Murino [ps.gz][pdf][bibtex]

Data Integration for Classification Problems Employing Gaussian Process Priors
Mark Girolami, Mingjun Zhong [ps.gz][pdf][bibtex][zip]

Correcting Sample Selection Bias by Unlabeled Data
Jiayuan Huang, Alex Smola, Arthur Gretton, Karsten M. Borgwardt, Bernhard Schölkopf [ps.gz][pdf][bibtex]

Cross-Validation Optimization for Large Scale Hierarchical Classification Kernel Methods
Matthias Seeger [ps.gz][pdf][bibtex]

---------------------------------------------------------------------
Interesting:
Image Retrieval and Classification Using Local Distance Functions
Andrea Frome, Yoram Singer, Jitendra Malik [ps.gz][pdf][bibtex][tgz]

Branch and Bound for Semi-Supervised Support Vector Machines
Olivier Chapelle, Vikas Sindhwani, Sathiya Keerthi [ps.gz][pdf][bibtex]

Max-margin classification of incomplete data
Gal Chechik, Geremy Heitz, Gal Elidan, Pieter Abbeel, Daphne Koller [ps.gz][pdf][bibtex]

Bayesian Ensemble Learning
Hugh Chipman, Edward George, Robert McCulloch [ps.gz][pdf][bibtex]

Map-Reduce for Machine Learning on Multicore
Cheng-Tao Chu, Sang Kyun Kim, Yi-An Lin, YuanYuan Yu, Gary Bradski, Andrew Ng, Kunle Olukotun [ps.gz][pdf][bibtex]

Hierarchical Dirichlet Processes with Random Effects
Seyoung Kim, Padhraic Smyth [ps.gz][pdf][bibtex]

Ordinal Regression by Extended Binary Classification
Ling Li, Hsuan-Tien Lin [ps.gz][pdf][bibtex]

Commercement!!

2006-12-14T19:48:00.000-07:00

OK. This is the 3rd time I tried to create a Blog.

I am wondering why I cannot last for blog posting. Who knows?

Hope this time, I can make it work:)