Designing kernels is important in supervised classification method such as in Support Vector Machine. However, there lacks in general guidelines for designing a proper kernel. In many applications, the observation data and the class label are linked through a latent state. Partial knowledge of the generative process, i.e., the likelihood of the observation given the latent state, is available. This knowledge can come from the actual physical sensing processes such as Computed tomography, or come from statistical models that well fit the dataset, e.g., the bag-of-word model for text document.
Given the knowledge of the sensing model, we show that the Bayes-optimum decision boundary is a hyperplane under a mapping defined by the likelihood function. This naturally yields a kernel SVM that can leverage knowledge of the sensing model in an optimal way. We could show that the learnt linear classifier is consistent in the sense that its expected risk converges to the Bayes optimal risk.
In our preliminary simulations, we illustrate the power of our sensing-aware kernel using the Bag-of-Words model, which is widely used to model text, image, video, and biological dataset. We consider two very different application – document classification and image scene reorganization. By applying our sensing-aware kernel to the baseline approaches, we could achieve similar performance as the state-of-the-arts based on sophisticated models.
- W. Ding, P. Ishwar, V. Saligrama, and W. C. Karl, “Sensing-Aware Kernel SVM”, in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’14), Florence, Italy, 4-9 May, 2014. [arXiv version]