Support Vector Machine Based on Type-2 Fuzzy Training Samples
- Author: Ha Ming-hu, Huang Jia-ying, Yang Yang, Wang Chao
- Organization: Ha Ming-hu; Huang Jia-ying; Yang Yang; Wang Chao
- Publish: Industrial Engineering and Management Systems Volume 11, Issue1, p26~29, 01 March 2012
-
ABSTRACT
In order to deal with the classification problems of type-2 fuzzy training samples on generalized credibility space. Firstly the type-2 fuzzy training samples are reduced to ordinary fuzzy samples by the mean reduction method. Secondly the definition of strong fuzzy linear separable data for type-2 fuzzy samples on generalized credibility space is introduced. Further, by utilizing fuzzy chance-constrained programming and classic support vector machine, a support vector machine based on type-2 fuzzy training samples and established on generalized credibility space is given. An example shows the efficiency of the support vector machine.
-
KEYWORD
Type-2 Fuzzy Training Samples , Mean Reduction Method , Fuzzy Chance-Constrained Programming , Support Vector Machine
-
The fuzzy set was proposed by Zadeh (1965) in 1965, it is an interesting extension of classical set. In classical sets the characteristic function can take only one of two values from the {0, 1} , in fuzzy sets the membership function can take any value from the interval [0, 1] . The membership function plays an important role in fuzzy set theory and its applications. But in some actual problems, it is often very difficult to determine the value of membership function (or membership degree for short). To solve this problem, the concept of type-2 fuzzy set was introduced by Zadeh (1975) in 1975. A type-2 fuzzy set is characterized by a special membership function, the value of which for each element of this set is a fuzzy set in [0, 1] . Compared with ordinary fuzzy set, type-2 fuzzy set could describe the objective phenomenon more accurately because of secondary membership function. It has been applied more efficiently to pattern recognition and other machine learning fields successfully (Mitchell, 2005).
Support vector machine built on statistical learning theory is a kind of effective machine learning method, has advantaged superiority in dealing with small samples classification problems, and has become the standard tool in machine learning field now (Cristianini and Shawe-Taylor, 2000). The traditional support vector machine is based on the real valued random samples and established on the probability space, and it is difficult to handle the small samples classification problems with non-real random samples on non-probability space. Naturally, it is a very interesting and valuable research direc-tion to construct the support vector machine based on non-real valued random samples and established on nonprobability space or probability space. On this basis, Lin and Wang (2002) proposed the support vector machine based on a non-real valued random samples-fuzzy samples, constructed the fuzzy support vector machine, which determines membership degree by different weight for each sample. Then Ha
et al. (2009, 2011) improved the fuzzy support vector machine in determining membership degree, and they constituted a new fuzzy support vector machine and an intuitionistic fuzzy support vector machine. Considering that above support vector machines only make the samples fuzzified and the training samples are not fuzzy input directly, Jiet al. (2010) constructed a kind of support vector machine for classification based on fuzzy training data, and discussed its application in the diagnosis of coronary. Nevertheless the support vector machine is limited to the ordinary fuzzy number samples-the triangular fuzzy samples, and it is difficult to deal with the classification problems of type- 2 fuzzy number training samples which is a generalization of the ordinary fuzzy number samples. In order to handle the problem, this paper firstly discusses the support vector machine based on type-2 fuzzy training samples and established on generalized credibility space.Definition 2.1 (Zadeh, 1965): A fuzzy setA inX is characterized by a mappingμ A :X → [0, 1],χ ?μ A (χ ),μ A is called the membership function ofA andμ A (χ ) is called the membership degree ofχ inA . The symbolrepresents
A . A collection of all fuzzy sets ofX can be denoted byDefinition 2.2: Supposea, b, c ∈R .a is a fuzzy set inR , and its membership function is as followsThen
a is called triangular fuzzy number, and is represented bya = (a ,b ,c ) (refer to Haet al. , 2010)Definition 2.3 (Zadeh, 1975): A type-2 fuzzy set inX , denotedis characterized by a secondary membership function
where
χ ∈X ,u ∈J x ⊆ [0,1], andcan also be expressed as
Definition 2.4 (Qinet al. , 2011): A type-2 fuzzy setis called triangular if its secondary membership function
is
for
x ∈ [r 1,r 2], andfor
x ∈ [r 2 ,r 3], whereθ 1,θ 2 ∈ [0, 1] are two parameters characterizing the degree of uncertainty thattakes the value
x . For simplicity, we denote the type-2 triangular fuzzy numberwith the above condition by (
r 1,r 2,r 3 ;θ1 ,θr ).Definition 2.5 (Qinet al. , 2011): LetX be a nonempty set,P (X ) be the class of all subsets ofX ,Pos be a possibility measure inP (X ). Supposea is a fuzzy number, then the possibility measure of fuzzy eventa ≤b is defined byPos ({a ≤b } ) = sup {μ a (χ ),χ ∈R ,χ ≤b } . Similarly,Pos ({a <b } ) = sup{μ a (χ ),χ ∈R ,χ <b },Pos ({a =b }) =μ a (b ).Definition 2.6 (Qinet al. , 2011): LetA be a fuzzy set inX , andbe a type-2 fuzzy set in
X .A is the reduction ofvia the mean reduction method, if it satisfies
We denote
Example 2.1 (Qinet al. , 2011): Letbe a type-2 triangular fuzzy number. According to the definition 2.6, we have the reduction
a ofand the membership function of
a is as follows.2.3 The Generalized Credibility Measure
Definition 2.7 (Qinet al ., 2011): Supposea is a fuzzy number. The generalized credibility measureof the event {
a ≤b } is defined byTheorem 2.1 (Qinet al. , 2011):Let ai be the mean reduction of type-2 fuzzy triangular number i = 1, 2, … ,n .Suppose a 1,a 2, … ,a nare mutually independent, and ki ≥ 0,i = 1, 2, … ,n .Given the generalized credibility level is equivalent to Proof: It can be easily got by type-2 fuzzy set theory in the reference.3. SUPPORT VECTOR MACHINE BASED ON TYPE-2 FUZZY TRAINING SAMPLES AND APPLICATION EXAMPLE
3.1 Support Vector Machine Based on Type-2 Fuzzy Training Samples
Consider the fuzzy type-2 training samples set
where
and
i = 1, 2, …,m ,j = 1,2 … ,n , whenyi = 1, thenis called a positive class; when
yi = -1, thenis called a negative class. The classification based on the type-2 fuzzy training set
is to find a decision function
such that the positive class and the negative class can be separated with the low classification error and good generalization performance.
Definition 3.1 (Jiet al ., 2010): For the type-2 fuzzy training samples setif for a given level
there exist
w ∈R ,n b ∈R , such thatThen the type-2 fuzzy training samples set
is strong type-2 fuzzy linear separable.
The support vector machine for strong linear separable type-2 fuzzy sample set is to solve the fuzzy chance constrained programming:
To solve the above programming,
is equivalent to
i = 1, 2, …,m ,j = 1, 2, … ,n according to Theorem 2.1. Then the fuzzy chance constrained programming is converted into the following classic convex quadratic programming:We can obtain its dual problem:
w* ,b* are the solution of above dual programming. For a given confidence levelif
then
belongs to the positive class; if
then
belongs to the negative class.
We shall apply the support vector machine for twoclass classification with type-2 fuzzy training samples to the diagnosis of Coronary. The data is sourced from the reference (Ji
et al. , 2010), the half of which are healthy (yi = 1), the others are Coronary patients (yi = -1). Letθ l, 1i =θ r, 1i = 0.1,θ l, 2i =θ r ,2i = 0.2, we can obtain type-2 triangular fuzzy numbersand
The programming was solved by SVM toolbox of matlab, when parameter
C =1,λ = 0.8. The accurate rate of diagnosis is 80% . The result of example illustrates that the proposed support vector machine is effective.This paper constructs firstly the support vector machine in which the training samples is type-2 fuzzy number input, and the proposed support vector machine is an interesting extension of the traditional support vector machine and the fuzzy support vector machine. We shall focus on the proposed support vector machine’s applications in some practical problems.