双子支持向量机研究博士论文

双子支持向量机研究博士论文

(SupportVectorMachines)

Twin

(TwinSupportVectorMachines)

Twin

Twin

1.

TwinTwin

TwinBoundedsupportvectormachines

(

TBSVM);

Twinsupport

vectormachines(

2.

CDMTSVM).

Twin

Twin

Twin

Twin

Twin

Twin3.

Twin

Twin

Twin

Twin

Twin

4.

Twin

Twin

(TwinTSVR

supportvectorregressor

TSVR)

-Twinsupportvectorregressor(

-TSVR),

-TSVR

ε-TSVR

:

Twin

双子支持向量机研究博士论文

Abstract

SupportVectorMachines(SVMs)aregenerallearningmethodswhicharebasedonStatisticalLearningTheory(SLT),andhavebeenproventobemorepowerfulthanexistingmethodsinmanyaspects.AsExtensionofSVMs,TwinSupportVectorMachines(TWSVM)wasproposedin2007.Duetoitslesscomputationcostandbettergeneralizationability,ithasreceivedextensiveattentionbytheacademiccommunity,andbecomeanewresearchfocus.Inthispaper,westudyin-depthabouttheTWSVM,includingclassi cationandregressionproblems,andproposefourimprovedvariantsasfollows:

1.

TBSVM:Fortheclassi cationproblems,weproposeanimprovedversion,namedtwin

boundedsupportvectormachines(TBSVM),basedonTWSVM.Thesigni cantadvantageofourTBSVMoverTWSVMfromtheoreticalpointofviewisthat,thestructuralriskminimizationprincipleisimplementedbyintroducingtheregularizationterm.Thisembodiesthemarrowofstatisticallearningtheory,sothismodi cationcanimprovetheperformanceofclassi cation.Inaddition,thee cientSORtechniqueisusedtosolvetheoptimizationproblemstospeedupthetrainingprocedure.Experimentalresultsshowthee ectivenessofourmethodinbothcomputationtimeandclassi cationaccuracy,thereforecon rmtheaboveconclusionfurther.

2.

CDMTSVM:Inordertoincreaseitse ciencyfurther,wepresentacoordinatedescent

marginbasedtwinvectormachine(CDMTSVM)comparedwiththeoriginalTWSVM.ThemajordisadvantagesofCDMTSVMlieintwoaspects:(1)Theprimalanddualproblemsarereformulatedandimprovedbyaddingaregularizationtermintheprimalproblemswhichimpliesmaximizingthe“margin”betweentheproximalhyperplaneandboundinghyperplane,yieldingthedualproblemstobestablepositivede nitequadraticprogrammingproblems.(2)Anovelcoordinatedescentmethodisproposedforourdualproblemswhichleadtoveryfasttraining.Asourcoordinatedescentmethodhandlesonedatapointatatime,itcanprocessverylargedatasetsthatneednotresideinmemory.OurexperimentsonpubliclyavailabledatasetsindicatethatourCDMTSVMisnotonlyfast,butalsoshowsgoodgeneralizationperformance.

3.UNH-MTSVM:Inordertocorrectlymeasurethedistanceofapatternfromthetwohyper-planesweneedtheunitynormofthenormalvectorsofthehyperplane.ButintheformulationofTWSVMtheseequalityconstraintswerenotconsidered.WehavereformulatedTWSVMandproposeanovelmarginbasedtwinsupportvectormachineswithunitynormhyperplanes(UNH-MTSVM).UNH-MTSVMconsidersunitynormconstraintsbyusingEuclideannorm.Meanwhile,aregular-izationtermwiththeideaofmaximizingsomemarginisintroduced.WesolvedUNH-MTSVMbyNewton’smethodandthesolutionisupdatedbyconjugategradientmethod.TheperformancesofboththelinearandnonlinearUNH-MTSVMareveri edexperimentallyonseveralbenchmarkandsyntheticdatasetsforbothlinearandnonlinearclassi ers.Experimentalresultsshowthee ectiveness

双子支持向量机研究博士论文

ofourmethodsinbothcomputationtimeandclassi cationaccuracy.

4.ε-TSVR:Fortheregressionproblems,weproposesanewregressor—ε-twinsupportvectorregression(ε-TSVR)basedontwinsupportvectorregression(TSVR).ε-TSVRdeterminesapairofε-insensitiveproximalfunctionsbysolvingtworelatedSVM-typeproblems.Thesigni cantadvantageofourε-TSVRoverTSVRisthat,thestructuralriskminimizationprincipleisimplementedbyintroducingtheregularizationterm.Thisembodiesthemarrowofstatisticallearningtheory,sothismodi cationcanimprovetheperformanceofregression.Inaddition,thee cientSORtechniqueisusedtosolvetheoptimizationproblemstospeedupthetrainingprocedure.Experimentalresultsforbotharti cialandrealdatasetsshowthat,comparedwiththepopularε-SVR,LS-SVRandTSVR,ourε-TSVRhasremarkableimprovementofgeneralizationperformancewithveryshorttrainingtime.

Keywords:Classi cationproblem,Regressionproblem,Supportvectormachines,Twinsupportvectormachines,Structuralriskminimizationprinciple.

双子支持向量机研究博士论文

RRn(xi,yi)lm1m2XYxixijxixijyiAB(x·x′)HΦww1w2wiwwibb1b2sgn(·)K(x,x′)K · Cc1c2

n

i

(

)

xijHilbert

i

xi

j

i

(

)

xx′

HilbertHilbert

w

i

Hilbert

w

i

(Gram

)

2-

双子支持向量机研究博士论文

c3c4ξξiiηηiiαLagrange

αiiγLagrange

γii

e

1

双子支持向量机研究博士论文

1.1

...........................................

1.2Twin................................1.3Twin................................

1.4.......................................1.5

.......................................

2.1...................................2.2

...................................

TwinBounded

3.1TwinBounded

..............................

3.2TwinBounded

.............................

3.3...........................................3.4...........................................

3.5

..............................................

Twin

4.1Twin

..............................

4.2Twin

.............................

4.3Twin......................

4.4...........................................

4.5

..............................................Twin

5.1Twin

.................................

5.2Twin

................................

5.3...........................................

5.4

..............................................

1

1358910

101520202325262730303 …… 此处隐藏：6331字，全部文档内容请下载后查看。喜欢就下载吧 ……