Mixtures of Gaussians 混合高斯模型

Mixtures of Gaussians 混合高斯模型,HMM 模式识别

MixturesofGaussians

ATutorialfortheCourseComputationalIntelligence

http://www.igi.tugraz.at/lehre/CI

BarbaraResch

SignalProcessingandSpeechCommunicationLaboratory

In eldgasse16c/II

phone873–4436

Abstract

ThistutorialtreatsmixturesofGaussianprobabilitydistributionfunctions.Gaussianmixturesarecombinationsofa nitenumberofGaussiandistributions.Theyareusedtomodelcomplexmulti-dimensionaldistributions.WhenthereisaneedtolearntheparametersoftheGaussianmixture,theEMalgorithmisused.InthesecondpartofthistutorialmixturesofGaussianareusedtomodeltheemissionprobabilitydistributionfunctioninHiddenMarkovModels.

Usage

Tomakefulluseofthistutorialyoushould

1.Downloadthe leMixtGaussian.zipwhichcontainsthistutorialandtheaccompanyingMatlabprograms.

2.UnzipMixtGaussian.zipwhichwillgenerateasubdirectorynamedMixtGaussian/matlabwhereyoucan ndalltheMatlabprograms.

3.AddthefolderMixtGaussian/matlabandthesubfolderstotheMatlabsearchpathwithacom-mandlikeaddpath(’C:\Work\MixtGaussian\matlab’)ifyouareusingaWindowsmachineoraddpath(’/home/jack/MixtGaussian/matlab’)ifyouareusingaUnix/Linuxmachine.1

1.1MixturesofGaussiansFormulasandDe nitions

GaussianMixturesarecombinationsofGaussian,or‘normal’,distributions.AmixtureofGaussianscanbewrittenasaweightedsumofGaussiandensities.

Recallthed-dimensionalGaussianprobabilitydensityfunction(pdf):

g(µ,Σ)(x)=T 11 1(x µ)(x µ)Σe,(1)

withmeanvectorµandcovariancematrixΣ.

AweightedmixtureofKGaussianscanbewrittenas

gm(x)=K

k=1wk·g(µk,Σk)(x),(2)

wheretheweightsareallpositiveandsumtoone:

wk≥0andK

k=1wk=1fork∈{1,...,K}.(3)

Mixtures of Gaussians 混合高斯模型,HMM 模式识别

Figure1:OnedimensionalGaussianmixturepdf,consistingof3singleGaussians

InFigure1anexampleisgivenforanonedimensionalGaussianmixture,consistingofthreesingleGaussians.

ByvaryingthenumberofGaussiansK,theweightswk,andtheparametersµkandΣkofeachGaussiandensityfunction,Gaussianmixturescanbeusedtodescribeanycomplexprobabilitydensityfunction.

1.2TrainingofGaussianmixtures

TheparametersofaprobabilitydensityfunctionarethenumberofGaussiansK,theirweightingfactorswk,andthemeanvectorµkandcovariancematrixΣkofeachGaussianfunction.

To ndtheseparameterstooptimally tacertainprobabilitydensityfunctionforasetofdata,aniterativealgorithm,theexpectation-maximization(EM)algorithmcanbeused.WehaveintroducedthisalgorithminthetutorialaboutGaussianstatisticstotrainparametersoftheGaussianpdfsforseveralclassesinunsupervisedclassi cation.Startingwithinitialvaluesforallparameterstheyarere-estimatediteratively.

Itiscrucialtostartwith‘good’initialparametersasthealgorithmonly ndsalocal,andnotaglobaloptimum.Thereforethesolution(towherethealgorithmconverges)stronglydependsontheinitialparameters.

1.2.1Experiment:Backandfrontvowels

Thedatausedinthisexperimenthavealreadybeenusedinprevioustutorials.Loadthedata leBackFront.mat.The lecontainssimulated2-dimensionalspeechfeaturesintheformofarti ciallygeneratedpairsofformantfrequencyvalues(the rstandthesecondspectralformants,[F1,F2]).The5vowelshavebeengroupedintotwoclasses,thebackvowelscontaining/a/and/o/,andthefrontvowels,containing/e/,/i/,/y/.SincethedistributionforeachvowelisGaussian(cf.tutorial‘GaussianStatisticsandUnsupervisedLearning’),eachofthetwoclasses,backvowelsandfrontvowels,shallconstituteaGaussianmixturedistribution.ThedatasamplesforthebackvowelsandfrontvowelsaresavedinbackVandfrontV.

Youcanvisualizethedatabymakinga2-dimensionalplotofthedata:

>>plot(frontV(1,:),frontV(2,:))

Youcanalsoplota2-dimensionalhistogram,withthefunctionhisto.

>>histo(frontV(1,:),frontV(2,:))Recallthatyoucanchangetheviewpointina3-dimensionalMatlabplotwiththemouse,choosingtherotateoptioninMatlab.

Itispossibletodepictthe‘real’pdfoftheGaussianmixturesaccordingtoequation2,consideringtheparametersforthepdfsofthesinglevowels.Themeanvectorsandcovariancematricesaresavedinmeansfrontandvarsfront.

Mixtures of Gaussians 混合高斯模型,HMM 模式识别

Themeansaresavedina3-dimensionalarray.FirstdimensioncoversthemeanµofF1andF2,theseconddimensionisofsize1,andthethirddimensioncontainsthenumberofGaussianmixtures.TheseconddimensionisusedifthepdfispartofaHiddenMarkovModel(HMM)anddenotesthenumberofthestateintheHMM(seeBNTtoolkit[1]).

To ndthemeanfor[F1,F2]forthesecondGaussianforthefrontvowels(i.e.,themeanoftheGaus-siandistributionofvowel/i/)use:

>>meansThecovariancematricesarestoredina4-dimensionalarray.FirstandseconddimensioncontainsthecovariancematrixΣ,thethirddimensionisofsize1(forHMMs,seeabove),andtheforthdimensiongivesthenumberofGaussianmixtures.TogetthecovariancematrixforF1andF2forthesecondGaussianforthefrontvowelsuse:

>>varsYoucanverifythevaluesgiveninmeansandvars,resp.meansbackandvarsback,bycomputingthemeanvectorandthecovariancematrixforeachvowelusingtheMatlabcommandsmeananscov.

Theweightingfactorswk(seeequation2)forthefront/backvowelsarestoredinmmfront/mmback,intheformofarowvector.

Withtheseparameterswecanplotthe‘true’pdfusingthefunctionmgaussv.Itisadvisabletospecifyanx-rangeanday-rangeforplotting.

>>%wetakethex-andy-rangeaccordingtothe2-dimensionalhistogram

>>figure;mgaussv(meansfront,varsfront,mm1.3TrainingofparameterswiththeEMalgorithm

http://doc.guandang.netetheGaussian-Mixtures-EM-Explorertodothat.Tocalltheexplorertype

>>BW(data,initialparameters)

The rstinputargumentarethedatathatshouldbemodeledbythepdf.Thesecondinputargumentiseitherasetofinitialparameters(consistingofinitialmeansµk,covariancesΣk,andweightingfactorswk)asacell(seethedatastructureofhmmFortypehelpBWinMatlab),orthenumberofGaussianmixturesKthatshouldbeusedforthepdf.Inthelattercasea rstguessofthe …… 此处隐藏：10613字，全部文档内容请下载后查看。喜欢就下载吧 ……