Mixture of Expert Agents for Handling Imbalanced Data Sets

Abstract — Many real-world data sets exhibit skewed class distributions in which almost all cases are allotted to a class and far fewer cases to a smaller, usually more interesting class. A classifier induced from an imbalanced data set has, typically, a

46ANNALSOFMATHEMATICS,COMPUTING&TELEINFORMATICS,VOL1,NO1,2003,PP46-55

MixtureofExpertAgentsforHandling

ImbalancedDataSets

S.B.KotsiantisandP.E.Pintelas

EducationalSoftwareDevelopmentLaboratory

DepartmentofMathematicsUniversityofPatras,Hellas

{sotos,pintelas}@math.upatras.gr

Abstract—Manyreal-worlddatasetsexhibitskewedclassdistributionsinwhichalmostallcasesareallottedtoaclassandfarfewercasestoasmaller,usuallymoreinterestingclass.Aclassi erinducedfromanimbalanceddatasethas,typically,alowerrorrateforthemajorityclassandanunacceptableerrorratefortheminorityclass.Thispaper rstlyprovidesasystematicstudyonthevariousmethodologiesthathavetriedtohandlethisproblem.Finally,itpresentsanexperimentalstudyofthesemethodologieswithaproposedmixtureofexpertagentsanditconcludesthatsuchaframeworkcanbeamoreeffectivesolutiontotheproblem.Ourmethodseemstoallowimprovedidenti cationofdif cultsmallclassesinpredictiveanalysis,whilekeepingtheclassi cationabilityoftheotherclassesinanacceptablelevel.

IndexTerms—supervisedmachinelearning,imbalanceddatasets,expertagents.

T

I.INTRODUCTION

Ypicallyclassi ersareexpectedtobeabletogeneralizeoverunseeninstancesofanyclasswithequalaccuracy.Forexample,inatwo-classdomainofpositiveandnegativeinstances,theclassi erwillper-formonanunseensetofinstanceswithequalaccuracyonboththepositiveandnegativeclasses.Thisofcourseistheidealsituation.Inmanyapplicationsclassi ersarefacedwithimbalanceddatasets,whichcancausetheclassi ertobebiasedtowardsoneclass.Thisbiasistheresultofoneclassbeingheavilyunderrepresentedinthetrainingdatacomparedtotheotherclasses.Itcanbeattributedtothewayinwhichclassi ersaredesigned.Inductiveclassi ersaretypicallydesignedtominimizeerrorsoverthetraininginstances.Learningalgorithms,becauseofthefactthatthecostofperformingwellontheover-representedclassoutweighsthecostofpoorperformanceonthesmallerclass,canignoreclassescontainingfewinstances.Moreover,thedif cultytodistinguishbetweentherarecases(i.e.,trueexceptions)andnoiseisalsoresponsibleforpoorperformanceontheminorityclass.

Foranumberofapplicationdomains,ahugedis-proportioninthenumberofcasesbelongingtoeach

classiscommon.Forinstance,indetectionoffraudintelephonecalls[9]andcreditcardtransactions[25],thenumberoflegitimatetransactionsismuchhigherthanthenumberoffraudulenttransactions.Moreover,indirectmarketing(LingandLi,1998),itiscommontohaveasmallresponserate(about1%)formostmarketingcampaigns.Otherexamplesofdomainswithintrinsicimbalancecanbefoundintheliteraturesuchasraremedicaldiagnoses[27]andoilspillsinsatelliteimages[16].Thus,learningwithskewedclassdistri-butionsisanimportantissueinsupervisedlearning.Themachinelearningcommunityhasmainlyad-dressedtheissueofclassimbalanceintwoways.Oneistoassigndistinctcoststotraininginstances[7].Theotheristore-sampletheoriginaldataset,eitherbyoversamplingtheminorityclassand/orunder-samplingthemajorityclass[15];[12].Althoughmanymethodsforcopingwithimbalanceddatasetshavebeenpro-posed,stillremainopenquestions.Oneopenquestioniswhethersimplychangingthedistributionskewcanimprovepredictiveperformancesystematically.Anotherquestioniswhetherwecantailorlearningalgorithmstothisspeciallearningenvironmentsothattheaccuracyfortheextremeclassvaluescanbeimproved.

Tohandletheproblem,wedevelopedanAgent-basedKnowledgeDiscovery(ABKD)methodthatcombinesthetwo eldsofDistributedArti cialIntelligenceandMachineLearning.InABKD,anagentisasoftwareentitythatcan1)interoperatewithitsdatasourceand/orotheragents,2)receive/gatherrawdata,3)processandlearnfromthedatasourceorfromothersources,and4)coordinatewithotheragentstoproducerelevantandusefulinformation.Adoptingagenttechnologyprovidesparallelism,improvesthespeedandthereliabilityoflearningandassistsdevelopersindesigningdistributedlearningsystems.TheeffectivenessofourapproachisevaluatedovereightimbalanceddatasetsusingtheC4.5[22],5NN[1],NaiveBayes[6]asclassi ersandthegeometricmeanofaccuraciesasperformancemeasure[16].Inthefollowingsectionwebrie ydescribethe

c2003AMCT/TEILarissaISSN1109-9305

Abstract — Many real-world data sets exhibit skewed class distributions in which almost all cases are allotted to a class and far fewer cases to a smaller, usually more interesting class. A classifier induced from an imbalanced data set has, typically, a

S.B.KOTSIANTISANDP.E.PINTELAS“MIXTUREOFEXPERTAGENTSFORHANDLINGIMBALANCEDDATASETS”47

usedmachinelearningtechniquesandweexplainthereasonsfortheirpoorperformanceinimbalancedatasets.Section3reviewstheattemptsforhandlingimbal-anceddatasets,whilesection4presentsthedetailsofourapproach.Section5presentsexperimentalresultscomparingourapproachtootherapproaches.Finally,section6discussestheresultsandsuggestsdirectionsforfuturework.

II.LEARNINGTECHNIQUESANDALGORITHMSAsmallimbalanceintheclassdistributionisnotserious,butwhensomeclassesareheavilyunder-represented,manymachine-learningmethodsarelikelytorunintoproblems.Casesbelongingtosmallclassesarelostamongthemorefrequentcasesduringlearning,and,consequently,classi erssuchasdecisiontrees,Bayesiannetworksandinstance-basedclassi ersareunabletoclassifycorrectlynewunseencasesfromtheminorityclasses.Inthefollowingsubsectionswebrie ydescribedecisiontrees,Bayesiannetworksandinstancebasedclassi ersandwerefertothereasonsfortheirpoorperformanceinminorityclassofimbalancedatasets.

A.Decisiontrees

Murthy[20]providesarecentoverviewofexistingworkindecisiontreesandatasteoftheirusefulnesstothenewcomersinthe eldofmachinelearning.Decisiontreesaretreesthatclassifyinstancesbysortingthembasedonfeaturevalues.Eachnodeinadecisiontreerepresentsafeatureinaninstancetobeclassi ed,andeachbranchrepresentsavaluethatthenodecantake.Instancesar …… 此处隐藏：35270字，全部文档内容请下载后查看。喜欢就下载吧 ……