Luận văn Phương pháp bayes và ứng dụng trong mạng nơron

Op,n6chiia nhanto'dambaat6ngxacsu1th~unghi<$mb~ng1.Vi v~y, c6th@vie'tlu~tphanlOpBayesvoivi dl)hailOpdu'Oid(;lngtu'dngdu'dng: Quye'tdinh lOp Cl ne'u P(XICI)P(CI)> P(XIC2)P(C2)va quye'tdinh lOp C2 ne'u P(XIC2)P(C2)>P(XICl)P(Cl) Trang 10 Vi cacxac sufftP(XICi)va P(cDco th€ tinhdu'<Jctu m§:udffcho,lu~tBayes(j trencoi nhu'dc,lllgthlfchanhcuabai toantho'ngke phanlOp.Ta thffyca hai xac sufftP(XICi) vaP(Cj)la quailtrQngtrongbai toantho'ngke phanlOpdambaaclfcti€u xacsufft cuasaiso'phanlOp. Vi du 1.1:X6tbaitoanphanlOploaichimvoiP(CI)=P("d~ibang")=0.8vaP(C2) ~ =P("di~uhau") =0.2va dffbie"tcachamm~tdQxacsufftP(XICI),P(XIC2)'Gia sa do'ivoi mQtloai chimmoi,dffdodu'<Jckichco cuano la x =45cm.Ta tinh du'<Jc p(45Ici)=2.2828X 10-2va p(45IC2)=1.1053X 10-2.Vi v~ylu~tphanlOpdt!doanla thuQclOpci("d~ibang")vi P(XICI)P(CI)>P(XIC2)P(C2)'Giasadffbie"txacsuffthong di~uki~nrex)co gia tr~la p(45)=0.3.Xac sufftcuasai so'phanlOpla : P(classification_errorlx)=mill {P(CI,x), P(C2,x)} =min { P(X Ic\)P(c\), p(x IC2)P(C2) } = {0.0754,0.07}=0.07 p(x) p(x) 1.4.Bai tminphanlopBayest6ngquat Chungta se t6ngquathoa lu~tphanlOpBayestrongtru'ongh<Jpcacdo'i tu'<JngthuQcv~nhi~uhonhai lOpva cacd~ctinhdodu'<Jccuacacdo'itu'<Jngco nhi~uhonmOtbie"nd~ctru'ng 1.4.1.Di;}ctrling bQi- vectordi;}ctrting X6tcacd~ctru'ngiatr~thlfc.Tathaythe"cacd~ctru'ngiatr~thlfccuamOt do'itu'<Jngla mOtvectorcOtn chi~ux ERn x\ X2 (1.18)x= Xn Trang 11 voi Xi E R laph~nta thli i cuavectord~ctning.MQtvectord~ctningx (mQtmftu trongkh6nggian d~ctru'ng)thaythe'cho mQtd6i tu'Qngva du'QCcoi la mQtdiSm trongkh6nggianEuclide n chi~u.Sv th6hit%ncv thS cuavectorx baog6mcacdQ docuad6i tu'Qng.Vi dv, d6i voi d<;libangva di~uhall co thSdo du'Qchai d~ctru'ng cuachunglakichcovadQnhanhnhyn(d<;lngvectord~ctru'ng2chi~uX E R2) ~ [ Xl ] [ kiChco ]X = X2 = de)nhanhnhyn (1.19) 1.4.2.Lu~tphanlopBayeschocaed6itu'Q'ngdalop,dad~ctr1ing Cacd6itvongco thSdu'Qcphanvaonhi~ubonhaildpphanbit%t(cactr<;lng thaitv nhien).Nhinchung,cothSgiasamQtd6itu'Qngcoth6thuQCv~Ildp phan bit%t(1tr<;lngthaitvnhienkhacnhau) C={Cj,C2,...,Cj} (1.20) Vi dV,cothSphancacloaichimthanh4 ldp.Ky hit%uP(Cj)la xacsuftti~nnghit%m mad6itu'Qngtie'ptheosethuQcv~ldpCj.XacsufttP(Cj)(i=l,2, ...,I) tu'onglingvoi ty It%cacloai chimtrongldp thli i tronggiOih<;lncuavo so'cac loai chimdaquailsat du'Qc. Hamm~tdQxacsufttco di~ukit%n(tr<;lngthaicuahamm~tdQxacsufttco di~u kit%n)du'Qcky hit%uchungchotfttcacacldpla p(xlcj)(i=l,2, ...,I). Hamm~tdQxac sufttdiSmky hit%ula p(Cj,x) la m~tdQxacsufttmad6i tu'QngtrongldpCjva co vectorgiatrid~ctru'ngla x.Hamm~tdQxacsufttcodi~ukit%nP(cjlx)(i=l,2,1)xac dinhmQtd6itu'Qngldpla Cjdachomagiatridodu'Qcuavectord~ctru'ngla x,xac sua'tP(cjlx)la xacsuftth~unghit%mvataco: I Ip(Ci Ix)=1 i=1 (1.21) Trang 12 Tli 1;'thuye'txacsua'tadabie'tcacquailh~saudayv6i xacsua'tiennghi~mva xacsua'th~unghi~m: p(Cj,x)=p(cjlx)p(x),i=l, 2 p(Cj,x)=p(xlcj)p(cD,i=l, 2 (1.22) trongdorex)la hamm~tdQxacsua'tkhongdieuki~ncuabie'nd?ctrlingx.Tabie't rang: , p(x)=LP(x Ic;)P(c;)=p(x ICl)P(Cl)+"'+p(x Ic,)P(c,) i=1 (1.23) vdi i =1,2, ...,1.Theo dinh1;'Bayes P(XICi)P(Ci) . 1suyra P(Ci Ix) = ,1=1,2,..., p(x) (1.24) , Vi p(x)=LP(x Ic;)P(c;) nen ta co: i=1 P(ei I x) = p(x Ic;)P(c;) .~ ( ,1=1,2,...,1 f:tp x Ic;)P(c;) (1.25) D6i vdi cac d6i tliQngnhieu Wp vdi vectord?c trling nhieu chieu, lu~tphanWp Bayesla: "ChotrlidcmQtd6i tliQngv6i vectorgia tri d?c trlingtlidngling la x, gall d6i tliQng vaomQtWpCjvdi xac sua'th~unghi~mco dieuki~nWnnha'tP(cjlx)" ho?c: "ChotrlidcmQtd6itliQngvdivectorgiatri d?ctrlingdachox,gan d6itliQngvao WpCjkhi : P(c)x) >P(cjlx),i =1,2,...,1;i:;t:j" (1.26) Trang 13 Vi xacsua'tc6di@uki~np(cilx)kh6xacdinh,dungdinhly Bayesc6th~diSntaxac sua'tnay du'oid<;tngP(XICi),P(Ci)vap(x).Khi d6lu~tphanlOpBayesdu'Qcvi€t du'oi d<;tngthl,l'chanhnhu'sail : "Voi mQtd6i tu'Qngcho tru'ocva dffbi€t vectorgia tri d~ctru'ngx, gall d6i tu'Qng vaolOpCjkhi : p(x ICj)P(c) >p(x IcJP(cJ , i = 1,2, ...,1;i;>tj" p(x) p(x) (1.27) Bop(x)dm~uso'cua2v€ ba'tphu'ongtrlnhtrentac6lu~tphanlOpBayes: "Voi mQtd6i tu'Qngchotru'ocva dffbi€t vectorgia tri d~ctru'ngx, galld6itu'Qng vaolOpCjkhi : p(x Ic)P(Cj) >p(x IcJP(cJ, i =1,2, ...,1;i;tj" (1.28) 1.5.Ctfcti~uhoarui ro tronghili tminphanlOp TrongmQts6lingdvng~hu'chfindoany khoa,chungtathu'ongmongmu6n lamcl,l'cti~uh6asai so'chfindoan. Gia sU',mQtquy€t dinhphanlOpdl,l'atrenvectord~ctru'ngdffchox xacdinhd6i tuQngthuQclOpCjtrongkhid6itu'Qngthl,l'csl,l'thuQcv~lOpCi. Hamthi€u hvttrong truonghQpnay du'Qcxacdinhnhu'sau: Lij=L(decision_classjltrue_classD (1.29) D6ivoibaitoanphanlOpc61lOp,cacphffntU'thi€u c6d<;tngmatr~nvuongL Lll L12'" Lll I L2l Ln ...L2lL= (1.30)... ... ...... Lll LI2...Lll Trang 14 Xet svphanlOpcuamQtdO'itli<;5ngvOivectorgia tri d~ctIling x dffbi€t. Ta kh6ng bi€t lOp thvc Sv cua dO'itli<;5ngnay, no co th~la mQttrong sO't~ph<;5pcac lOp {Cl, C2,...,cd. GiasaP(Ci,x) Ia xac sua'tco di~ukit%nmadO'itli<;5ngthuQCv~lOpthvcSVCivoi di~ukit%nx. Ta sexacdinhky vQngco di~ukit%nthi€u lien quaildenvit%cquy€t. dinhdO'itli<;5ngthuQcv~lOpCj,trongkhi dO'itli<;5ngay thuQcv~lOpCj i =1,2, ...,1; i:;t:jla : I R(cj Ix) =IL(decision - classj Itrue- classJP(c/ Ix) /=1 (1.31) ho~cd d(;lngng~nhon : I Rj =I LijP(c/ I x) j=1 (1.32) Ky vQngthi€u Rj =R(Cj Ix) dli<;5cgQila di€u kit%nrui rooBi€u kit%nrui ronayco lien quail den quy€t dinh Cjdli<;5cquy€t dinhbdi vectord~ctIling x. N€u dffbi€t vectord~ctIling la x thl co th~lam cvc ti~udi~ukit%nrui ro Rj bftngvit%cdlia ra quy€t dinhphanlOpCj(jE {1,2, ...,I}) Lu~tphanlOp(quy€t dinhphanlOp)gallmQtlOpchom6i vectord~ctIling.Co th~ Xclcdinhrui ro toanbQla ky vQngthi€u co lienquaildenquy€t dinhphanlOpdff bitt,xetta'tcacacd(;lngcoth~cocuax cuakh6nggianvectord~ctIlingnchi~utli Rx I R =J R(cjIx)dx=J ILijP(c/ Ix)dx Rx Rx /=1 (1.33) Rui ro toaDbQR dli<;5ccoi nhlimQtlieu chufinphanlOp lam cvc ti~uhoa rui ro co lienquaildenquy€t dinhphanlop. Tli dinhnghlacua rui ro R ta tha'yrftngquy€t Trang 15 dinhphan lOp CjduQcduara dS dambaa di~ukit%nflU ro Rj =R(cj [x)cangnho dng tBtdBivoi m6id(;lngcuavectord~ctIlingx. UBi voi bai tminphanlOpt6ngqmlthall,taco lu~tphanlOpBayesnhusau: "UBi voi mQtdBi tuQngco vectorgia tri d~ctIling x dffbie't,UOCluQngta'tca di~u kit%nrui ro chota'tca caclOpco thSduQcCjU =1,2, ...,1) I R(cj Ix) =I LijP(cl Ix), U =1,2, ...,1) 1=1 (1.34) vachQnmQtquye'tdinh(mQtlOp)Cjsaochodi~ukit%nruiro R(cj Ix) lanhonha't: R(Cj[x)<R(ckIx),k= 1,2,...,I;k;t:j" (1.35) Rlii ro toanbQR d(;ltduQccl,l'ctiSu la ke'tquacuaquye'tdinhphanlOpBayesduQc gQila ruiroBayes. Dl;(avaodinhnghiadi~ukit%nrui ro R(ciIx), lu~tphanlOpBayesduQcvie'tduoi d~ng: "UBi voi mQtdBi tuQngco vectorgia tri d~ctIling x dffbie't,chQnmN quye'tdinh (mQtlOp)Cjsaocho I I I LijP(c;[x)<ILlkP(CI [x), k =1,2, ...,I; k;t:j" ;=1 ;=1 (1.36) Tli dinh19Bayes, P(CI Ix)=p(x IcJP(cJ p(x) (1.37) CothSvie'tl~ilu~tBayesnhusau: "ChQnmQtquye'tdinh(mQtlOp)Cjsaocho f L p(xlc;)P(c;) ~L p(xJcJP(cJ k =1 2 I.k ."L Ij <L Ik " ,..., , :;i:J 1=1 p(x) 1=1 p(x) (1.38) Bop(x)(j cahaive'cuaba'tphuongtrlnhtren,tacolu~tphanlOpBayesnhusau: Trang 16 "ChQnmOtquy€t d~nh(mOtlOp)Cjsaocho I I I Lijp(x IcJP(cJ<LLikP(X IcJP(cJ, k =1,2, ...,1;k:;t:j" iz] iz] (1.39) 1.5.1.Ctfcti~uhoariii ro trongbili tminphanlopBayes Thongthu'ong,mOtbairoanphanlOpcolienquaild€n vi~cquy€td~nhlOpCj (i=I,2,...,1)mamOtdo'itu'QngmoithuOcv€ lOpnay.Giasacacquy€Cd~nhphanlOp saitrongh~uh€t caclOp.VI v~y,n€u mOtdo'itu'Qngdu'QcphanlOpthuOcv€ lOpCj trongkhi thvct€ la lOpCjthl khi j=i, SvphanlOpIa dung,ngu'Qcl~ikhi i;t:jta co sai s6phanlOp. Ta d~nhnghlad.;mgdo'ixung0 - 1cuahamthi€u : { o i=j Lij =L(decision- classj Itrue- classi)= 1'. ., 1"*} (1.40) Hamthi€u (j tren du'Qcgall 0 khi phanlop dungva gall 1 khi phanlOpsai.Tfit ca cacsaiso'h~u nhu'la nhu'nhau. Taco: I I R(cjIx)=LLijP(Ci Ix)= LP(Ci Ix)=1- P(CjIx) iz] iz],i..j (1.41) P(CiIx)la xacsufitcodi€u ki~nmaquy€td~nhphanlOpCjla dungdo'ivoix dffchao Tli (1.36)ta dffbi€t lu~tphanlOpBayesdu'ara mOtquy€t dinhphanlOpcvc ti€u hoa di€u ki~n rui ro P(Cj Ix). Trong (1.41),d€ cvc ti€u di€u ki~nrui ro phai tlm quy€t d~nhphanlOpCjsaocho 1- P(Cj Ix) (1.42) nhonhfit,nghlala xacsufith~unghi~mcodi€u ki~nP(CjIx)IOnnhfit.Tli dotaco lu~tphanlOpBayescvcti€u xacsufittrungblnhcuasaiso'phanlOpnhu'sail: "ChQnmOtquy€t dinh(mOtlOp)Cjsaocho Trang 17 P(Ci IX»P(Ck Ix),k= 1,2,...,I;kt:j" (1.43) Theacachtrlnhbay nay, co th~timduQclu~tphanlOpdambaa cl;t'cti~uxac sua't trungbinhcuasai s6phanlOp.MQtxacsua'trungbinhcuasai s6phanlOpduQccai nhula mQttieuchuffnqtc ti~ud~h;l'achQnquy€t dinhphanlOpt6tnha't. Ne'uma tr~nthi€u L duQcchQnrheacachgall gia tri 1 ehata'tea caequy€t dinh ~ phanlOpkhongchinhxac va gall gia tri 0 ehaquy€t dinhphanlOpehinhxac(Ljj)=O (1.44) thilu~tphanlOpBayesqtc ti~urui ro clinggi6ngnhulu~tphanlOpBayescl;t'cti~u xacsua'trungbinhcuasai s6phanlOp. 1.5.2.T6ngquathoas1/phanloptheotieuchuftnhqply c1/cd~i D6i vOicacd6ituQngnhi6ulOp,nhi6ud~etrung,coth~dinhnghlamQtcach t6ngquattys6hQpIy ehalOpCjvaej p(xIe) . . - 1 2 I .",J,I-, ,..."J*1 p(x Ic;) (1.45) vagiatringuongt6ngquat (L - L )P(c. ) eoo = lJ II , lJ (Loo -Loo)P(c.).I' .1.1 .I (1.46) vi v~yrui ra nh6nha'tronglu~tphanlOprheatieuchuffphQpIy el;t'cd(;lico th~ du'Qcgiai thichla chQnmQtlOpsaGehaty s6 hQpIy IOnhonta'tea caegia tri ngu'ongkhacd6ivoicaelOpkhaenhau: "Quy€tdinhlOpCjn€u voimatrn thiu : 0 1 1 1 .1 0 1 1 L= I .... .... .... .... 1 1 1 0 Trang 18 p(xlei) >e. , i =1, 2, ...,1;i:;t:j"Jl p(x IeJ (1.47) D6i voi cachamthie'ud6i KungLjj =Ljj voi Ljj =0,tacolu~tphanlOpsauday: "Quye'td~nhlOpCjne'u p(x Ie) >P(eJ , i =1, 2, ...,1;i:;t:j" p(x IeJ pee) (1.48) 1.6.Quye't~nhmi~n,xacsua'tsais6 MQtquye'td~nhphanlOpchiakh6nggiand~ctIlingthanh1mi€n quye'td~nh roi nhau(kh6nggianquye'td~nhcon)RI, R2,...,RI' Mi€n Rj la kh6nggianconma m6i vector gia tri d~c tIling x rdi vao trong mi€n nay dliQc gall vao lOp Cj (Xemhlnh 1.2) Xacxua't P(XICI)P(CI) Quye'td~nhbien (j t6i1fu 1c:'"Quye'td~nhbien kh6ng t6i u'u EE 7~ RI R2 Hinh 1.2 : Quye'tdinh bien 3;.x Nhln chung,mi€n Rj (i =1,2, ...1)kh6ngc~nk€ nhauva co th~dliQcchia thanh nhi€umi€n conroi nhau,tuynhienta'tcacacvectortrongcacmi€n naysethuQcv€ lOpgi6ng nhau Cj. Cacmi€n giaonhauvacacbiengiuacacmi€n k€ nhaudliQcgQila quye'td~nhbien, VIcacquye'tdinhphanlOpcothayd6iquam6ibien. Trang 19 Cong vi~cphan lOp 1atim cac 1u~tphanlOp dam baa vi~cchia khonggian d~c tIlingvao cac miSn quye'tdinht6i uu R1,R2,...,Rl (voi cacquye'tdinhbien t6i uu). Cacquye'tdinhbient6iuucoth€ khongdambaavi~cphanlOpkhongco sail~m nhlingdambaasaisO'trungbinhnhonhfittheolieuchu§:nCVcti€u dffchQn. Ta segiaiquye'tkhokhantrongbai loanphanlOpnay d€ timcacmiSnva cac ~ quye'tdinhbient6i uub~ngcachphantichmQtphanlOphai lOpvoi vectord~c tIlingx,n chiSu.Gia sukhonggiand~ctningdliQcchia (coth€ khongt6iuu)thanh haimiSnkhacnhau,Rl tlidngling voi lOpCl va R2tlidngling voi lOpC2.Co hai SV ki~nlam xufithi~nsai sO'phanlOp. D~ulien, sai sO'xufithi~nkhi mQtvectord~c tIlingx do dliQcrdi vao miSnRl dliara quye'tdinhCl trongkhi do lOpthvcsv 1aC2. Thli hai, vectord~ctIling x rdi vaomiSnR2trongkhi lOpthvcsv 1aCl.Vi cahai sai s6naylo~itf\llin nhau,tacoth€ tinhxacsufitt6ngcQngcuacacsaisO'phanlOp khacnhau(DudaandHart, 1973;Bishop,1995) P(classification- error)=P(x E Rl ,c2) +P(x E R2,C1) (1.49) =P(x E Rl I C2)P(C2)+P(x E R2 IC1)P(C1) Sadl;lngdinh1;'Bayestaco: P(classification- error) = r p(x Ic2 )P(c2)dx + r p(x Ic1)P( c1)dx JRI JR2 (1.50) BaygiGtacoth€ dliafa bailoanphanlOpt6iuu1avi~cchQn1vaquye'tdinhmiSn Rt, R2 (vi v~y xac dinh dliQc lu~t phan lop) lam cvc d~i xac sufit P(c1assification_error)(j lIen (la mQtlieu chu§:nt6i uu). Xet mQtd6i tliQngvoi vectord~c trling x dff chaoXac sufit cua sai sO'phan lOp dliQc cvc ti€u ne'u p(xlct)P(ct)>P(XIC2)P(C2),tasechQnmQtmiSnRl vaR2saochox thuQctrongmiSn R1,vi diSunaydambaasaisO'phanlOpnhohdn(tieuchu§:ncvcti€u). DiSunay 20 tu'dngdlidngvoi lu~tphanlop Bayeslamct!cti€u xacsuc1tcuasais6trungblnh b~ngcachchQnmQtlOpvoixacsuc1th~unghi~mIOnnhc1t. MQtcongthucphanlOpt6iuudt!alIenvi~cct!cti€u xacsuc1tsais6phanlOpc6th€ du'<;lct6ngquath6ad6ivoi d6itli<;lngnhi€u lOpnhi€u d~ctIling.B6i voibailoan nhi€u lOp,vi~cxacdinhxacsuc1tquyetdinhphanlopdungd6hall.Cacd6itli<;lng du'<;lcphanlOpthanhI lOpphanbi~tCbC2,...,c],voivectord~ctIlingnchi€u KERn,ta coXaCsuc1ttrungblnhphanlOpdungcuacacd6i tli<;lngmOidt!alIen vectord~c tIlingx : I I P(classification_correct) =Ip(x ER;,cJ =Ip(x E R; IcJP(cJ ;=1 ;=1 (1.51) I =I JR p(x IcJP(c;)dx;=1 I trongd6Rj la mi€n quyetdinhc6 lienquaildenlopCj Nhu'v~ybai loan phan lOp la chQnIDQtquyetdinh mi€n Rj lam ct!cd(;lixac suc1t reclassification_correct)la mQtlieuchuffnt6iuu.Tieuchuffnnaydli<;lcct!cd(;libdi vi~cchQnmi€n Rj saochom6ivectorgiatri d~ctIlingx dli<;lcgallvaomQtlOpma tichphan t;p(x Ic;)P(c;)lact!cd(;li(d6ivoimQix) 1.7.Caehambi~tthue MQttrongnhl1'ngd(;lngc6di€n nhc1tcuavi~cphanlOpladi6ntad(;lngbailoan htachQnlOpdt!alIenvectorgiatrid~ctIlingx trongd(;lngkinhdi€n sad\lngt~ph<;lp rad€ xacdinhcachambi~thuc diCK),i =1, 2, ...,I (1.52) Trang 21 M6i bi~t thuc c6 lien quail de'nmOtlOp C1,1th~dii du'Qcnh~nd£;lngCj (i=I, 2, ...,1). B~titmin phan lOp sa d1,1ngcac ki~ubi~tthuc niiy gall mOt d6i tu'Qngvoi vector gia trid~ctIlingx dii bie'tcholOpCjne'u dj(x)>dj(x),voi mQii = 1,2, ...,1,i:;t:j. (1.53) N6i cachkhac,biii toanphanlOpgall mOtd6i tu'Qngviio mOtlOpsaochobi~tthuc tu'ongling c6 gia tri IOnnha't. Bi~tthucphanlOpdu'Qcthie'tke'Iii mOth~th6ng(Xem hlnh 1.3)chuat~phQpcac bi~tthucdj(x)(i =1,2, ...,1)c6 lien quailvoi m6i lOpCj(i =1,2, ...,1)clingvoi vi~c IvachQncacbi~tthucc6giatri IOnnha't dj(x), i =1,2, ...,1 (1.54) max(diCx»,i =1,2,...,1 Lu~tphanlOpdu'Qcdi~ntanhu'sau: Chotru'ocmOtd6itu'Qngvdivectorgiatrid~ctIlingx: 1.Tinhgiatrib~ngsO'cuata'tcacachambi~tthuctheox dj(x),i =1,2, ...,1 (1.55) 2. ChQnlOpCjIii mOttien doancua lOpth1;l'cs1;l'saocho gia tri cua hiimbi~tthuc dj(x)Iii IOnnha't,tucIii l1;l'achQnmOtlOpCjsaochod/x) =max(dj(x»,i =1,2,...,1 Hambi~tthucc6th~xacdinhb~ngnhieucachd1;l'atrentieuchucfnphanlOpt6iUu. Ch~ngh£;lnta xac dinh cac hambi~tthucchos1;l'phanlOpBayes c1;l'cti~uxac sua't cuasai sO'phanlOp.D6i voi bai toanBayes,s1;l'l1;l'achQncac hiim bi~tthucIii xac sua'th~unghi~mc6 dieuki~nP(cjlx) diCx)=P(cjlx), i =1,2, ...,1 (1.56) Sad1,1ngdinhly Bayes,xacdinhd£;lngth1;l'chiinhcua hiimbi~tthuc 22 d ( )=p(x ICj)P(cJ .=1 2 IiX, 1 " ..., p(X) di(x)=p(xlci)P(Ci),i= 1,2,...,1 (1.57) (1.58) Tatha'yding,chicogiatq cuahambi~tthlicla quailtrQngtrongvi~cxacdinhlOp. VI v~y,cachamdondi~utangf(dj(x))cuadj(x)sedu'aramQtquy€tdinhphanlOp d6ngnha't.Coth~tlmd~ngtu'ongdu'ongcuabi~thlicBayesb~ngcachiffylogaritt! nhiendiCx)cuahambi~tthlic. di(x) =Inp(x ICi)+InP(Ci)' i =1,2, ...,I (1.59) Lu~tphanlOpdu'<;1cxacdinhb~ngcachchQncachambi~tthlicdiCx)chom6ilOpCj. Tucla, khonggiand~ctru'ngdu'<;1cchia thanhI mi€n khacnhauRj (i =1,2, ..,I).M6i mi€n co lienquaild€n quy€t dinhphanlOp,n€u vectorgiatri d~ctru'ngx roi vao mi€n Rj thl mQtd6i tu'<;1ngdu'<;1cphan lop thuQcv€ lop Cj. Hlnh1.3 NghIala,n€u chotru'ocvectorgiatrid~ctru'ngx cod/x) >dj(x),i =1,2, ...,I, i:t:j,thl x seroi vao mi€n tu'ongling Rj va quy€t dinhphan lOpgallmQtd6i tu'<;1ngmoi vao Bit thlic d)(x) Lop, cholOpc) It!a chQn Bit thlic dz(x) Max.... cho lOpCz d)(x) Bit thlic ..... .... cholOpc) Trang23 lOpC}Cachambi<$tthucnayxacdinhcacquy€t dinhbien.Cac quy€t dinhbien xacdinhmQtb~m~trongkh6nggiand~ctntng,d d6cacquy€t dinhphanlOpthay d6i.D6ivoicacmi~nk~nhauRjvaRj,quy€tdinhbienchiacacmi~nd6c6th~tim du'Qcbdi phu'ongtrlnh quail h<$ham bi<$tthuc. d/x) =dj(x) (1.60) ~ Cacquy€tdinhbienkh6ngbianhhu'dngbdicachambi<$tthucdondi<$utang. T6ngquat,nli ro nhonha'trongphanlOpBayesd1,iatrencachambi<$tthucdu'Qc xacdinhla dj(x)=- R(cjlx), i =1, 2, ...,1 (1.61) 7.1.1.Hambi~tthucGausstrongbaitminnh~nd~nghailOp D6i voi bai loanphanlOphailOpClva C2,tadffxacdinhhaihambi<$tthuc d1(x)va dz(X)c6 lienquaild€n m6ilOp.Cachambi<$tthucnayxacdinhhaimi~n Ri vaRz trongkh6nggiand~ctru'ngdu'Qchiabdiquy€t dinhbiend d6cacham bi<$tthucb~ngnhau di(x)=dz(x) (1.62) vaquy€t dinhphanlOpthayd6i tITlOpCl sanglOpC2.MQt d6i tu'Qngvoi vectord~c tru'ngx du'QcphanlOpvao lOpCl (thuQcv~mi~nquy€t dinh R1)n€u d1(x)> dz(x) vangu'Qcl£,tid6itu'Qngd6du'QcphanlOpvaolOpCz(thuQcmi~nquy€tdinhRz).Ta tha'yrangtrongbai loanphanlOp2 lOp,kh6nggiand~ctru'ngdu'Qchiathanhhai mi~nphanbi<$t,vi v~ytac6th~xayd1,ingmQthamgQila hamphand6ivoiham bi<$tthucdon d(x)=di(x) - dz(x) (1.63) Voi x chotru'oc,tinhgia tri phand6icuahambi<$tthucdond(x)va gallvaomQtlOp d1/atrenda'ucuagia tri nay. 24 B6ivoicachambi~thucduQcilIachQnd1!avaolu~tBayes,tacoth€ xacdinhduQc hamphandoi voibailoannh~nd~.mg2lOp d(x)=P(CIIx)- P(C2Ix) d(x)=p(x ICI)P(Cl) - p(x IC2)P(C2) (1.64) (1.65) ho~cd(x)=Inp(x ICI) +In P(CI) p(x IC2) P(C2) (66) 1.7.2.Bi~tthucd~ngtoimphtidngvatoytn tinhtheolo~UBayes Trongcacphfintruoctada:cod;;mgsaildaycuahambi~tthucBayes di(x)=lnp(x IcJ+ InP(cJ, i=1,2,...,1 (1.67) BailoanphanlOpd1!atrencacbi~thucnaygallmQtd6ituQngvoivectord~ctrung x cholOpCjvoi hambi~tthucla IOnnha't.Ghl savectord~ctrungx cophanph6i chu£n hi€u chi€u Gausstrongph(;lmvi m6ilOp.VI v~ym6ithanhphfincuavector d~ctrungcophanph6ichu£nnhi€u chi€u ho~cphanph6iGausstrongph(;lmvi m6i lOp.D(;lngvectorcuaphanph6ichu£nho~cphanph6iGausscuahamm~tdQxac sua'tp(xICj)d6ivoivectord~ctrungxtrongph(;lmvi lOpCjduQc hobdibi€u thuc I 1 [ IT ,,-I ]p(x Ci)= /2 1 1 1/2 exp- 2(x-pJ "-i (x-pJ(271Y Ii (1.68) voiPiIatrungbinhcuavectord~ctrUnglOpthui, Ii la matr~nhi~pphuongsaicua vectord~ctrunglOpthui, IIi Ila dinhthuccuamatr~nhi~pphuongsai,n Ia sO' chi€ucuakhonggianvectord~ctrungx. Thayphuongtrinh(1.68)vaophuongtrinh(1.67),tacod(;lngsaildaycuahambi~t thuc di(x)=In 1 1/2exp[ - !(x - pJT I~I(x- PJ ] +lnP(cJ, i =1,2, ...,1 (27l"r/2IIil 2 (1.69) Trang25 d;(x)=-~lnIL;I-~(X- JiJT L~l(X- JiJ-; In(27l")+lnP(e;),i =1,2,...,1 (1.70) Io~itrll %In(27l"), taco d;(x)=_! InIL;I- !(x - JiJT L~l(x - Ji;) +InP(eJ, i =1,2, ...,12 2 (1.71) Bi~tthuc(j tren1amOtd~nghamb~chaicuavectord~ctIlingx d6ivoi P(eJ va I;. No duQCgQi1abi~tthucb~chai.Cacquye"tdinhbiengiii'acaclOpi vaj co diCK)=d/x) 1ahamsieub~chaitrongkh6nggiand~ctIlingn chi~u. Bai toanphanlOpdlfatrenbi~tthucb~chaiBayesduQcKaydlfngnhusau: rho m~ux, gia tri m~tdOxacsua"tco di~uki~np(xlcj) va xacsua"tti~nnghi~m P(cDd6i voi ta"tCelcac lOp i =1, 2, ...,1 1. Tinh gia tri cua vectortrungbinh Ji; va matr~nhit%phuongsai L; chota"tCel caclOpi =1,2, ...,1dlfatrent~phua"n1uy~n 2. Tinh cacgia tri hambi~tthucchota"tCelcaclOp d;(x)=- ~In[L;I-k(x- Ji;f L~l(X- JiJ +InP(eJ, i =1,2,...,1 (1.72) 3. ChQnlOpCj1amOtdlfdoancualOpthlfcslf saochogiatri cuahambi~tthuc d/x) 1aIOnnha"t Tilc la, chQnlOpCjsaochod/x) =max(diCx)),i =1,2, ...,1 Ke'tqua:Lop daduQCdlfdoan - 1.7.3.Bi~tthuctuye'ntinh : DiingthuctronglOpmatri.inhiepphuongsai Gielsli'cachi~pphuongsaib~ngnhautrongta"tCelcaclOpL; =L (i =1, 2, ...,1). TrangtruonghQpnay,tronghambit%tthucb~chai,ph~ntli' InIL;!=ILlla lOpdOc I~p,vi V?yco th~vie"thambit%tthuc(j d,;mg Trang26 d;(x) =-~(x- f.1;f 2:-I(X- f.1;)+InP(c;), i =1,2,...,12 (1.73) Taco: I ( )T"'-I ( ) 1 T,,-I 1"'-1 1 T,,-I 1 T",-I- X-II. L.. X-II. =-x L.. X--L.. 11.--11. L.. X+-II. L.. II. 2 r, r, 2 2 r, 2r, 2r, r, VI 2:1ad6i Kungnen 2:-1clingd6i Kungva d day ~XT2:-1f.1i=~f.1r2:-1x, hdnnil'a,2 2 ~ ph~n tlY ~XT2:-1X 1alOpdQCl~p,VI the"no co th€ bi lot;litIll, khi do taco dt;lngham bi9tthuc 1 d;(x)=f.1r2:-1X--f.1r 2:-1f.1i+InP(ci), i = 1,2, ...,12 (1.74) Bi9tthucd tren1amQthamtuye"ntinhcuavectord~ctIlingx, nodliQcgQila ham bi9tthuctuye"ntinh.Quye"tdinhbiengiualOpi valOpj saochodiCK)=dj(x)la mQt ph~ncuasieuph£ngtrongkhonggiand~ctIlingnchi~u. D6ivoi bai tmlnphanlOphai lOpvoi vectord~ctIlinghaichi~u,quye"tdinhbien giil'acaclOp1adliongth£ng(Xemhlnh1.4) QuatrlnhphanlOpslYd\lngbi9tthuctuye"ntinhdliQcKayd\l'ngnhlisau: 1.Voi x dffcho,Hnhgiatrib~ngso'cuacachambi9tthucchota'tcacaclOp di (x) =f.1r 2:-1X - ~f.1r2:-1 f.1i+1nP(cJ, i =1,2, ...,12 (1.75) 2. ChQnmQtlOpCjsaochogiatri cuahambi9tthucd/x) 1aIOnnha't. Tuc la, chQnmQtlOpCjsaochod/x) =max(diCx)),i =1,2,...,1 Trang27 X2 :7 Xl Hlnh 1.4 Vi du1.2 Gia sam~u2 d~ctIling x E R2 tu hai lOpCl=0 va C2=1com~tdQphanph6ichu§'n Gauss(Xembang1.1) Bantll : T~phu~nluy<%ndffduQcchia thanh2 lOpkhacnhau Tli t~phQpm~unay,tinhvectortrungblnhvamatr~nhi<%pphuongsaichom6ilOp. GiatritrungblnhduQcuocluQngla: ~ 1 5 [/11="5L:>LdPl,i = 2.2]1=1 .0 (1.76) 0 Ham phan d6i 0 dl(x) =d2(x) 0 0 0 x 0 /x-lOP 10 x 0/ 0 0 - lOp 2 x 0 x x Lop 1 Xl X2 Lop 1 2 0 2 2 0 2 3 0 3 1 0 3 2 0 Lop 2 Xl X2 Lop 6 8 1 7 8 1 8 7 1 8 8 1 7 9 1 Trang28 ~ 1 5 [Jl2 ="5LXLdP 2,i = 7.2]l~ 8.0 (1.77) voi xLdpj,iky hi~um~uthui tu lOpj. Doc luQngmatr~nhi~pphuongsaieholOp11a t =5~1t(XLdp l,i - 111)(XLdP l,i - 111)T= (1.78) 1 [ 1- 2.2 ] [ 2- 2.2 ] [ 2- 2.2 ] -[ [1-2.2,2-2]+ [2-2.2,2-2]+ [2-2.2,3-2]+ ~ 4 2-2 2-2 3-2 [ 3- 2.2 ] [ 3- 2.2 ] 1 [ 2.8 -1.° ] [ 0.7 - 0.25 ] [3- 2.21- 2]+ [3- 2.22- 2]=- = 1- 2 ' 2- 2 ' . 4 -1.0 2.00 - 0.25 0.5 Tu'ong tl;l', I2 = [ 0.7 - 0.25 ]- 0.25 0.5 (1.79) Ta tha'y, caeuoe luQngeuama tr~nhi~pphuongsai d6i xungla nhunhau, II =f2 =fi, VIv~yd<;lngtuye'ntinhdongianhoneuacaebi~tthueduQesadvng di(x)=I1rf-l x-~l1r f-l l1i+lnP(cJ, i=l, 22 (1.80) Ta tinh I-I = [ 1.73913040.8695652 ]0.8695652 .4347826 (1.81) ftr f-I =[5.5652174,607826087] ft~f-l =[19.478261,25.739130] (1.82) (1.83) l~T~-I~ 1 "2JlI L, JlI =2x25.808696 (1.84) 1 ~ T ~-I ~ 1 2. Jl2 L, Jl2 =2x 346.15652 (1.85) Trang29 va InP(cl) =InP(cz)=InO.5=-0.6931 Cachambil$thuctuye'ntint la dj(x)=5.5652174xj+6.7826087x2- 22.9043- 0.6931 d2(x)=19.478261xj+25.739130X2- 173.0833- 0.6931 (1.86) (1.87) Hamphand6i trongbai toanphanlOphai lOpthaychoquye'tdint bienBayesgiua hailOpdu'Qcmint hQad Hinh 1.5 d(x)=dj(x) - d2(x)=0 d(x)=-13.9130xj- 18.9565x2+150.1790=0 (1.88) ho~c X2=-0.7339xj+7.9219 D5thihamphand6iHimQtdu'ongth~ngchiadamdulil$ucuacacm~uthuQcv~hai lOp.Quye'tdint phanlOpm~u"tren"(d(x)<O)thuQcv~lOpC2va quye'tdint phan lOpm~u"du'oi"(d(x»O)thuQcv~lOpCj Tatha'ydingcacm~udulil$ukh6ngch5ngcheolennhauvaytu'dngphanlOpcua m~uhua'nluyl$nnaycoth~th\fchi~ndu'Qcbdicacbi~tthuctuye'ntinhvacacquye't dinhbien tuye'ntint. D€ xay d\fngbi~tthuctuye'ntinhchovi~cphanlOpm~umoi Xli =[4, I]T tatinhcac gia tri cua hambi~tthucchohai lOpdj([4, I]T) =5.4461va d2([4,I]T)=-51.3416, gallm~unayvaolOpCjVIdj([4,I]T)>d2([4,I]T). Voi m~uxj2=[6,7]T, tacodj([6,7]T)=57.2722vad2([6,7]T)=123.2671,gall m~u nayvao lOpC2VI dj([6,7]T)<d2C[6,7f) X2 X f.!l /?X . X . It x x d(x)>0 0 11 Trang 30 0 0 0 . 0 f.!? 0 Quye'"tdinhbien d(x)=dl(x) - d2(x)=0 d(x)<0 Xl Hlnh 1.5 Ta co th~sa dl;[nghamphand6i chobai loanphan!dp,vOiXll =[4, If, d([4, I]T)= 75.5705> 0, khi do gall m§:unay thuQcv€ !dp CI va Xl2=[6, 7]T , d([6,7]T)=- 65.9945<0, khi do gallm§:unaythuQcv€ !dpC2. Hambi<$tthuctuye'"ntinhclingco th~trlnhbayd:;mg"M~ngnorontuye'"ntinh" di(x)=w;x+u';o,i= 1,2, ...,1 voivectortrQngluQngduQcdinhnghIala ,,-1 W. =L..., II.I r, vanguongduQCxacdinhbdi WiD=-fl; I-I fli +InP(ci) (1.89) (1.90) (1.91) D6ivoi cacmi€n k€ nhauRj vaRjcho!dpi vaj, quye'"tdinhbiengifi'acac!dpnay quailh<$voinhautheophuongtrlnhtuye'"ntinh di(x) - dj(x) =bx+U'ijD voi b=(Wi- W)T =(fli I-I -flj I-I)T (1.92) (1.93) WijD=WiD - WiD=-fl; I-I fli +InP(cJ +f1~I-I flj -lnP(c) (1.94) 31 Sieuph~ngchialOpi valOpj nhinchungkhongtn!cgiaovoidu'ongth~ngdiquacac trungbinhJiiva Jij (DudaandHart,1973). Neugiasamatr~nhi~pphu'ongsaivacacxacsua"ttiennghi~mb~ngnhaud6ivoi ta"tcacaclOp,tucla Ii =I (i =1,2, ...,I) vaP(Ci)=P thi co th€ boquaso"h~;lllg InP(Cj),khi dobi~tthuccodl.lng T ,,-lIT ,,-1 . 1 2 Id;(x)=Jii L., x--Ji; L., Ji;, 1= , , ...,2 (1.95) Tatha"yr~ng,vi~cphanlOpd\l'atrendinhthucdtrenIagallmQtd6itu'Qngvoivector giatrj d~ctru'ngx vaolOpj saochobinhphu'ongkhoangcach(x - Ji;)T I-I (x - Jii) cuax denvectortrungbinhJij la nhonha"t.Noi cachkhac,vi~cphanlOpd dayla h!achQnlOpCj saochogiatri x g~nnha"ttheonghlakhoangcachMahalanobis, tlfonglingvoi vectortrungbinh Jij . Bai toanphanlOpnaydu'QcgQilaphanlOptheo khoangcachnhonha"tMahalanobis. Ta co th€ till dl.lngtuyentinhcuabi~tthuctrongbai toanphanlOptheokhoang cachnhonha"tMahalanobis(khaitri€n khoangcachb~chaiMahalanobis) d;(x)=Jir I-I x - ~Jir I-I Jii , i =1,2,...,I2 (1.96) f)~ngthuctrencodl.lnghamtuyentinhcuavectord~ctru'ngx Thu~ttoancuabaitoanphanloptheokhoangcachnhonha"tMahalanobisdu'Qcxay dtfngnhu'sau: Cho trtioc : Vectortrungbinhchota"tcacaclOp Ji j , i =1,2,...,I vachotru'ocgiatri cuavectord~ctru'ngx. 1.Tinhgiatq b~ngso"cuakhoangcachMahalanobis(x - JiJT I-I (x - JiJ giuax vavectorgiatritrungbinh Trang32 2.ChQnlOpCjla d1,1'dmlncualOpth1,1'cS1,1'saochokhoangcachMahalanobisnho nha't (x- f.1JT 2:-1(x- f.1i)=min(x- f.1if2:-1(x-f.1J, i = 1,2, ...,1I (1.97) Ke'tqua: Lop daduQCd1,1'doan Hambi~tthucse trd nen don gianne'ugia sa ding ma tr~nhi~pph~ongsai b~ng nhautrongta'tca caclOpva cacd~ctIlingla nhl1ngmftuth6ngke dQcl~p.Trong truonghQpnay,m6i mftud6uco phuongsaia2va matr~nhi~pphuongsaitrd thanhmatr~nduongcheo I =a21 (1.98) voi I Ia matr~ndonvi dip nxnva IIil =O'2n,I~l =I-2 I , khi do taco di~mgbi~t thucdongianhon Ilx- f.1i112 . dj(x) = 2 +InP(cJ, 1=1,2, ...,1 20' (1.99) voi 11.IIIaky hi~ucuachuftnEuclide Ilx - f.1ill =~(x - f.1i)T(x - f.1i) (1.100) Taco: (x - f.1i)T(X - f.1i) =xTx - 2f.1{x+ f.1{f.1i (1.101) 1 T T T ~ dj (x) =---dx x - 2f.1ix +f.1if.1i]+InP(cJ20' VI s6h"mgxTx lanhunhautrongta'tcacaclOpnennoduQcboqua,khido (1.102) 1 TIT . di (x) =2 f.1iX - z f.1if.1i+In P( c,.), 1= 1, 2, ...,1a 20' (1.103) Bi~tthuctuye'ntinhnaycothSduQcvie'tduoidqng"mqngnoron" di (x) = w{x + WiD (1.104) Trang33 vectortrQnglu'Qngdu'Qcdinhnghiala 1 Wi =~ f.1ia- (1.105) vangu'ongtrQnglu'Qngla 1 T WiD=- 2a-2f.1if.1i+lnP(cJ (1.106) Quye'tdinhbien la nhungph~nsieuph£ngdu'Qcxac dinhboi phu'dngtrlnhdiCx)- dix). D6i vdi cacmi€n k€ nhauRj va Rj cualOpi va j vdi xac suffth~unghi~mIOn nhfft,cacquye'tdinhbiengiuacaclOpdu'Qcxacdinhtheophu'dngtrlnhtuye'ntinh di(x)-dj(x)=bx+wijO (1.107) voi TIT b =(Wi - W j ) =_2 2 (f.1i- f.1j ). a- (1.108) va 1 T T wijO=-~(f.1i f.1i-f.1jf.1j)+lnP(cJ-lnP(c j .)2a- . (1.109) 1.7.4.B~litoaDphan lOptheokhoangeachEuclide nho nha't Ne'ugia sa cac matr~nhi~pphu'dngsai b~ngnhaud6i vdi tfftca cac lOpva cacd~ctIlingla nhungth6ngke dQcl~pthltaco I =a-2I. Hdnnua,ne'ucacxac sufftti€n nghi~mb~ngnhau P(cD =P trongtfftca cac lOpthl co thSbo quaso'h<;lllg InP(cj).Til do,tacod<;tngddngianhdncuacacbi~tthuc Ilx - f.1i112 . di(x)=- 2 ,1=1,2,...,12a- (1.110) voi Ilx - f.1ill=~(x - f.1JT(x - f.1J (1.111) Trang34 Ta tha'yding, vi~cphanlOpdlfa tren cac dinhthlic (J tren gall mQtdO'itu'Qngvoi vectorgia tri d~ctru'ngx dffbie'tvitolOpj saochoImmingcachEuc1ideIlx- ,ujlltux Wi vectortrungblnh ,uj Ia nho nha't.N6i cachkhac,vi~cphanlOpchQnlOpCjsao chogia tri x Ia g~nnha'tvoi vectortIlingblnh ,uj tu'dngling.CachphanlOpnay du'QcgQiIaphanlOpdlfatrenImmingcachEuc1idenhonha't. D~ngtuye'ntinhcuadinhthlicphanlOpdlfatrenImmingcachEuc1idenhonha'tIa: di (x) =,urx - ,ur,ui' i =1,2, ...,I (1.112) Thu~troanphanlOpdlfatrenImmingcachEuc1idenhonha't Chotnioe : VectortrungblnhcuacaclOp,ui(i =1,2, ...,1)va giatrix cuavector d~ctru'ng 1.Tinhgiatri bangsO'cuakhoangcachEuc1idegiuax va trungblnh ,uichota't ca cac lOp n Ilx- ,uill= JL)Xk- ,uk,i)2 , i =1,2,...,I k=l (1.113) 2.ChQnlOpCjIa giatri tiendoanlop thlfcslf saochogiatri cuakhoangcach Euc1ideIanhonha't: IIx- ,uj II =minllx- ,uiII, i =1,2, ...,I (1.114) Ke'tqua: Lop dffdu'Qcdlf doan 1.8.Doc hiQngm~tdQxaesua't C6 th~xay dlfngthutl;lCphanlOpto'iu'uBayessaochoxacsua'tcuasaisO' phanlOpla nhonha'tkhidffbie'tcacxacsua'ti~nnghi~mP(Ci)valOpm~tdQxac sua'tc6di~uki~np(xlcj)voi ta'tcacaclOpCj (i =1,2, ...,1).Trongthlfcte',chungta Trang35 h~unhukhongbie"the"tcacd~cHnhxacsua'tcuacacm~u,cacd6itu<Jngmasl!hi€u bie"tnaykhongd~ydu,moh6 £)@uoclu<Jngcacxacsua'ti~nnghi~m,chungtadungcongthuc A nc.. P(cJ =---'-,1= 1,2, ...,1N (1.115) voi nCila s6d6i tu<JngtranglOpthu i cuam~ugiOih~n,N la tangs6~cacd6i tu<Jng trongmau. TrangquatrlnhBayes,c~nph.Huoclu<Jngm~tdQxacsua'tcodi~uki~np(xlcj).Co 3phuongphapg~ndungdu<Jcs\i'd\lngd€ doclu<Jngm~tdQxacsua't(Dudaand Hart,1973;Bishop,1995) I.Phuongphapthamso'(voi gia thie"tC\lth€ v~d~nghamcua hamm~tdQxac sua't) 2.Phuongphap phi thamso'(khongco gia thie"tC\lth€ v~d~nghamcuaham m~tdQxacsua't) 3.Phuongphap mJ'athamso'(ph6i h<Jpgiua phuongphap thamso'va phuong phap phi thamso') - Phuongphapthamso'gias\i'd~nghamC\lth€ cuahamm~tdQxacsua'tvoi mQtso'C\lth€ cacthamso'.Vi~cdoclu<Jngtrongphuongphapthams6la uoc lu<Jngcacgiatrit6iu'ucuacacthamso'. - Phuongphapphithamso'khonggias\i'd~nghamC\lth€ cuahamm~tdQxac sua'tmaphaixacdinhhamnaydl!avaocacdli li~udffchao - Phuongphapn\i'athamso'ph6ih<Jpgiuaphuongphapthamso'vaphuongphap phi thamso'.Phuongphapnay gia s\i'd~nghamtangquatcua hamm~tdQ xac sua'tva so'cacthamso'co th€ co. 36 Ngoaira, phlidngphapm~ngndrondIng dliQcdungd~lidc lliQnghamm~tdQxac sua't(DudaandHart, 1973;Bishop,1995) 1.8.1.Phu'dngphap thams6 Gia sa da:co cac quail satv€ cac d6i tliQngva mftutlidngling co N phffnta thuQcv€ 1lOp Cj (i =1, 2, ...,1) x ={xl,x2,...,xN} (1.116) CacmftudliQcd~ttentheocaclOp,coth~chiat~phQpta'tcacacmftuX theolOp thanh1t~phQproinhau XCt,XC2""'XC/' UXi=X i~I.2 c/ (1.117) i M6it~phQpXcchuaNjphffntathuQcv€ lOpCjtlidngling,giasacacmftutITt~phQp I Xc dQcl~pvdinhauvacohamm~tdQxacsua'tcodi€u ki~np(xlcj),(i =1,2, ...,1). I Giasad~nghamm~tdQxacsua'tla p(xIci,BJ (1.118) voiBi=(BipBi2,...,BimflamQtvectorthams6mchi€u cualOpthui. Vi d\l,ne'uham m~tdQxacsua'tcod~ngGausschu~n I 1 [ IT ,,-I ]p(x c;)= /2 1 " 1 1/2 exp-_2(x- pJ L...i(x- pJ(2nY L...i (1.119) thivectorthamsO'coth~dliQct~othanhvdivectortIlingbinhPi vacacphffntacua matr~nhi~pphlidngsaiLi °i=(P{,LiI'Li2,...,Linf (1.120) voi Lij la phffntahang thu i cua ma tr~nhi~pphlidngsai Li' Trong phanph6i chu~n,vectorthams6mQtchi€u chuatIlingbinhvaphlidngsai Trang37 0,=[::,] (1.121) VI oagiasacacm~utrongm6ilOpHioQcl~ptoanbQnenvi<$cUDCluQngcactham s6cuahamm~toQxacsufitouQctinhtudngtt;l'.U@odngian,taky hi<$up(xI0) thayVI p(x ICi,Oi)' Bai toanUDCluQngcacthams6t6i uuco th@ouQcthie'tl~pnhusau:Gia sacho truDcm~ug8mN ph~ntaXi oQCl~ptoanbQ X={X1,X2,...,XN} (1.122) Taclinggiasacacthams6di;mgp(xI0) cuahamm~toQxacsufitcooi€u ki<$noa bie't.C6ngvi<$cUDCluQngla tlmgiatrit6iuucuavectorthams6 0 mchi€u. VI cacph~ntaXi oQCl~p theo phan ph6i p(x I0) nenxacsufitoi@mcuaHitcacac m~utut~pdii'li<$uX ouQcvie'tdUDid(;lngtichcacxac sufit N L(O)=p(X I0)=TIp(Xi 10) i=l (1.123) Hamm~toQxacsufitL(O)la hamtheovectorthams6 0 d6ivDit~phQpm~uX oa cho,noouQcgQilahamhQply cua0 o6ivDit~phQpm~uX Ham L(0) ouQcchQnla mQttieu chugno@HmUDCluQngt6i uu cua 0, UDCluQng naygQila UDCluQnghQply ct;l'co(;licua thams6 O. Trongky thu~tnay,gia tr!t6i uu e cuavectorthams6 ouQcchQno@lamct;l'co(;liham L(O). £)@tinhtoanouQcd€ danghdn,lfiy logarittt;l'nhien cua ham hQply L(O) va UDC lu'Qngt6i UUouQcchQnb~ngcachtlmct;l'cti@ucuaham J(O) (tudngling vDivi<$ctlm evco(;licuaham L(0)) N J(O) =-lnL(O) =-z)np(xi 10) i=! (1.124) Trang38 Bai toanuocluQngthamsf)tf)iu'uhQply qic d::tiduQcphatbi~unhusau.Gia sti' chotruocmQtt~phQpduQcgiOih::tng6mN phffntti'dQcl~ptoanbQdf)ivoi lOpdff cho x ={xl,X2,...,XN} (1.125) Cling gia sti'dffbie'td::tngthamsf) p(x Ie) cuahamm~tdQxacsua'tcodi~uki~n. Ml,lcdichcuabaitoanuocluQngla tlmgiatritf)iu'ucuavectorthamsf)e mchi~u theotieuchu:1nqic ti~u N l(e) =- I)np(xi Ie) i=l (1.126) Giatriqic ti~uhamkhavi cuae duQctlmtut~phQpmphuongtrlnh al(e) ael al(e) ~l(e) = I ae: I =~ [ - i)np(xi Ie) ]ae ae ~ (1.127) al(e) aem a[-I:lln p(xiIe)1 ael a[-I:lln p(XiIe)) ae2 =0= a[- tlnP(X' 10)] aem Trang39 Do'ivoid<;lngehu§'nN(I',L) euahamm~tdOxaesufftvoi tham§'nI' vaL, dStlm vectorthamso'8 taco thSHmmOte6ngthued<;liso'ehou'oelu'c;1nghc;1ply eved<;li (DudaandHart, 1973;Bishop,1995) ~ 1 ~ .I'=- L,.x' N ;=1 (1.128) ~ 1~( i A )( i A )TL.=- L,. x -I' x -I' N i=1 (1.129) Doc lu'c;1nghc;1ply eved<;lif1 do'ivoi trungblnh I' la trungblnhm~uva u'oelu'c;1ng hc;1ply eved<;liI euamatr~nhi~pphu'dngsaiL latrungblnhso'hQeeuaN matr~n (Xi - f1)(xi- f1)T Doc lu'c;1ngd<;lngehu§'neuahamm~tdOxaesufft 1 [ 1 2 ]p(x) = 2 1/2 exp -~(x-I')(2nD-) 20" (1.130) Do'ivoi m~uvectorx mOtehi~uvoi 2 §'n 8]=I' va 82=0"2ta co d<;lngvectord;:ie tru'ng8 haiehi~u e=[:'] (1.131) Tieu ehu§'nu'oelu'c;1nghc;1ply eve d<;litrong tru'onghc;1Pnay co thS vi€t nhu'sail 1 NIl. J(8)=- L[-ln2n-B2 +-(x' -8])2] N i~12 282 (1.132) Lffy d<;lohameuahamlieu ehu§'nnay theo 8, ta co k€t quau'oelu'c;1nghc;1ply eve d<;liehocaethamso' ~ 1 ~ .8,=f1=- L,.x' N i~1 (1.133) Trang40 eA A2 1~( ; )22 =(5 =- L..x -J1 N ;=1 (1.134 ) Ke'tquanay co th~mdrQngtudngtVd6i voi trudnghQpnhi€u chi€u. Doc luQnghQply qic d(;licuamatr~nhi~pphudngsai L la uoc luQngch~ch.Tilc la giatriky vQngcuai: khongb~ngL, vi d\ld6ivoim~umQtchi€u, giatrikyvQng cua(5 Ia E[a-]=N -1 (52N (1.135) dayla uoc luQngch~ch,khi N ~ 00 thluocluQngtrdthanhuocluQngkhongch~ch. DocluQngkhongch~chcoth~tinhtheocongthilc ~ 1 ~( ; A )( ; A )TL,=-L.. X -J1 X-J1 N -1 ;=1 (1.136) 1.8.2.Phu'ongphap phi thalli 86 Trong cacbai toanthlfcte",d(;lnghamm~tdQxac sua"tkhongduQcbie"ttruoc. Cac phudngphaptangquathdnd~tlmuoc luQngm~tdQxac sua"thoantoandlfa trenduli~udaco,d(;lnghamm~tdQxacsua"tclingnhugiatqb~ngsO'cuacactham s6.Phudngphapm(;lngndronclingduQcdungd~uocluQnghamm~tdQxacsua"t. NhungphudngphapnayduQcgQila phudngphapphi thamsO'(DudaandHart, 1973;Bishop,1975;HolmstrometaI.,1996) Y tu'dngt6ngquatv~phu'ongphapphi thalli 86 Gia sU't~phQpm~uduQcgioi h(;lng6m N phffn tU'x ={Xl,X2,...,xN} dQC l~p toan bQ d6ivoi lOpdachotheohamm~tdQxacsua"tfinp(x).CffnxacdinhmQtUOCluQng flex)cuam~tdQxacsua"tthlfcp(x) Trang41 D\,iatrenykhainit$mcuahamm~tdQxacsua"t,coth~tlmduQcxacsua"tmam~u mdix roi VaGtrongmi~nR duQCchobdi p =f- p(x)diXER (1.137) Coth~vie'tgiatrig~ndungcuaxacsua"t p =f- p(x)di ~ p(x)VXER (1.138) vDiV Ia th~tichcuami~nR (V =J ifi). D\iatrenxa"pxi nayco th~UDClu'Qngp(x)XER d6iVDix dacho,bie'txacsua"tP laxacsua"tmax seroiVaGmi€n R. f)~tinhP, d~utientinhxacsua"tmak ph~ntU'tuN ph~ntU'trongm~useroi VaG mi~nR duQcchobdi lu~tnhi thuc p, - N! pk (1- P) N-k ktuN - k!(N -k)! (1.139) Trungblnhty so'cuacacph~ntU'roiVaGmi~nR laE[kIN]=P vDibie'ndQngquanh trungblnhla E[(kIN - p)2]=P(1-P)/N. VI phanph6i nhi thucPktirN co dC;lnghlnh chopquanhtrungblnhkhi N -+ 00 vi v~ycoth~giasU'dng ty so'kIN la u'DClu'Qng K hK ? ~ K P k totn at cuaxacsuat : p~- N Tll (1.138)va(1.140)taco,p(x)=p=~NV (1.140) (1.141) D~dambaam~tdQxacsua"tmi~ncobantrensehQitl,1v~p(x)th\icdoihoiquye't dinhmi~nR phaidungd~n.S\,il\,iachQnmi~nR duQccoi la t6i uune'uthoacac di~ukit$nmallthu~n.D~utien,d~dambaadingxa"pxi la P ~ ~thl mi~nR c~n phailOn,tu do dam baaxftpxi P =p(x)VIa dunghon ne'uR (dodoV) Ia nhobon, Trang42 khid6rex)h~unhuc6d~nhtn3nmi~ntichphan.Docluejngt6tnhfitdoihoiphaitim mQts6mi~nt6iu'uR. C6 haiky thu~tco band€ uocluejnghamm~tdQxacsufitdlfatn3nslfh,tachQn mi~nvatinhs6ph~ntttcuam~uroivaomi~n,d6la cacphuongphap: . Phuongphapnhanco sd(Kernel-based) . PhuongphapIan c~ng~nnhfit( K-nearestneighbors) Trangphuongphapnhanco sa,mi~nRIa c6 d~nh(va VIv~ymi~nV clingc6d~nh) vas6cacph~ntttcuam~uroi vaotrongmi~nduejcde-mtu t~phejpdli lit%u TrongphuongphapIanc~ng~nnhfit,k ph~ntttcuam~uduejcxacd~nhvadlfatren giatr~naymQtmi~nduejcxacd~nhtudli lit%uvoi th€ tichtuongling. Phuongphap naydaduejcDuda,Hart,1973duafa. Cahaiky thu~tnay,m~tdQxacsufitsehQit\lv~rex)thlfckhiN --+CX) mienla mi~nth€ tichnhol~ikhin tangva k tangtheoN. 1.8.2.1.Phu'dngphapKernel- based.ParzenWindow MQttrongnhlingky thu~tdongiannhfitd€ uocluejngm~tdQxacsufitrex)cho m~umoix dlfatrent~phufinluyt%nTtrachliaN ph~ntttduejcgQila phuongphap Kernel- based. Phuongphapnayc6 d~nhvectorx trenmi~nR va tinhs6ph~ntttcuam~uhufin luyt%nroivaomi~nnaybangcachsttd\lnghamnhandi:icbit%tc6quailht%voimi~n nay.Hamnay duejcgQila hamParzenwindow(Parzen,1962;Duda,andHart, 1973). LvachQnmi~nc6d~nhchom~un chi~ula hinhsieul~pphuongn chi~uc6dQdai qnh bangh t~ptrungKungquanhdi€m x.Th€ tichcuahinhsieul~pphuongnayla V =hn (1.142) Trang 43 C6ngthlic tinhs6 k ph~ntil'cuam§:utu t~phuffnluy~nroi vao tronghlnh sieul~p phuongnhusau: _ { I IYil::::;1/2,i=1,2,...,n ~(y) - 0 N . kh'01 ac (1.143) Hamnhannaytuonglingvdihlnhsieul~pphuongdonvi (codQdaiq.nhb~ng1) macacph~ntU't~ptrungtc;tig6c.Di€u dochophepxacdinhm§:udiichocoroivao tronghlnhsieul~pphuongdonvi kh6ng.Quye'tdinhnayco th~ma.rQngchohlnh sieul~pphuongcodQdaicc;tnhla h. Co th~thffydng d6i vdi ph~ntU'Xi,hamnhan ~((x - Xi)/h)=1 ne'uXi roi vao trong hlnhsieul~pphuongcodQdaiq.nhIaht~ptIlingtc;tidi~mx, giatrihamnhanb~ng 0tc;tinoi khac. Vdi t~phQpm§:ug6mN ph~ntU',t6ngs6 ph~ntU'cuam§:uroi vao tronghlnhsieu l~pphuongt~ptrungxungquanhdi~mx duQcchobai ( i J N x-x k=t;~ ~ (1.144) Thaythe'phuongtrlnh(1.144)vaophuongtrlnh(1.141),taco ~ 1 N 1 [ X_Xi ) 1 N ( X_Xi ]p(x) =Nt;Jlf ~ =N hnt;1f/ ~ (1.145) D€ co udcluQngtrailcuahamm~tdQxacsufft,tadungmQtdc;tngkhaccuaham nhan~(x).Hamnhanthoahaidi€u ki~n ~(y)~0 Ivy~(y)dy =1 (1.146) C6 th€ chQnnhanGaussnhi€u chi€u d6i xlingtam Trang44 IjI(Y) = 1 [ exp_llif ](2nr/2 2 (1.147) i voi Ilyll=~yTY , Y =x ~x chotruoctronguocluQngsau A 1 N l -llx-XiI12 )p(x)=(2nr/2hnNt; exp 2h2 = (1.148) 1 N ((2nr/2hnNLexp -(x-xif(x-Xi) ]1=1 2h2 sO'h<;lngIlx- xiii =((x - Xi)T (x - Xi))1/2 Ia ImmingcachEuclidegiii'ax vaXi Ky hi~usO'ph~nta lOp thli k la Nk va ph~nta thli i tu t<%phua'nluy~n tu lOp CkIa Xk,i khido,lOphamm<%tde)xacsua'tcodi~uki~nla ( k' J 1 Nk 1 x - X ,1 jJ(XICk)= Ni t;h; IjI hk (1.149) voihkIa thamsO'trail lOpphl;lthue)c.£>6ivoi nhanGausstaco A 1 Nk [ -llx-xk,iI12 )p(xICk) =(2nr/2h;Nkt;exp 2hf (1.150) Vi du1.3 Sa dl;lngl<;libe)dii'li~utuvi dl;l1.2,t<%p hQphua'nluy~ndii duQcchiathanhcac lOpkhacnhaunhutrongbang1.2 Tangso"ph~ntacuat<%phua'nluy~nlaN =10,voi N1=5ph~nta tronglOpClvaN2 =5 ph~nta tronglOpC2.Xacsua'ti~nnghi~mchom6ilOpduQcuocluQngla A N, P(CI) =~ =0.5(i=l,2)N Trang45 Ban~ :T~phua'nluy~ndffdu<jchiathanh2 lOpkhacnhau XetmOtm~uvaoX=[8,6]Td6ki6mtra.ChQnthams6trollla h =0.5chocahai lOp,gii trj cuahamnhiin \"(x-t') d6ivdi mlluhua'nluy~n xI.i tir Wpc[ (0) Iii: ( 1,1 JVOixU =[1,2]\ If/ x-hx =1f/(([8,6]T- [1,2]T)/0.5)=5.54E-58 (1.151) ( 1,2 JVoi XI,2=[2,2]T, If/ x -hx =If/ (([8,6]T- [2,2]T)/0.5)=1.084E-46 ( 1,3 JVoi XI,3 =[2,3]T,If/ x-; = If/(([8,6]T- [2,3]T)/0.5)=1.304E-40 ( 1,4 JVoi XI,4=[3,I]T, Ij/ x-; =1j/(([8,6]T- [3,I]T)/0.5)=5.92IE-45 [ 1,5 JVoi XI,5 =[3,2f, If/ x-hx =1f/(([8,6]T- [3,2]T)/0.5)=3.888E-37 Lop 1 Xl X2 Lop 1 2 0 2 2 0 2 3 0 3 1 0 3 2 0 Lop2 Xl X2 Lop 6 8 1 7 8 1 8 7 1 8 8 r 7 9 1 Trang46 Doc luQngcua m~tdQ xac sua'tvoi m~ux =[8, 6]T cua lOp Cl (0) la ~ 1 N, [ -llx-XI,iI12 )p(x [CI) =(2nr/2htN1~exp 2hl2 = (3.48111E-57+6.81356E-46+8.19401E-40+3.72008E-44+2.4426E-36)/7.85= =3.1111E-37 V~y jJ(x[c])=3.1111E-37 Tu'ongtV,tatinhduQcgiatrinhanvam~tdQxacsua'tchom~uthli'x =[8,6]T voi m~uhua'nluy~nX2,i tu lOpC2(1) ( 2,1 JVoi X2,1=[6,8]T,IJI x-: = IJI(([8,6]T- [6,8]T)/0.5)=7.1642E-08 ( 2,2 JVoi X2,2=[7,8]T,IJI x-; = 1JI(([8,6]T- [7,8]T)/0.5)=2.8902E-05 ( 2,3 JVoiX2,3=[8,7]T, IJI x-: = IJI(([8,6]T- [8,7]T)/0.5)=8.6200E-02 [ 2,4 JVoi X2,4=[3,1]T,IJI x-: = 1JI(([8,6]T- [8,8]T)/0.5)=2.1356E-04 [ 2,5 JVoi X2,S=[7,9]T, IJI x-: = IJI(([8,6]T- [7,9]T)/0.5)=1.3122E-09 Doc luQngcua m~tdQ xac sua'tvoi m~ux =[8, 6]T cua lop C2(1) Ia ~ 1 N2 [ -llx-X2,iI12 ) p(x[c2)= (2 r/2hn Iexp 2 =0.0173n 2N2 1=] 2h V~y jJ(xlc])=0.0173 Ta tha'ym~tdQxacsua'tcualOpC2IOnhonm~tdQxacsua'tcualOpCl(0.0173> 3.1111E-37) Trang47 TuongtV,ta tinhduQcm~tdQxac sua'tchom~unh~px =[2, I]T dO'ivoi lOpclla jJ(x ICj)=0.0392va dO'ivoi lOpC2la fl(x IC2)=4.4323E-58. Com~tdQxacsua'tvabiStuocluQngxacsua'ti~nnghil$mP(cJ (i =1,2)tacoth€ tinhduQCbil$thucBayesvasaudophanlOpm~unh~pdljatrenlu~tBayes. Bil$thucdO'ivoim~ux=[8,6]TcualOpclla T A d] (x) =d]([8,6] ) = fl(x I c] )P(c\) =3.1111E-37xO.5=1.5555E-37 (1.152) vacualOpc2la T A d2(x)=d2([8,6]) =fl(x IC2)P(C2)=0.0173xO.5=8.6399E-3 (1.153) VI d2([8,6f) >d]([8,6f) (8.6399E-3 >1.5555E-37)nenrhealu~tto'iu'uBayes,m~u vaox=[8,6]T seduQcgallcholOpC2 Bil$thucdO'ivoim~ux=[2, I]T cualOpclla T A dj (x)=d\ ([2,1] ) = fl(x Ic])P(c\) =0.0392xO.5=0.00196 (1.154) vacualOpC2la T A d2 (x) =d2([2,1] ) =fl(x IC2)P(C2) =4.4323E-58xO.5=2.2161E-58 (1.155) VI dj([2,lf) >d2([2,lf) (0.00196>2.2161E-58)nenrhealu~tto'iu'uBayes,m~u nh~px=[2, I]T seduQcgall cholOpCl 1.8.2.2.Phu'dngphapHinc~ngdnnha't: K - nearestNeighbors MQtphuongphapdongiand€ tiocluQngm~tdQxacsua'tmadQIoncuacac mi~ncoth€ thayd6iduQcdoIa phuongphapK - nearestNeighbors.Trangphuong phapnay,sO'm~uk trongmi~nduQccO'dinhnhungnguQcl(;lidQIOncuami~n(va th€ richV cuami~n)co th€ thayd6iphl,lthuQcvaodii'lil$u.Trangky thu~tuoc h.iQngm~tdQxacsua'tdO'ivoi m~umoichotruocx, dljatrent~phua'nluyl$nTtra g6mN ph~ntti'duQcthljchil$nrheacachsau.D~utien,mQthinhc~un chi~uduQc Trang48 xacd~nhtrongkhonggianm~ut~ptrungt~idi~mx. Saudo,bankinhcuahinhc~u nayduQcmarQngd@nkhihinhc~unayv~nconchuamQtsacad~nhk m~utut~p huffnIuy~ndffchaoDocIuQngm~tdQxacsufftjJ(x) duQctinhIa k jJ(x)=NV (1.156) PhuongphapK - nearestNeighborschotruocmQtUOCIuQngm~tdQric sufftnhung m~tdQxacsufftnaykhongdungVItichphancuahamm~tdQxacsu(tkhonghQitv v~khonggianm~u. Luluphan[upK - nearestNeighbors Gia sU'chotruoct~phuffnIuy~nTIra g6mN ph~ntaXl, X2, ..., XN duQc d~t ten theo 1 lOp va lOpCjchuaNj ph~nta(i =1,2, ...,1;I:Nj =I). Voi x dffcho,k Ianc~ng~n nhffttu t~phuffnIuy~nHmduQcd1;1'av o mQtm~udffxac d~nhdu'QcdQdo khoang cachoSaudo, k Ian c~ng~nnhfftdu'Qc11;1'achQn,sa nithuQCv~lOpCicuam~uduQc tinh.LopduQctiendoanCjduQcgallchox tu'dnglingvoilOpmanjIa IOnnhfft. C6th~lienh~giuaphuongphapk Ianc~ng~nnhfftvoi Iu~tphanlOpGauss,gall chox mQtlOpmaxac suffth~unghi~mco di~uki~nP(cjlx)Ia IOnnhfft.Sa dvng phuongphapk Ianc~ng~nnhffttac6uocIuQng(xffpxl) m~tdQxacsufftc6di~u ki~ncualOptrongmi~nchuak Ianc~ng~nnhfft. nj p(x IcJ =NVI (1.157) vam~tdQxacsufftkhongdi~uki~n nj p(x) =N (1.158) Chungtaclingcoth~HmduQcxffpXlxacsuffti~nnghi~m Trang49 n. ~ I P(c;) =N (1.159) Tli dinhIy Bayestaco: P(Ci Ix) =p(x Ic;)P(c;) ~ ni p(x) ~ k (1.160) * Luljtphanlap[ancljngUnnhfli( NearestNeighbor) TrangIu~tphanlOpk Ianc~ngftnnhtt,s6 Ianc~nk b~ng1.Lu~tphanlOpnay rung duejcgQiIa Iu~tphanlOpIan c~ngftnnhtt, gall rho mQttr,!ngthaimai x mQt lOp cuam~uIan c~ngftnnhtt tli t~phutn Iuy<$nXl, X2,...,XN.Thu~ttoancuaIu~t phanlOpIanc~ngftnnhttduejcxaydl;(ngnhusau: Chotru'oc: MQtt~phutnIuy<$nTtrag6mN phftntii'Xl,x2,...,XNduejcdc1ittentheo caclOpvarhotruacmN m~umaix. I.Tinh Ianc~ngftnnhttXjcuax dii rhotli t~phutnIuY<$ndftydudl;(atrenm~u dii xac dinhdQdokhoangcach(x,Xl) 2. Gan rho x lOpCjIa Ian c~ngftnnhtt rho x Ke'tqua: Lop dii duejctien Joan 1.8.3.Phu'dngphap miathams6 Trangcacphftntruacchungtadii xetcacphuongphapuacIuejngm~tdQxac StittdoIa phuongphapthams6vaphithams6.Phuongphapthams6giasii'diibie't d~ngcuahamm~tdQxacStittcuatoanbQt~pdftli<$uva nhi<$mV\lcuavi<$cuac IuejngIa Hmcacthams6tincuahamdl;(avaot~pdftli<$udii rho.Mc1ith,!nche'cua phuongphapthams6Ia vi<$cchQnhamm~tdQxacStitt cuatoanbQt~phejpdft li<$u(trangkh6nggiandc1ictIling)co khanangkh6ngphilhejpvai mQts6vilngdft li<$udc1icbi<$t.Phuongphapphithams6rhophept6ngquathoacacd,!nghamm~t Trang50 de)xacsua'tnhungh~nchS'cuaphuongphapnayla cacthams6cos6chi~uIOndVa trenco cuadfi'li~u,ngoairaphuongphapnaycondoi hoi sVco m~tcuatoanbe) t~pdfi'li~utrongvi~cuaclu<;1ngm~tde)xacsua'tcuam~umai.B6i vaim~uIOnthl kythu~tnaykhongkhathivabi chiph6ibaithaigian. SVlienkS'tgifi'aphuongphapthams6va phuongphapphi thams6la co sacua phuongphapmYathams6 1.8.3.1.Xa'pxi ham Phuongphapnay gia s\i'xa'pXlm~tde)xac sua'tb~ngquailh~tuyS'ntint cuam hamcosacjJi(X)(Bow, 1992) m jJ(x)=g(x,a)=LaicjJi(X) i=1 (1.161) vai ai(i=l, 2, ...,m) la cacphftnt\i'cuavectorthams6aERffi Me)tvi dl,lv~hamco sa la hamdo'ixungxuyentam cd sd (RadialBasisFunction- RBF)saochome)txa'pXlcuam~tde)xacsua'tcodu<;1cb~nghamtrailvahuanglien tl,lCdu<;1cdintnghlatrongR m g(x,a) = LaicjJi(llx -xiii) =alcjJl(11x-XIII) +a2cjJ2(11x-x211)+...+amcjJm(11x- xmll) i=1 (1.162) vai Ilx-xill la khoangcachgiuahaivectortrongRil.Ta tha'ydng, me)txa'pXlham g(X,a) la me)tquailh~tuyS'ntint cuahamd6i xungxuyenHimco sa cjJi(llx-Xill) (i=l,2,...,m)t~ptrungxungquanhvectorXi. X6tme)thamxuyentamcosatrailva huang(me)tnhan)cjJc(llx-xcll)du<;1cxacdint trongkhoang[0, 1]va t~ptrungxung quanhdi€m xcERn.Hamnayla d6ixungxuyentamcosa. C6nhi~uxa'pXlkhacnhaud€ IVachQnhamcosanhungthongthuangIa s\i'dl,lng hamcosabankinhGauss(nhanGauss) 51 ~c(llx- xcii)=~c(llx- xcll,O"c)=exp [ - (x _XJT (x - Xc) ]20"2c (1.163) voihaithams6: Xcla HimcuahambanldnhGaussva 0";la thams6chu~nhoacua nhanGaussva Ilx - xcll2 =(x- xcf (x- xJ la blnhphuongkhofmgcachEuclidegiua haivectorX va Xc.Tham s6 chu~nhoa 0"la dQl~chchu~ncuaphuongphapchu~n ~ Gaussm~cdlinoduQcuocluQngtheocachkhac,giatrinhanGaussla Immingtit0 toi1.Tilc la Ianc~ncuavectorx latamXccuanhanGaussIOnhongiatricuaham. Ml;lCdichcuavi~cxtp Xlhamm~tdQxacStittla tlmmQtgiatri t6iuucua vector thams6theotieuchu~nt6iuudiixacdinh,vi dl,l lea)= r [p(x)- jJ(x)]2dxJvx (1.164) Thaythe'mQtxtpxi hamg(x,a)chojJ(x) taco: m lea) =Iv [p(x)+Iai~i(x)] x i=1 (1.165) D~tlmcacthams6t6iuu,c~nphiiiqtcti~utieuchu~nJ(a),chodC;lOhamriengtheo ab~ngO. ol(a) =0 oa (1.166) ho~c ol(a) =O,j =1,2, ...,mGa. 1 (1.167) Taco m o~~~)=2Iv)p(x) - ~ai~i(x)]~/x)dx=0I (1.168) m ~ Ivx~j(x)p(x)dx= Ivx~/x)[~ai~i(x)]dx (1.169) Trang52 Theod~nhnghla, Lxq)/x)p(x)dxla mQtgia tr~ky v<;mgE[q)/x)] cuahamq)/x) Sapx€p l~icacphuongtrlnhlIen,t~phQpmphuongtrlnhtuy€n tinhcuacactham s6t6i u'uaI, a2,...,am m 1>; Lxq)/X)q);(x)dx=E[q)/x)], j =1,2,...,m i=1 (1.170) ~ f)~tlmnghi~mcuah~phuongtrlnhlIencgnphaibi€t m~tdQxacsua'tp(x).Co th~ lingdvngxa'pXlsailday E[q)/x)]=Lxq)j(x)p(x)dx~ ~q)j(Xk),j= 1,2,...,m (1.171) Voi N la s6cacphgntucuam~u,taco m 1 m LP; fvxq)/x)q);(x)dx=- Lq)/Xk), j =1,2, ...,m;=1 N k=1 (1.172) T~phQpmphuongtrlnhtuy€n tinhcoth~giaiduQcd6ivoi m hamcobanq)/x). N€u hamcosatn,1'cchuc1nduQcsudvng,thoadiSuki~nsail { li=i r q)(x)q);(x)dx= '. . Jvx J 0, 1::f::.J (1.173) thltaco uocluQngto'iu'ucuacacthamso' aj =~Iq)/Xk),j =1,2,...,m N k=1 (1.174) ~N- ~ [M~N ,j, ( N+1)]a. - 1Va. +'f. X J N+l J J (1.175) voia; va at! la cach~so'to'iu'ud~tduQcchoN vaN +1phgntum~utuongling. Bi€t duQcgiatr~cuathamso'to'iu'uvahamcosatacoxa'pXlhamm~tdQxacsua't m flex) =La;q);(x) ;=1 (1.176) 53 ThuatloanxdJ2..xl hammatdoxacsudt Ch t '~(j~ T" h ,.:' 1 " T ;, N h;, ? 1 2 N b ' ki h h' ?0 ru, c: <;lpuan uy~n ITagom P an tux , x , ...,x . m an n amcoso tnjcchu~ntPi(X),i =1,2,...,m vacacthams6 1.Tinh u'dclu'<;jngcuacacthams6~n Gj =~ftPj(Xk), j =1,2,...,m N k=! (1.177) 2.D;;mgmahinhcuahamm~tdQxacStittlahamxtpXl m jJ(x)=LGitPi(X) i=! (1.178) Ke'tqua:C6m~tdQxacStittcuacacm~u *M6hinhh6nh(Jp Mahinhh6nh<;jpdu'<;jcdl;(atrenquailh~tuye'ntinhcacthams6cuahamm~tdQxac Stittdii bie't(vi dl,lm~tdQxac Stittchu~n)du'<;jcphanvungtrongmQtvungcuadii' li~u(Hinltonetat. , 1995;HastieandTibshirani,1994,1996).MQtmahinhh6nh<;jp coth€ laphanph6ih6nh<;jptuye'ntinhsauday(Duda,Hart,1973) m p(x)=p(x I0) =p(x IB,P) =LPJx IBJ~ i=! (1.179) voiPi (X IBJ la thanhph~nm~tdQthlii voivectorthams6Bp mla s6thanhph~n m~tdQ vaPi la thams6h6nh<;jpthlii. B vaP la kyhi~uthanhph~nm~tdQvector thams6vavectorthams6h6nh<;jptu'ongling.0 lakyhi~uvectorthams6baog6m ()vaP. B€ dongiantrongky hi~u,tasematamahinhh6nh<;jpla m p(x)=LPJx)~ i=! (1.180) Trang54 Phanph6ih6nhQptuye'ntinhd trentu'ongtt;(dinhnghlacuam~tdQxac sua'tkhong di~uki~n.Th~tv~y,taxetPi la xac sua'ti~nnghi~mma m~ux du'Qct6ngquathoa bdithanhph~nm~tdQthil i Pi(X).Hon mIa,taco thS1t;(achQnPi thoa m IF; =1 ;=1 (1.181) O-s,P-s,l I vagiasacacthanhph~nm~tdQdu'Qchu5nhoa LxPi (x)dx =1 (1.182) khidoco thSsosanhvoi lOpm~tdQxacsua'tco di~uki~n.MQtcacht6ngquat: £)~utien,mQtthanhph~nm~tdQthil i du'Qc1t;(achQnng~unhienvoi xacsua'tPi. Saudom~ux du'Qct6ngquathoatheost;(1t;(achQnm~tdQxac sua'tPi (x) H~uhe'tmo hinh h6n hQpm~tdQchu5nGaussdu'Qcsa d\lngcho mo hinhm~tdQ xacsua't.Mo hinhh6nhQpdongvaitrc>quailtrQngtrongvi~cthie'tke'bankinhco sdcuacacm~ngnoronvast;(h6nhQpcuacach~chuyengia. 1.8.3.2.Khoang each giua cae m~t d{)xac sua't.Khoang each Kullback- Leibler M\lc dich cua cac thut\lc u'oc1u'Qngm~tdQ1atim mQtmo hinhm~tdQcang g~nm~tdQ tht;(ccang t6t. Ta co d~ngdinh nghla chinh xac cua khoangcach d(p(x),jJ(x))giua hai m~tdQ: m~tdQxac sua'tht;(cp(x) va u'oc1u'Qngxa'pXl flex) cuano. La'y logaritW nhiencuahamhQp1y L =p(X) =IT:1P(Xi) d6i voi N ph~n . 1 2 N 1,tux,x,...,x a' N -till =- Ilnp(xi) ;=1 (1.183) Trang55 D6ivoimahlnh fl(x),trungblnhcualogaritcuahamh<jply du<jcoinhula giatri kyvQng E[-lnL]=-lim~ flnfl(xi)=- i p(x)lnfl(x)dx N-->ooN i=] Vx (1.184 ) D6ivoim~tdQxacsua'td6ngnha'tp(x)=fl(x),giatqkyvQngla - r p(x)lnp(x)dxJvx (1.185) laentropycuap(x) Thaythe'entropynaytubiguthucky vQngE[-InL] taco dQdokhoangcachgiua p(x) va flex) d(p(x),flex))=- r p(x) In flex)dx Jvx p(x) (1.186) du<jcgQilakhodngeachKullback- Leibler(Kullback,1959)