Metod najmenshih kvadrativ metod znahodzhennya nablizhenogo rozv yazku nadlishkovo viznachenoyi sistemi Chasto zastosovuyetsya v regresijnomu analizi Na praktici najchastishe vikoristovuyetsya linijnij metod najmenshih kvadrativ sho vikoristovuyetsya u vipadku sistemi linijnih rivnyan Zokrema vazhlivim zastosuvannyam u comu vipadku ye ocinka parametriv u linijnij regresiyi sho shiroko zastosovuyetsya u matematichnij statistici i ekonometrici Zmist 1 Motivacijnij priklad 1 1 Vikoristannya kvadratichnoyi modeli 2 Linijnij vipadok 2 1 Odna nezalezhna zminna 2 2 Mnozhinna regresiya vipadok bagatoh nezalezhnih zminnih 2 3 Vivedennya formuli 2 4 Chislovi metodi dlya obchislennya rozv yazku 2 5 Statistichni vlastivosti 3 V matematichnomu modelyuvanni 4 Primitki 5 Div takozh 6 DzherelaMotivacijnij priklad Redaguvati nbsp Grafik tochok danih chervonim liniya najmenshih kvadrativ sinim i vidstani zelenim Nehaj v rezultati deyakogo doslidu otrimano chotiri x y displaystyle x y nbsp tochki danih 1 6 displaystyle 1 6 nbsp 2 5 displaystyle 2 5 nbsp 3 7 displaystyle 3 7 nbsp i 4 10 displaystyle 4 10 nbsp na malyunku livoruch poznacheni chervonim Potribno znajti pryamu y b 1 b 2 x displaystyle y beta 1 beta 2 x nbsp yaka najkrashe pidhodit dlya cih tochok Inakshe kazhuchi mi hotili b znajti chisla b 1 displaystyle beta 1 nbsp i b 2 displaystyle beta 2 nbsp yaki priblizno rozv yazuyut nadviznachenu linijnu sistemu b 1 1 b 2 6 b 1 2 b 2 5 b 1 3 b 2 7 b 1 4 b 2 10 displaystyle begin alignedat 3 beta 1 1 beta 2 amp amp amp amp 6 amp beta 1 2 beta 2 amp amp amp amp 5 amp beta 1 3 beta 2 amp amp amp amp 7 amp beta 1 4 beta 2 amp amp amp amp 10 amp end alignedat nbsp chotiroh rivnyan z dvoma nevidomimi v deyakomu najkrashomu sensi Pidhid najmenshih kvadrativ rozv yazannya ciyeyi problemi polyagaye u sprobi zrobiti yakomoga menshoyu sumu kvadrativ pohibok mizh pravoyu i livoyu storonami ciyeyi sistemi tobto neobhidno znajti minimum funkciyi S b 1 b 2 6 b 1 1 b 2 2 5 b 1 2 b 2 2 7 b 1 3 b 2 2 10 b 1 4 b 2 2 displaystyle begin aligned S beta 1 beta 2 amp left 6 beta 1 1 beta 2 right 2 left 5 beta 1 2 beta 2 right 2 amp left 7 beta 1 3 beta 2 right 2 left 10 beta 1 4 beta 2 right 2 end aligned nbsp Minimum viznachayut cherez obchislennya chastkovoyi pohidnoyi vid S b 1 b 2 displaystyle S beta 1 beta 2 nbsp shodo b 1 displaystyle beta 1 nbsp i b 2 displaystyle beta 2 nbsp i pririvnyuvannyam yih do nulya S b 1 0 8 b 1 20 b 2 56 displaystyle frac partial S partial beta 1 0 8 beta 1 20 beta 2 56 nbsp S b 2 0 20 b 1 60 b 2 154 displaystyle frac partial S partial beta 2 0 20 beta 1 60 beta 2 154 nbsp Ce privodit nas do sistemi z dvoh rivnyan i dvoh nevidomih yaki nazivayutsya normalnimi rivnyannyami Roz yazkom SLAR budut b 1 3 5 displaystyle beta 1 3 5 nbsp b 2 1 4 displaystyle beta 2 1 4 nbsp zvidki otrimuyemo y 3 5 1 4 x displaystyle y 3 5 1 4x nbsp sho ye rivnyannyam pryamoyi yaka prohodit najblizhche do podanih chotiroh tochok Minimalna suma kvadrativ pohibok ye S 3 5 1 4 1 1 2 1 3 2 0 7 2 0 9 2 4 2 displaystyle S 3 5 1 4 1 1 2 1 3 2 0 7 2 0 9 2 4 2 nbsp nbsp Rezultat pidgonki sukupnosti sposterezhen x i y i displaystyle x i y i nbsp chervonim kvadratichnoyu funkciyeyu y b 1 b 2 x b 3 x 2 displaystyle y beta 1 beta 2 x beta 3 x 2 nbsp sinim U linijnih najmenshih kvadratah funkciya ne povinna buti linijnoyu u svoyemu argumenti x displaystyle x nbsp a lishe shodo svoyih parametriv b j displaystyle beta j nbsp yaki treba viznachiti dlya otrimannya najkrashogo rezultatuVikoristannya kvadratichnoyi modeli Redaguvati Vazhlivo sho u metodi linijnih najmenshih kvadrativ mi ne obmezheni vikoristannyam pryamoyi yak modeli yak u poperednomu prikladi Napriklad mi mogli vibrati obmezhenu kvadratichnu model y b 1 x 2 displaystyle y beta 1 x 2 nbsp 1 Cya model vse she linijna v sensi parametru b 1 displaystyle beta 1 nbsp otzhe mi vse she mozhemo zdijsnyuvati toj samij analiz buduyuchi sistemu rivnyan z tochok danih 6 b 1 1 2 5 b 1 2 2 7 b 1 3 2 10 b 1 4 2 displaystyle begin alignedat 2 6 amp amp beta 1 1 2 5 amp amp beta 1 2 2 7 amp amp beta 1 3 2 10 amp amp beta 1 4 2 end alignedat nbsp Chastkovi pohidni shodo parametriv cogo razu lishe odnogo tak samo obchislyuyutsya i pririvnyuyutsya do 0 S b 1 0 708 b 1 498 displaystyle frac partial S partial beta 1 0 708 beta 1 498 nbsp Rozv yazok otrimanogo rivnyannya b 1 0 703 displaystyle beta 1 0 703 nbsp sho prizvodit do viznachennya najbilsh pidhodyashoyi modeli y 0 703 x 2 displaystyle y 0 703x 2 nbsp Linijnij vipadok RedaguvatiOdna nezalezhna zminna Redaguvati Nehaj mayemo linijnu regresiyu zi skalyarnoyu zminnoyu x y x b 1 b 0 displaystyle y x beta 1 beta 0 nbsp a takozh vibirku pochatkovih danih y i x i displaystyle y i x i nbsp rozmiru M Todi b 0 1 M i y i b 1 M i x i b 1 M i x i y i i x i i y i M i x i 2 i x i 2 displaystyle beta 0 frac 1 M sum i y i frac beta 1 M sum i x i beta 1 frac M sum i x i y i sum i x i sum i y i M sum i x i 2 sum i x i 2 nbsp Mnozhinna regresiya vipadok bagatoh nezalezhnih zminnih Redaguvati Dlya nadlishkovo viznachenoyi sistemi m linijnih rivnyan z n nevidomimi b j m gt n displaystyle beta j quad m gt n nbsp j 1 n X i j b j y i i 1 m j 1 n displaystyle sum j 1 n X ij beta j y i quad i overline 1 m quad j overline 1 n nbsp chi v matrichnij formi zapisu X b y displaystyle X boldsymbol beta mathbf y nbsp zazvichaj ne isnuye tochnogo rozv yazku i potribno znajti taki b yaki minimizuyut nastupnu normu a r g m i n b i 1 m y i j 1 n X i j b j 2 a r g m i n b y X b 2 displaystyle underset boldsymbol beta operatorname arg min sum i 1 m left y i sum j 1 n X ij beta j right 2 underset boldsymbol beta operatorname arg min big mathbf y X boldsymbol beta big 2 nbsp Takij rozv yazok zavzhdi isnuye i vin ye yedinim b X X 1 X y displaystyle hat boldsymbol beta X top X 1 X top mathbf y nbsp hoch dana formula ne ye efektivnoyu cherez neobhidnist znahoditi obernenu matricyu Vivedennya formuli Redaguvati Znachennya S i 1 m y i j 1 n X i j b j 2 displaystyle S sum i 1 m left y i sum j 1 n X ij beta j right 2 nbsp dosyagaye minimumu v tochci v yakij pohidna po kozhnomu parametru rivna nulyu Obchislyuyuchi ci pohidni oderzhimo S b j 2 i r i r i b j 0 j 1 2 n displaystyle frac partial S partial beta j 2 sum i r i frac partial r i partial beta j 0 j 1 2 dots n nbsp de vikoristano poznachennya r i y i j 1 n X i j b j displaystyle r i y i sum j 1 n X ij beta j nbsp Takozh vikonuyutsya rivnosti r i b j X i j displaystyle frac partial r i partial beta j X ij nbsp Pidstavlyayuchi virazi dlya zalishkiv i yih pohidnih oderzhimo rivnist S b j 2 i 1 m X i j y i k 1 n X i k b k 0 displaystyle frac partial S partial beta j 2 sum i 1 m X ij left y i sum k 1 n X ik beta k right 0 nbsp Danu rivnist mozhna zvesti do viglyadu i 1 m k 1 n X i j X i k b k i 1 m X i j y i j 1 2 n displaystyle sum i 1 m sum k 1 n X ij X ik hat beta k sum i 1 m X ij y i j 1 2 dots n nbsp abo v matrichnij formi X X b X y displaystyle mathbf X top mathbf X hat boldsymbol beta mathbf X top mathbf y nbsp Chislovi metodi dlya obchislennya rozv yazku Redaguvati Yaksho matricya X X displaystyle X top X nbsp ye nevirodzhenoyu ta dodatnooznachenoyu tobto maye povnij rang todi sistema mozhe buti rozv yazana za dopomogoyu rozkladu Holeckogo X X R R displaystyle X top X R top R nbsp de R displaystyle R nbsp verhnya trikutna matricya R R b X y displaystyle R top R hat boldsymbol beta X top mathbf y nbsp Rozv yazok otrimayemo v dva kroki Otrimayemo z displaystyle mathbf z nbsp z rivnyannya R z X y displaystyle R top mathbf z X top mathbf y nbsp Pidstavimo i otrimayemo b displaystyle hat boldsymbol beta nbsp z R b z displaystyle R hat boldsymbol beta mathbf z nbsp V oboh vipadkah vikoristovuyutsya vlastivosti trikutnoyi matrici Statistichni vlastivosti Redaguvati Odnim iz najvazhlivishih zastosuvan linijnogo MNK ye ocinka parametriv linijnoyi regresiyi Dlya zadanogo naboru danih y i x i 1 x i p i 1 n displaystyle y i x i1 ldots x ip i 1 n nbsp buduyetsya model y i b 0 b 1 x i 1 b p x i p e i x i b e i i 1 n displaystyle y i beta 0 beta 1 x i1 cdots beta p x ip varepsilon i x i beta varepsilon i qquad i 1 ldots n nbsp abo v matrichnij formi y X b e displaystyle y X beta varepsilon nbsp de y y 1 y 2 y n X x 1 x 2 x n x 11 x 1 p x 21 x 2 p x n 1 x n p b b 1 b p e e 1 e 2 e n displaystyle y begin pmatrix y 1 y 2 vdots y n end pmatrix quad X begin pmatrix x 1 x 2 vdots x n end pmatrix begin pmatrix x 11 amp cdots amp x 1p x 21 amp cdots amp x 2p vdots amp ddots amp vdots x n1 amp cdots amp x np end pmatrix quad beta begin pmatrix beta 1 vdots beta p end pmatrix quad varepsilon begin pmatrix varepsilon 1 varepsilon 2 vdots varepsilon n end pmatrix nbsp V cih formulah b displaystyle beta nbsp vektor parametriv yaki ocinyuyutsya napriklad za dopomogoyu metodu najmenshih kvadrativ a e displaystyle varepsilon nbsp vektor vipadkovih zminnih U klasichnij modeli mnozhinnoyi linijnoyi regresiyi prijmayutsya taki umovi y i b 0 b 1 x i 1 b p x i p e i x i b e i i 1 n displaystyle y i beta 0 beta 1 x i1 cdots beta p x ip varepsilon i x i beta varepsilon i qquad i 1 ldots n nbsp E e i 0 displaystyle operatorname E varepsilon i 0 nbsp E e i e j s 2 i j 0 i j displaystyle operatorname E varepsilon i varepsilon j begin cases sigma 2 amp i j 0 amp i neq j end cases nbsp tobto vipadkovi zminni ye gomoskedastichnimi i mizh nimi vidsutnya bud yaka zalezhnist Rang matrici X rivnij p 1 tobto mizh poyasnyuyuchimi zminnimi vidsutnya linijna zalezhnist Dlya takoyi modeli ocinka b displaystyle hat boldsymbol beta nbsp oderzhana metodom najmenshih kvadrativ volodiye vlastivostyami Nezmishenist Ocinka b displaystyle hat boldsymbol beta nbsp ye nezmishenoyu tobto E b X b displaystyle operatorname E hat beta X beta nbsp Spravdi E b E X X 1 X X b e b E X X 1 X e b X X 1 X e E e b displaystyle operatorname E hat beta operatorname E Big X X 1 X X beta varepsilon Big beta operatorname E Big X X 1 X varepsilon Big beta Big X X 1 X varepsilon Big operatorname E varepsilon beta nbsp Kovariacijna matricya ocinki b displaystyle hat boldsymbol beta nbsp rivna Var b s 2 X X 1 displaystyle operatorname Var hat beta sigma 2 X X 1 nbsp Ce viplivaye z togo sho Var Y Var e displaystyle operatorname Var Y operatorname Var varepsilon nbsp i E b Var X X 1 X Y X X 1 X Var Y X X X 1 displaystyle operatorname E hat beta operatorname Var X top X 1 X top Y X top X 1 X top operatorname Var Y X X top X 1 nbsp s 2 X X 1 X X 1 X X s 2 X X 1 displaystyle sigma 2 X X 1 X top X 1 X top X sigma 2 X X 1 nbsp dd dd Efektivnist Zgidno z teoremoyu Gausa Markova ocinka sho oderzhana MNK ye najkrashoyu linijnoyu nezmishenoyu ocinkoyu Zmistovnist Pri dovoli slabkih obmezhennyah na matricyu X metod najmenshih kvadrativ ye zmistovnim tobto pri zbilshenni rozmiru vibirki ocinka za imovirnistyu pryamuye do tochnogo znachennya parametru Odniyeyu z dostatnih umov ye napriklad pryamuvannya najmenshogo vlasnogo znachennya matrici X X displaystyle X top X nbsp do bezmezhnosti pri zbilshenni rozmiru vibirki Yaksho dodatkovo pripustiti normalnist zminnih e displaystyle varepsilon nbsp to ocinka MNK maye rozpodil b N b s 2 X X 1 displaystyle hat beta sim mathcal N big beta sigma 2 X X 1 big nbsp V matematichnomu modelyuvanni RedaguvatiNehaj mi mayemo vibirku pochatkovih danih f x i y i i 1 n displaystyle f x i y i i overline 1 n nbsp Funkciya f displaystyle f nbsp nevidoma Yaksho mi znayemo pribliznij viglyad funkciyi f x displaystyle f x nbsp to zadamo yiyi u viglyadi funkcionalu F x i a 0 a m y i displaystyle F x i a 0 ldots a m approx y i nbsp de a 0 a m displaystyle a 0 ldots a m nbsp nevidomi konstanti Nam potribno minimizuvati vidminnosti mizh F displaystyle F nbsp ta f displaystyle f nbsp Dlya cogo berut za miru sumu kvadrativ riznic znachen cih funkcij u vsih tochkah x i displaystyle x i nbsp i yiyi minimizuyut tomu metod tak i nazivayetsya I a 0 a m i 0 n y i F x i a 0 a m 2 min displaystyle I a 0 ldots a m sum i 0 n y i F x i a 0 ldots a m 2 to min nbsp Koeficiyenti a j displaystyle a j nbsp v yakih taka mira minimalna znahodyat z sistemi I a 0 a m a 0 0 I a 0 a m a m 0 displaystyle begin cases displaystyle frac partial I a 0 ldots a m partial a 0 0 ldots displaystyle frac partial I a 0 ldots a m partial a m 0 end cases nbsp Primitki Redaguvati Povne kvadratne rivnyannya u zagalnomu vipadku maye tri nenulovi koeficiyenti i maye viglyad y b 1 x 2 b 2 x b 3 displaystyle y beta 1 x 2 beta 2 x beta 3 nbsp Div takozh RedaguvatiVidstan Kuka Test Brojsha Pagana Metod instrumentalnih zminnihDzherela RedaguvatiKartashov M V Imovirnist procesi statistika Kiyiv VPC Kiyivskij universitet 2007 504 s Gnedenko B V Kurs teorii veroyatnostej 6 e izd Moskva Nauka 1988 446 s ros Gihman I I Skorohod A V Yadrenko M V Teoriya veroyatnostej i matematicheskaya statistika Kiyiv Visha shkola 1988 436 s ros Metod najmenshih kvadrativ Visha matematika v prikladah i zadachah Klepko V Yu Golec V L 2 ge vidannya K Centr uchbovoyi literaturi 2009 S 358 594 s Louson Ch Henson R Chislennoe reshenie zadach metodom naimenshih kvadratov M Nauka 1986 Prikladnaya statistika Osnovy ekonometriki Uchebnik dlya vuzov V 2 t 2 e izd ispr T 2 Ajvazyan S A Osnovy ekonometriki M YuNITI DANA 2001 432 s ISBN 5 238 00305 6 Bjorck Ake 1996 Numerical methods for least squares problems Philadelphia SIAM ISBN 0 89871 360 9 Greene William H 2002 Econometric analysis 5th ed New Jersey Prentice HallV inshomu movnomu rozdili ye povnisha stattya Least squares angl Vi mozhete dopomogti rozshirivshi potochnu stattyu za dopomogoyu perekladu z anglijskoyi Divitis avtoperekladenu versiyu statti z movi anglijska Perekladach povinen rozumiti sho vidpovidalnist za kincevij vmist statti u Vikipediyi nese same avtor redaguvan Onlajn pereklad nadayetsya lishe yak korisnij instrument pereglyadu vmistu zrozumiloyu movoyu Ne vikoristovujte nevichitanij i nevidkorigovanij mashinnij pereklad u stattyah ukrayinskoyi Vikipediyi Mashinnij pereklad Google ye korisnoyu vidpravnoyu tochkoyu dlya perekladu ale perekladacham neobhidno vipravlyati pomilki ta pidtverdzhuvati tochnist perekladu a ne prosto skopiyuvati mashinnij pereklad do ukrayinskoyi Vikipediyi Ne perekladajte tekst yakij vidayetsya nedostovirnim abo neyakisnim Yaksho mozhlivo perevirte tekst za posilannyami podanimi v inshomovnij statti Dokladni rekomendaciyi div Vikipediya Pereklad nbsp Ce nezavershena stattya zi statistiki Vi mozhete dopomogti proyektu vipravivshi abo dopisavshi yiyi Otrimano z https uk wikipedia org w index php title Metod najmenshih kvadrativ amp oldid 36886014
