Nano'nun Günlüğü…

Ideallerimi gerceklestirmek icin arastiriyorum, Unutmamak icin yaziyorum!

  • Bulundugunuz Sayfa: 
  • Ana Sayfa
  • ID3 (Iterative Dichotomiser 3 – Tekrarlanan ikili yapi)

ID3 (Iterative Dichotomiser 3 – Tekrarlanan ikili yapi)

Gönderim Haziran 3rd, 2014

patregOzunde Entropy hesaplamasi kullanan bir algoritmadir. Entropy, bir veri kumesindeki verilerin belirsizliginin sayisallastirilmasi demektir.  Algoritmanin amaci, egitim kumesindeki verilerin agacin olusturulmasi esnasinda birbirine benzetilmesi gerekiyor, agac derinliginin minimum olmasi, karmasikliginda minimum olmasini saglarken kazancin maksimum olmasi gozle gorulur bir fark alacaktir.

ID3 (Iterative Dichotomiser 3 – Tekrarlanan ikili yapi ) :

Ozunde Entropy hesaplamasi kullanan bir algoritmadir. Entropy, bir veri kumesindeki verilerin belirsizliginin sayisallastirilmasi demektir.  Algoritmanin amaci, egitim kumesindeki verilerin agacin olusturulmasi esnasinda birbirine benzetilmesi gerekiyor, agac derinliginin minimum olmasi, karmasikliginda minimum olmasini saglarken kazancin maksimum olmasi gozle gorulur bir fark alacaktir. Entropy deger araligi 0 < entropy < log2n arasinda olmalidir. Entropy degeri log2n’e yaklastikca belirsizligin artmasi, 0’a yaklasmasiyla belirsizligin azalmasi olarak bilinecektir. Ilk hesaplanmasi gereken entropy, tum data setin hesaplanmasi ile olusur. Sonrasinda datasetin farkli nitelikleri icinde entropy hesaplanmaktadir. Tum bu islemleri bilgi edinim icin kullanilmaktadir. Bu kavramdaki kazanc ise, ilk hesaplanan entropy ile her bir alt kumenin olusumundaki entropylerin arasindaki fark hesaplandiktan sonra, farki buyuk olan karar agacinin sagligi acisindan en dogru dallanmayi yapmis olacaktir.

Avantaj : Olasilik kurallari icin egitim verileri kullanilir ve tum data set’teki veriler agac olusturulmasi icin analiz edilir, sonucta kisa agaclar olusturdugu icin en hizli yapiya sahip olmus olur. Bu da, Test sayilarinin azalmasi ve test verilerinin budanmasini saglamaktadir.

Dezavantaj : Dataset’ten aldigimiz egitim verilerimizin boyutu kucukse agac test edildiginde cikan sonucun basarisiz olma olasiligi cok yuksektir.

ID3 UYGULAMASI

1.Adim : Dataset

Week    Weather    Temperatures (Isi)    Wetness     Wind (Ruzgar)    Game
W1    Sunny    Warm    High    Slightly    No
W2    Sunny    Warm    High    Strong    No
W3    Cloudy    Warm    High    Slightly    Yes
W4    Rainy    Warmish    High    Slightly    Yes
W5    Rainy    Cold    Normal    Slightly    Yes
W6    Rainy    Cold    Normal    Strong    No
W7    Cloudy    Cold    Normal    Strong    Yes
W8    Sunny    Warmish    High    Slightly    No
W9    Sunny    Cold    Normal    Slightly    Yes
W10    Rainy    Warmish    Normal    Slightly    Yes
W11    Sunny    Warmish    Normal    Strong    Yes
W12    Cloudy    Warmish    High    Strong    Yes
W13    Cloudy    Warm    Normal    Slightly    Yes
W14    Rainy    Warmish    High    Strong    No

Game = {No, No, Yes, Yes, Yes, No, Yes, No, Yes, Yes, Yes, Yes, Yes, No}
P(Game,No) = 5/14
P(Game,Yes) = 9/14
E(Game) = – ( P (Game,No) log2 P (Game, No) + P (Game,Yes) log2 P (Game,Yes) )
E(Game) = – ( 5/14 log2 5/14 + 9/14 log2 9/14 )
E(Game) = 0.940

1. Adim – Isi niteligi Entropy – Gain Cozumu

[Temperatures,Cold] = 4
[Temperatures,Warmish] = 6
[Temperatures,Warm] = 4
E(Game) = 0.940
E(Temp,Game) = P(Temp,Cold) * E(Temp,Cold) + P(Temp,Warmish) * E(Temp,Warmish) + P(Temp,Warm) * E(Temp,Warm)
E(Temp,Game) = 4/14 E(Temp,Cold) + 6/14 E(Temp,Warmish) + 4/14 E(Temp,Warm)
E(Temp,Cold) = – (1/4 log2 1/4 + 3/4 log2 3/4 )
E(Temp,Cold) = 0.811
E(Temp,Warmish) = – (2/6 log2 2/6+ 4/6 log2 4/6 )
E(Temp,Warmish) = 0.918
E(Temp,Warm) = – (2/4 log2 2/4 + 2/4 log2 2/4 )
E(Temp,Warm) = 1.00
E(Temp,Game) = 4/14 * 0.811 + 6/14 * 0.918 + 4/14 * 1.00
E(Temp,Game) = 0.911
Gain(Temp,Game) = E(Game) – E(Temp,Game)
Gain(Temp,Game) = 0.940 – 0.911
Gain(Temp,Game) = 0.029

1. Adim – Hava niteligi Entropy – Gain Cozumu

[Weather,Sunny] = 5
[Weather,Rainy] = 5
[Weather,Cloudy] = 4
E(Game) = 0.940
E(Weather,Game) = P(Weather,Sunny) * E(Weather,Sunny) + P(Weather,Cloudy) * E(Weather,Cloudy) + P(Weather,Rainy) * E(Weather,Rainy)
E(Weather,Game) = 5/14 E(Weather,Sunny) + 4/14 E(Weather,Cloudy) + 5/14 E(Weather,Rainy)
E(Weather,Sunny) = – (3/5 log2 3/5 + 2/5 log2 2/5 )
E(Weather,Sunny) = 0.971
E(Weather,Rainy) = – (2/5 log2 2/5+ 3/5 log2 3/5)
E(Weather,Rainy) = 0.971
E(Weather,Cloudy) = – (4/4 log2 4/4)
E(Weather,Cloudy) = 0
E(Weather,Game) = 5/14 * 0.971+ 5/14 * 0.971 + 4/14 * 0
E(Weather,Game) = 0.694
Gain(Weather,Game) = E(Game) – E(Weather,Game)
Gain(Weather,Game) = 0.940 – 0.694
Gain(Weather,Game) = 0.247

1. Adim – Nem niteligi Entropy – Gain Cozumu

[Wetness,High] = 7
[Wetness,Normal] = 7
E(Game) = 0.940
E(Wetness,Game) = P(Wetness,High) * E(Wetness,High) + P(Wetness,Normal) * E(Wetness,Normal)
E(Wetness,Game) = 7/14 E(Wetness,High) + 7/14 E(Wetness,Normal)
E(Wetness,Sunny) = – (4/7 log2 4/7 + 3/7 log2 3/7 )
E(Wetness,Sunny) = 0.985
E(Wetness,Rainy) = – (1/7 log2 1/7+ 6/7 log2 6/7)
E(Wetness,Rainy) = 0.592
E(Wetness,Game) = 7/14 * 0.985+ 7/14 * 0.592
E(Wetness,Game) = 0.789
Gain(Wetness,Game) = E(Game) – E(Wetness,Game)
Gain(Wetness,Game) = 0.940 – 0.789
Gain(Wetness,Game) = 0.151

1. Adim – Ruzgar niteligi Entropy – Gain Cozumu

[Wind,Slightly] = 8
[Wind,Strong] = 6
E(Game) = 0.940
E(Wind,Game) = P(Wind,Slightly) * E(Wind,Slightly) + P(Wind,Strong) * E(Wind,Strong)
E(Wind,Game) = 8/14 E(Wind,Slightly) + 6/14 E(Wind,Strong)
E(Wind,Slightly) = – (2/8 log2 2/8 + 6/8 log2 6/8 )
E(Wind,Slightly) = 0.811
E(Wind,Strong) = – (3/6 log2 3/6+ 3/6 log2 3/6)
E(Wind,Strong) = 1.00
E(Wind,Game) = 8/14 * 0.811 + 6/14 * 1.00
E(Wind,Game) = 0.892
Gain(Wind,Game) = E(Game) – E(Wind,Game)
Gain(Wind,Game) = 0.940 – 0.892
Gain(Wind,Game) = 0.0.48

1. Adim – Birinci Dallanma Sonucu

*weather : 0.246
temperature : 0.029
weatness : 0.151
wind : 0.048

2. Adim

Hava ozniteliginin gunesli degeri icin dallanma degerleri
Game = {No, No, No, Yes, Yes}
P(Game,No) = 3/5
P(Game,Yes) = 2/5
E(Game) = – ( P (Game,No) log2 P (Game, No) + P (Game,Yes) log2 P (Game,Yes) )
E(Game) = – ( 3/5 log2 3/5 + 2/5 log2 2/5 )
E(Game) = 0.970

2. Adim – Isi niteligi Entropy – Gain Cozumu

[Temperatures,Cold] = 1
[Temperatures,Warmish] = 2
[Temperatures,Warm] = 2
E(Game) = 0.970
E(Temp,Game) = P(Temp,Cold) * E(Temp,Cold) + P(Temp,Warmish) * E(Temp,Warmish) + P(Temp,Warm) * E(Temp,Warm)
E(Temp,Game) = 1/15 E(Temp,Cold) + 2/5 E(Temp,Warmish) + 2/5 E(Temp,Warm)
E(Temp,Cold) = – (1/1 log2 1/1)
E(Temp,Cold) = 0
E(Temp,Warm) = – (2/2 log2 2/2)
E(Temp,Warm) = 0
E(Temp,Warmish) = – (1/2 log2 1/2 + 1/2 log2 1/2 )
E(Temp,Warmish) = 1.00
E(Temp,Game) = 1/5 * 0 + 2/5 * 0 + 2/5 * 1
E(Temp,Game) = 0.4
Gain(Temp,Game) = E(Game) – E(Temp,Game)
Gain(Temp,Game) = 0.970 – 0.4
Gain(Temp,Game) = 0.570

2. Adim – Nem niteligi Entropy – Gain Cozumu

[Wetness,High] = 3
[Wetness,Normal] = 2
E(Game) = 0.970
E(Wetness,Game) = P(Wetness,High) * E(Wetness,High) + P(Wetness,Normal) * E(Wetness,Normal)
E(Wetness,Game) = 3/5 E(Wetness,High) + 2/5 E(Wetness,Normal)
E(Wetness,Sunny) = – (3/3 log2 3/3)
E(Wetness,Sunny) = 0
E(Wetness,Rainy) = – (2/2 log2 2/2)
E(Wetness,Rainy) = 0
E(Wetness,Game) = 3/5 * 0 + 2/5 * 0
E(Wetness,Game) = 0
Gain(Wetness,Game) = E(Game) – E(Wetness,Game)
Gain(Wetness,Game) = 0.970 – 0
Gain(Wetness,Game) = 0.970

2. Adim – Ruzgar niteligi Entropy – Gain Cozumu

[Wind,Slightly] = 3
[Wind,Strong] = 2
E(Game) = 0.970
E(Wind,Game) = P(Wind,Slightly) * E(Wind,Slightly) + P(Wind,Strong) * E(Wind,Strong)
E(Wind,Game) = 3/5 E(Wind,Slightly) + 2/5 E(Wind,Strong)
E(Wind,Slightly) = – (2/3 log2 2/3 + 1/3 log2 1/3 )
E(Wind,Slightly) = 0.918
E(Wind,Strong) = – (1/2 log2 1/2+ 1/2 log2 1/2)
E(Wind,Strong) = 1.00
E(Wind,Game) = 3/5 * 0.918 + 2/5 * 1.00
E(Wind,Game) = 0.951
Gain(Wind,Game) = E(Game) – E(Wind,Game)
Gain(Wind,Game) = 0.970 – 0.951
Gain(Wind,Game) = 0.019

2. Adim – Ikinci Dallanma Sonucu

temperature : 0.570
*weatness : 0.970
wind : 0.019

3. Adim

Hava ozniteliginin bulutlu degeri icin dallanma degerleri
Game = {Yes, Yes, Yes, Yes}
P(Game,Yes) = 4/4
E(Game) = – ( P (Game, Yes) log2 P (Game, Yes) )
E(Game) = – ( 4/4 log2 4/4)
E(Game) = 1

4. Adim

Hava ozniteliginin yagmurlu degeri icin dallanma degerleri
Game = {Yes, Yes, No, Yes, No}
P(Game,No) = 2/5
P(Game,Yes) = 3/5
E(Game) = – ( P (Game,No) log2 P (Game, No) + P (Game,Yes) log2 P (Game,Yes) )
E(Game) = – ( 2/5 log2 2/5 + 3/5 log2 3/5 )
E(Game) = 0.970
4. Adim – Isi niteligi Entropy – Gain Cozumu

[Temperatures,Cold] = 2
[Temperatures,Warmish] = 3
E(Game) = 0.970
E(Temp,Game) = P(Temp,Cold) * E(Temp,Cold) + P(Temp,Warmish) * E(Temp,Warmish)
E(Temp,Game) = 2/5 E(Temp,Cold) + 3/5 E(Temp,Warmish)
E(Temp,Cold) = – (1/2 log2 1/2 + 1/2 log2 1/2 )
E(Temp,Cold) = 1
E(Temp,Warmish) = – (2/3 log2 2/3 + 1/3 log2 1/3 )
E(Temp,Warmish) = 0.918
E(Temp,Game) = 2/5 * 1 + 3/5 * 0.918
E(Temp,Game) = 0.951
Gain(Temp,Game) = E(Game) – E(Temp,Game)
Gain(Temp,Game) = 0.970 – 0.951
Gain(Temp,Game) = 0.019

4. Adim – Ruzgar niteligi Entropy – Gain Cozumu

[Wind,Slightly] = 3
[Wind,Strong] = 2
E(Game) = 0.970
E(Wind,Game) = P(Wind,Slightly) * E(Wind,Slightly) + P(Wind,Strong) * E(Wind,Strong)
E(Wind,Game) = 3/5 E(Wind,Slightly) + 2/5 E(Wind,Strong)
E(Wind,Slightly) = – (3/3 log2 3/3)
E(Wind,Slightly) = 0
E(Wind,Strong) = – (2/2 log2 2/2)
E(Wind,Strong) = 0
E(Wind,Game) = 3/5 * 0 + 2/5 * 0
E(Wind,Game) = 0
Gain(Wind,Game) = E(Game) – E(Wind,Game)
Gain(Wind,Game) = 0.970 – 0
Gain(Wind,Game) = 0.970

4. Adim – Dorduncu Dallanma Sonucu

temperature : 0.019
*wind : 0.970

Keyifli Calismalar Dilerim.

Etiketler: , , , , , , ,
Bulundugu Konu Etiketleri Akademik, Matlab, Oruntu Tanima/ Pattern Recognition, Yazilim |

Lutfen Yorumlarinizi Burdan Ulastiriniz!...

You must be Kullanici Adiniz : to post a comment.

Istatistik

  • 1 Uye
  • 334 Yazi
  • 16 Yorum Var