Series about decision tree

Dear Readers,

Good morning and Eid Mubarak to all moslem brothers and sisters. I will be writing this post in Indonesia language, along with some English quotation.

I met "Prana Ugi" a keen statistician from Medan (the capital of North Sumatera). He's working a lot with SPSS on various subjects. What can I say, he was graduated from math and statistics program. No wonder huh. Today we'll be covering on the decision tree subjects. R codes will be developed later based on this (or these) post (or posts).

This first draft would still be a QA session between Prana Ugi and I. We will develop the text shortly.

# First post 

The QA was:

Me:

Seandainya ada banyak kolom ttg riwayat penyakit, apakah bisa menghasilkan risiko rendah sedang tinggi. Jadi yg bilang risikonya sebesar apa, adalah data. Seperti itu sepertinya data yg saya kirim tempo hari.

Prana Ugi:

Pak Dasapta Erwin Irawan => dari data excel yang bapak kirim, Ugi bingung pak untuk menentukan, kolom (variabel) mana yang dependen dan independen. Dari data excel bapak dimulai dari kolom A (no), sampai kolom AE (lag1). Boleh diberi keterangan pak, kolom yang berupa dependen dan independen.

Me:

Menurut saya yang independen hanya koordinat. kolom yang lain akan saling dependen kepada kolom yang lain. Skenario saya justru bagaimana mencari yang terkuat.

# Second post

no comment

# Third post

Me:

Proses learningnya apakah menggunakan prinsip regresi?

Prana Ugi:

Pak Dasapta Erwin Irawan => Ada banyak algoritma untuk menginduksi (membuat) pohon keputusan (decision tree), seperti: CART (C&RT), ID3, C4.5, SLIQ, SPRINT, QUEST, DTREG, THAID, CHAID, dan sebagainya. Pada CART, Metode klasifikasi CART terdiri dari dua metode yaitu metode pohon regresi dan pohon klasifikasi. Jika variabel dependen yang dimiliki bertipe kategorik maka CART menghasilkan pohon klasifikasi (classification trees). Sedangkan jika variabel dependen yang dimiliki bertipe kontinu atau numerik maka CART menghasilkan pohon regresi (regression trees). Pada Gambar yang Ugi posting, variabel dependennya bersifat kategori.
Prana Ugi Classification and Regression Trees (CART) adalah salah satu metode atau algoritma dari teknik pohon keputusan. CART adalah suatu metode statistik nonparametrik yang dapat menggambarkan hubungan antara variabel respon (variabel dependen) dengan satu atau lebih variabel prediktor (variabel independen). Menurut Breiman dkk (1993), apabila variabel respon berbentuk kontinu maka metode yang digunakan adalah metode regresi pohon (regression trees), sedangkan apabila variabel respon memiliki skala kategorik maka metode yang digunakan adalah metode klasifikasi pohon (classification trees). Semoga bermanfaat pak, diskusinya.

# Forth post

Data Mining, Concepts, Models, and Techniques
By: Florin Gorunescu, Springer

no comment

# Fifth post

Prana Ugi:

Alhamdulillah, proses perhitungan manual Classification and Regression Tree (CART) dengan algoritma ID3 (iterative dichotomiser 3) dan C4.5 dengan kriteria entropy (impurity) selesai. Algoritma diimplementasikan ke Excel berhasil, sekarang pindah ke R, semoga ada package-nya, kalau nggak ada, buat sendiri. 

Me:

coba R package berikut:
- CARET (https://cran.r-project.org/web/packages/caret/index.html),
- RPART (https://cran.r-project.org/web/packages/rpart/index.html), dan
- TREE (https://cran.r-project.org/web/packages/tree/index.html)```

We'll keep updating this post.

randomForest package exercise

Dear friends,

You don't need a decision tree to choose which one to eat first right.

I was working on the tree and randomForest code when I found the following link. This page was written by Wes Stevenson and also Quick-R. In this post, he explains some key points of using the packages: 

+ rpart

+ tree

+ randomForest

+ party

+ maptree

+ evtree

+ and several other classifier packages

Just finished working on the randomForest package and posted an ioslide about the  on Rpubs. The R code is posted at Evernote. It was still in show and tell level. But I will update it with some explanations and maybe additional try out with the other listed packages.

Please drop your comments. Thank you.

@dasaptaerwin 

#rstats   #multivariatestats  

Linear regression: Excel and R

(image from: en.wikipedia.org)

Dear friends,

Some of us think that point and click softwares are more efficient than their command line competitors. Here's a comparison.

Suppose you are going to make a linear regression between two variables, x and y, then make a plot out of it with regression line in the plot. 

In Ms Excel, you would do something like this:

1. Open the data:

• Menu > 
• Open > 
• select the data

2. Making the plot: 

• block the columns > 
• insert chart > 
• selecting chart type 

3. Making the regression line: 

• selecting the points > 
• insert > 
• trend line > 
• select linear regression > show equation

In R, you type the following lines:

1. Open the data:
   • data <- read.csv="" yourdata.csv="">
2. Making the model:
   • linear <- data="" li="" lm="" x="" y="">
   • summary(linear)
3. Making the plot:
• plot(data$y ~ data$x)

The point and click technique might be more efficient, but it is rather hard to be reproduced without having to screen capturing every step.

A bit about R Markdown

---
title: "About R Markdown Try Out"
author: "Dasapta Erwin Irawan"
date: "Friday, September 12, 2014"
output: 
  word_document:
   fig_width: 5
   fig_height: 5
   fig_caption: true
---

Dear all,

I've finally experienced an R history, especially in the evolution of R Markdown. I first knew R in 2011. It was when I attended a short course about Top Model package, delivered by Wouter Bouytaert of Imperial College London (pardon for mis-spelling). 

This post was written in R Markdown. For those of you that havent't used it, just go to File -> New -> R Markdown. The R Markdown (Rmd) was one of the flavour of the original Markdown markup that was made mainly for HTML. You can write web page easier with MD then convert it to HTML before then upload it to your web page.

If we use the older R studio, the conversion process is based on command lines using `knitr` package following this flow:

from Rmd -> rendered to md -> rendered to html (the basic final format). 

Knitr is an R package written by Yihui Xie. With this package you can convert an Rmd file to HTML, directly with in R Studio. The newer `knitr` package had included the `pandoc` ability. So you can convert Rmd to pdf, LaTex, odt, or even Word. 

Then what's Pandoc? 
It is a command line based software, written by John MacFarlane. Previously we had to separately installed `pandoc`  outside R std. Then convert Rmd to many formats including Word, via terminal or Windows prompt.to pdf or word. Pandoc project is still running now. With this software you can amazingly turn your txt file in to a full size book without any word processor software. 

Back to Rmd, the following sections talk about the components and basic features of Rmd. It basically consists of YAML header and body text.



# 1. YAML Header

The YAML (Yet Another Markup Language) header is placed on top of the Rmd file, and it must be started and closed with three dashes `---`. I tried with two or five dashes, it didn't work.

If we use the newest version of R Studio (Version 0.98.1049), Rstd will automatically make the the YAML header, based on our choices in new file window.

We can insert in YAML header our figure size using this command: 

+ `fig_width:`, 
+ `fig_height:`, and 
+ `fig_caption:` 



# 2. Embedding R code

The default `echo` will be `TRUE`. If we use option `{r, echo = FALSE}` then Rstd will still run the command but not showing the actual code in the final doc / html / pdf document.

We can use 

+ three back ticks without ```{r}``` to make a dummy code. R std will show the code but not running it.
+ or use ```{r, eval=FALSE}```

We can also name the chunk by typing ```{r chunkname}```, with no spaces. So then we can use it to make an in-text reference.

Here's an example.

```{r chunk1}
A <- 2="" font="">
B <- 2="" font="">
C <- a="" b="" font="">
```

We can also cache a chunk so it won't be re-run by R Std. If the result is figure, then the figure is named based on the chunk name.

Example: Here's an equation 

```{r chunk2, cache=TRUE}
A <- 2="" font="">
B <- 2="" font="">
C <- a="" b="" font="">
```

# 3. Embedding image

We can automatically embed R plots as you show me. We also can embed offline images by using this command.

```
```

Here's an example

 

from [Wikipedia](http://en.wikipedia.org/wiki/Citroën_DS)


# 4. Bold, italics and bullets

We can use under score symbol to print bold and italic font:

+ __Bold__
+ _italic_ 

We can use symbols like `-` or `+`. Add two `tabs` for nested bullets.

+ Bullet 1
+ Bullet 2
    + Bullet 2.1
    + Bullet 2.2
+ Bullet 3

Or combined with number:

1. List 1
2. List 2
    + Bullet 2.1
    + Bullet 2.2
3. List 3

How to make QQ plot in R

Dear friends,

For those of you that are still having some problems making chart in R, here’s an example for QQ plot. I made it steps.

1. Read data

Use your own data.

QQTest <- read.table(file="QQTest.dat",sep=",",head=TRUE)

2. Make a QQplot

qqnorm(QQTest$vals)

3. Annotate the plot

qqnorm(QQTest$vals,
       main="Normal Q-Q Plot of ...",
       xlab="Observed",
       ylab="Modeled")

4. Make the same x and y limit

qqnorm(QQTest$vals,
       main = "Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3)) 

5. Change to + symbol

qqnorm(QQTest$vals,
       main = "Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 

Add the QQ line

qqline(QQTest$vals, col = "blue", lwd = 2, lty = 2)

Or make it red

qqline(QQTest$vals, col = "red", lwd = 2, lty = 2)

Or make it not dotted

qqline(QQTest$vals, col = "blue", lwd = 2, lty = 1)

Or make it thinner

qqline(QQTest$vals, col = "blue", lwd = 1, lty = 1)

6. Add the grid

qqnorm(QQTest$vals,
       main = "Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+",
       panel.first = grid(7, lty = 1, lwd = 2)) 

Not good right?

Try this one

qqnorm(QQTest$vals,
       main = "Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 
abline(v=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")
abline(h=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")

7. Make two columns plot: side by side

First delete your old plot using plot.new() then change the layout

plot.new()
par(mfrow=c(2, 1))

Make the left plot

qqnorm(QQTest$vals,
       main = "Upper Plot -> Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 
abline(v=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")
abline(h=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")

Then make the right plot

qqnorm(QQTest$vals,
       main = "Right Plot -> Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 
abline(v=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")
abline(h=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")

Or make it two rows: upper and lower

plot.new()
par(mfrow=c(1, 2))

Make the upper plot

qqnorm(QQTest$vals,
       main = "Upper Plot -> Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 
abline(v=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")
abline(h=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")

Then make the lower plot

qqnorm(QQTest$vals,
       main = "Lower Plot -> Normal Q-Q Plot of ...",
       xlab = "Observed",
       ylab = "Modeled",
       xlim = c(-3, 3),
       ylim = c(-3, 3),
       pch = "+") 
abline(v=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")
abline(h=(seq(-3, 3, by=1)), col="lightgray", lty="dotted")

There you go. You make more tweaks it based on your needs. You can also use a specific package with specific style like: lattice or ggplot2. Make sure you read the documentation first, as each package may have different commands, but the basic principle is the same.

Good luck.

@dasaptaerwin