Data Management : Lists in R Language

Vectors, matrices, and arrays is that each of these types of objects may only contain one type of data.

For example, a vector may contain all numeric data or all character data.

A list is a special type of object that can contain data of multiple types.

Lists are characterized by the fact that their element do not need to be of the same object type.

Lists can contain elements of different types so that the list elements may have different modes.

Lists can even contain other structured objects, such as lists and data frames which allows to create recursive data structures.

Lists can be indexed by position.
  So x [ [5] ] refers to the fifth element of x.

Lists can extract sublists.
 - So x [c (2,5) ] is a sublists of x that consists of the second and fifth elements.

Lists elements can have names.
 - Both x [ ["Students"] ] and x$Students refer to the element named "Students".

Difference between a vector and a list :

• In a vector, all element must have the same mode.
• In a list, the elements can have different modes.

Modes :

Every objects has a mode.

The mode indicates how the object is stored in memory: as a 

1. number
2. character string 
3. list of pointers to other objects,
4. function etc.

Mode function give us such information.

Syntax

mode ( )

Example

> mode (1.234)
 [1]  "numeric"

> mode ( c(5,6,7,8) )
  [1]  "numeric"

> mode ("India")
  [1]  "character"

> mode ( c( "India" , "USA") )
  [1]  "character"

  
Example

> mode (factor (c ("UP" , "MP") )   )
  [1]  "numeric "

> mode (list ("India", "USA") )
 [1]  "list"

>mode (data.frame (x=1:2, Y=c ("India", "USA" ) ) )
  [1]   "list"

> mode (print)
  [1]  "function"

Read More

Loops in R Language

The concept of executing a block of code multiples time.

Loops in R , are of following types

• For Loop : Executes statements multiple times, checks condition at the end.
• Repeat Loop : Repeats the code multiple times.
• While Loop : Executes code till a condition is satisfied.

For Loop :-
                     This loop iterates over a collection of values.

Syntax :-
                  for any_variable in collection_of_values
                     {
                            // Code to be executed
                      } 

Examples :-

While Loop :-
             Executes a set of statements till a condition is true,

Format :-
             While (Condition)
              {
                    // Code to be executed
               }

Examples :-

Repeat Loop :-
          A repeat loop executes a set of statements till a terminate statement is found.

The syntax of a repeat loop is :
                repeat
                         {
                              #Commands
                              if(condition)
                               {
                                   break
                                }
                          }

Examples :-

Read More

Analysis of Variance (ANOVA) and F-test in R Language

Analysis of Variance (ANOVA)

A statistical method for making simultaneous comparisons between two or more means; a statistical method that yields values that can be tested to determine whether a significant relation exists between variables.

Examples :-

A car company wishes to compare the average petrol consumption of THREE similar models of car and has available six vehicles of each model.

A teacher is interested in a comparison of average percentage marks attained in the examinations of FIVE different subjects and has available the marks of eight students who all completed each examination.

1. What ANOVA looks at is the way groups differ internally versus what the difference is between them. To take the above example:
2. ANOVA calculates the mean for each group.
3. It calculates the mean for all the group combined - the Overall Mean.
4. Then it calculates, within each group, the total deviation of each individual's score from the Group Mean - Within Group Variation.
5. Next, it calculates the deviation of each Group Mean from the Overall Mean - Between Group Variation.
6. Finally, ANOVA produces the F statistic which is the ratio - Between Group Variation to the Within Group Variation.
7. If the Between Group Variation is significantly greater than the Within Group Variation, then it is likely that there is a statistically significant difference between the groups.

One way Analysis of Variance

setwd("C:\\PERSONAL\\Irawen_Business_Analytics_With_R\\WIP\\Class-8 data-1")
data.ex1<-read.table("Class-8 data.txt",sep=',',head=T)
aov.ex1 = aov(Alertness~Dosage,data=data.ex1)
summary(aov.ex1) 

Two way Analysis of Variance

Data are from an experiment in which alertness level of male and female subjects was measured after they had been given one of two possible dosages of a drug. Thus, this is a 2x2 design with the factors being Gender and Dosage 



Used of Analysis of Variance

→ ANOVA is a particular form of statistical hypothesis testing heavily used in the analysis of experimental data.

→ A statistical hypothesis test is a method of making decisions using data. A test result (calculated from the null hypothesis and the sample) is called statistically significant if it is deemed unlikely to have occurred by chance, assuming the truth of the null hypothesis. A statistically significant result, when a probability (p-value) is less than a threshold (significance level), justifies the rejection of the null hypothesis, but only if the a priori probability of the null hypothesis is not high.

→In the typical application of ANOVA, the null hypothesis is that all groups are simply random samples of the same population. This implies that all treatments have the same effect (perhaps). Rejecting the null hypothesis implies that different treatments results in altered effects.   


More on ANOVA

ANOVA is the synthesis of several ideas and it is use for multiple purposes. As a consequence, it is difficult to define concisely or precisely.

Classical ANOVA for balanced data does three things at once:
- As exploratory data analysis, an ANOVA is an organization of an additive data decomposition, and its sums of squares indicate the variance of each component of the decomposition (or, equivalently, each set of terms of a linear model).
- Comparisons of mean squares, along with F-tests ... allow testing of a nested sequence of models.
- Closely related to the ANOVA is a linear model fit with coefficient estimates and standard errors. In short, ANOVA is a statistical tool used in several ways to develop and confirm an explanation for the observed data.

Additionally:

- It is computationally elegant and relatively robust against violations of its assumptions.
- ANOVA provides industrial strength (multiple sample comparison) statistical analysis.
- It has been adapted to the analysis os a variety of exprimental designs.

F-test

An F-test is any statistical test in which the test statistic has an F-distribution under the null hypothesis.

It is most often used when comparing statistical models that have been fitted to a data set, in order to identify the model that best fits the population from which the data were sampled.

Exact "F-test" mainly arise when the models have been fitted to the data using least squares.

The name was coined by George W. Snedecor, in honour of Sir Ronald A.Fisher.

Fisher initially developed the statistic as the variance ratio in the 1920s.

The formula for the one-way ANOVA F-test statistic is:

  

Read More

Decision Making in R Language

Decision Making

Decision Making structure, evaluate a condition and depending on the result some other code is processed.

There are three types of decision making statements in R

• If Statement
• If-Else Statement
• Switch Statement

If Statement

An if statement consists of a Boolean expression followed by one or more statement.

Syntax of is Statement :-

if(Boolean expression)
{
   // Code to be executed
}

If-Else Statements

An if statement can be followed by an optional else statement, which executes when the Boolean expression is false.

The syntax of if else is :

if (Boolean_expression)
{
   // Code to be executed
} 
else
{
    // Code to be executed
}

Example :-

   
Switch Statement

A switch statement allows a variable to be tested for equality against a list of values.

R has following Switch Statement :

    switch (expression, case1, case2..)
→ We can have any number of cases in switch.
→ No Default is available in switch.

Read More

Operators use in R Language

Operators

Operators are special symbols or phrases that programmers use to check, combine or change values.

E.g., '+' is a operator, used to add two values, like 2 +7

There are two types of operators:

• Unary Operators : Operators on one operand, E.g.  -4 , 1x
• Binary Operators : Operators on two operands. E.g  5+7

Operators are further divided into

• Arithmetic Operators
• Relational Operators
• Logical Operators

Arithmetic Operators

    It is used to perform arithmetic and mathematical operations.

 Examples :-

Relational Operators

Relational operators is used to compare two values or see the relation between values.

Examples :-

Logical Operators 

It is used to perform logical operation on values.

Examples :-

Read More

Variables and Data types in R Language

R- Variables

A Named storage that can be used by our program.

A Variable can be assigned value in three ways:

• var_name <- Value                   # leftward operator
• var_name = value                     # assignment operator
• var_value -> Variables_name   # rightward operator

To see the data type of a variable we use class( ) function.

To see the list of variables we use ls( ) function.

To delete a variable we use rm( ) function.

R- Data types

When we create some variable in a memory, then it is stored according to the value inside that variable.

Based on the type of value the memory is allocated.

Following are the basic Data Types:

• Vectors : a combination of values.
• List : Can contain many different type of objects.
• Matrices : A two dimensional Data Set.
• Arrays : Multi-dimensional Data Set.
• Factors : Factors store Vectors along with labels.
• Data Frames : Tabular Data objects, can have multiple types.

Read More

Data Management : Sorting and Ordering in R Language

Sorting

Sort function sorts the values of a vector in ascending order (by default) or descending order.

Syntax

sort (x, decreasing = FALSE , ....,)
sort (x, decreasing = FALSE , na . last = NA, ...)

x                       Vector of values to be sorted

decreasing        Should the sort be increasing or decreasing

na.last               for controlling the treatment of NAs.
                          If TRUE, missing values in the data are put last;
                          If FALSE, they are put first;
                          If NA, they are removed.


Example

> y  <-  c(8,5,7,6)
> y
 [1]  8 5 7 6

> sort (y)
 [1]  5 6 7 8

> sort (y , decreasing = TRUE)
 [1]   8 7 6 5

Ordering

Order function sorts a variable according to the order of variable.

Syntax

order (x , decreasing = FALSE, ...,)
order (x, decreasing = FALSE, na.last = TRUE, ...)

x                         Vector of values to be sorted

decreasing          Should the sort be increasing or decreasing

na.last                 for controlling the treatment of NAs.
                           If TRUE, missing values in the data are put last;
                           If FALSE, they are put first;
                           If NA, they are removed. 

Example

> y  <-    c(8,5,7,6)
> y
[1]   8 5 7 6

>  order (y) 
 [1]   2 4 3 1

> order (y, decreasing = TRUE)
 [1]  1 3 4 2

Read More

Data Management : Sequences in R Language

Sequences

The regular sequences can be generated in R.

Syntax :-

seq ( )

seq (from = 1, to = 1, by = ( ( to - from) / (length.out - 1) ) , length.out = NULL, along.with = NULL, ....)

Examples:

> seq (10)
   [1]  1  2  3  4  5  6  7  8  9  10

is the same as
> seq (1 : 10)

Assignment of an index-vector

> x <- c (9,8,7,6)
> ind <- seq (along = x)
> ind
   [1]  1 2 3 4


Accessing a value in the vector through index vector
→ Accessing an element of an index-vector
> x [ind [2] ]
   [1] 8 


Generating sequence of dates

Generating current time and date

Sys.time ( )  command provides the current time and date from the computer system.
> Sys.time ( )
  [1]  "2017-01-01  09:17:01  IST"

Sys.Date ( ) command provides the current date from the computer system.
> Sys.Date ( )
    [1]  "2017-01-01"

Usage
seq (from, to, by, length.out = NULL, along.with = NULL, ...)

Arguments
from             starting date (Required)
to                  end date (Optional)
by                 Increment of the sequence.  "day" , "week" , "month" , "quarter" or "year".
length.out     Integer, optional. Desired length of the sequence.
along.with    take the length from the length of this argument.

Sequence of first day of years

> seq (as.Date ("2010-01-01") , as.Date ("2017-01-01"), by = "Years")
[1] "2010-01-01"  "2011-01-01"  "2012-01-01"  "2013-01-01"
[5] "2014-01-01"  "2015-01-01"  "2016-01-01"  "2017-01-01"

Sequence of days

> seq (as.Date ("2017-01-01") , by = "days", length = 6)

Sequence of months

> seq (as.Date ("2017-01-01") , by = "days", length = 6)

Sequence of years

> seq (as.Date ("2017-01-01") , by = "years", length = 6)

To find sequence with defining start and end dates

Read More

R for Everyone: Advanced Analytics and Graphics

Using the open source R language, you can build powerful statistical models to answer many of your most challenging questions. R has traditionally been difficult for non-statisticians to learn, and most R books assume far too much knowledge to be of help. R for Everyone, Second Edition, is the solution.

Buy Book: R for Everyone: Advanced Analytics and Graphics

Download PDF : R for Everyone: Advanced Analytics and Graphics

   About the Author - 

Jared P. Lander is the Chief Data Scientist of Lander Analytics, a New York-based data science firm that specializes in statistical consulting and training services, the organizer of the New York Open Statistical Programming Meetup—the world’s largest R meetup—and the New York R Conference and an adjunct professor of statistics at Columbia University. With an M.A. from Columbia University in statistics and a B.S. from Muhlenberg College in mathematics, he has experience in both academic research and industry. Very active in the data community, Jared is a frequent speaker at conferences, universities and meetups around the world. His writings on statistics can be found and been featured in publications such as Forbes and the Wall Street Journal.

Read More

Basics Calculations in R Language

- > is the prompt sign in R


- The assignment operators are the left arrow with dash <- and equal sign =.
    > x <- 20 assigns the value 20 to x.
    > x = 20 assigns the value 20 to x.
    Initially only <- was available in R.


-  > x = 20 assigns the value 20 to x.
   > y = x + 2 assigns the value 2 * x to y.
   > z = x + y assigns the value x + y to z.


# : The character # marks the beginning of a comment. All characters until the end of the line are ignored.
  > # mu is the mean
  > # x <- 20 is treated as comment only.

Capital and small letters are different.
 > x <- 20 and > x <- 20 are different

The command c (1,2,3,4,5) combines the numbers 1,2,3,4 and 5 to a vector.

 R as a calculator
 
 > 2 + 3              # Command
 [1] 5                  # Output

  > 2 * 3              # Command
  [1]  6                 # Output

Multiplication and Division x * y , x/y

> c ( 2,3,5,7) * 3
[1]   6 9 15 21 

Addition and Subtraction x + y , x - y

> (2,3,5,7) + 10
[1] 12 13 15 17

Read More

Business Analytics with R Language

Business Analytics 

Definition 
    "Study of business data using statistical technique and programming for creating decision support and insights for achieving business goals"

Why uses it ? How?
 - Across Domain
         * Dashboard
         * Models
 - Across A Company

Who creates it ? How?
  - Skills Needed Business Perception

Business Intelligence
  → Business Intelligence is a set of theories, methodologies, processes, architecture, and technologies that transform raw data into meaningful and useful information for business purposes.

What is Data Science ?
→ Science of Studying Data:
   Programming + Statistics + Business

Read More

PHP: The Complete Reference

PHP is a server-side programming language mainly used for web development and is also used as a general purpose programming language. It has become a rage in the Internet world. PHP: The Complete Reference, as the name suggests is a complete reference guide to the widely popular PHP.

This book deals with explaining how to personalize the PHP work space, define operators and variables, manipulate strings and arrays and the way in which one can use HTML. It also covers details on how to access database information, track client-side preferences using cookies, execute FTP and e-mail transactions and publish your applications to the Web. Additionally, this book deals in PHP's next generation Web 2.0 design features including AJAX, XML and RSS.

One can also learn to use PHP's object-oriented tools to build blogs, guest books and feedback pages with server-side file storage. PHP: The Complete Reference is a step by step guide to mastering PHP. Starting from the basic to the most advanced level, this book covers each aspect in great detail. This book was published by McGraw-Hill Education on 30 November 2007 and is available in paperback.

Key Features

• Detailed coverage of PHP's next-generation Web 2.0 design features, including AJAX, XML and RSS is included.

Buy : PHP: The Complete Reference

Your One-Stop Guide to Web Development with PHP--Covers PHP 5.2

Build dynamic, cross-browser Web applications with PHP--the server-side programming language that's taken the Internet by storm. Through detailed explanations and downloadable code examples, this comprehensive guide shows you, step-by-step, how to configure PHP, create PHP-enabled Web pages, and put every advanced development tool to work.

PHP: The Complete Reference explains how to personalize the PHP work space, define operators and variables, manipulate strings and arrays, deploy HTML forms and buttons, and process user input. You'll learn how to access database information, track client-side preferences using cookies, execute FTP and e-mail transactions, and publish your applications to the Web. You'll also get in-depth coverage of PHP's next-generation Web 2.0 design features, including AJAX, XML, and RSS.

• Install PHP and set up a customized development environment
• Work with variables, operators, loops, strings, arrays, and functions
• Integrate HTML controls, text fields, forms, radio buttons, and checkboxes
• Accept and validate user-entered data from Web pages
• Simplify programming using PHP's object-oriented tools
• Build blogs, guest books, and feedback pages with server-side file storage
• Write MySQL scripts that retrieve, modify, and update database information
• Set cookies, perform FTP transactions, and send e-mails from PHP sessions
• Build AJAX-enabled Web pages
• Draw graphics on the server
• Create XML components and add RSS feeds

About the Author

About the Author: Steven Holzner is an author of over 100 technology books, many of which are bestsellers. His works mainly pertain to online applications and components of Ajax including JavaScript, XML, browser objects and Web services. Steven also teaches programming classes at Fortune 500 companies and has also been a faculty at the Cornell University and MIT. His well known works include books like Ajax For Dummies and the Ajax Visual Blueprint. Steven has also worked as a contributing editor for PC magazine.

Read More

Tasks in Data Mining for R Language

Anomaly Detection 
→ Identification of unusual patterns, outliers, which help us in understanding the variation in data.
 Example:-

Association Rule Mining
→ Also referred to as market basket analysis, this method is used for discovering interesting "association" patterns among the variables.
Example :- The beer-diaper syndrome

Clustering
→ Identifying groups/classes in data which are similar to each other.
     The similarity inside the "cluster" is high and between the "clusters" is low.

Classification
→ Classification is the process of identifying to which category does an observation belong.
Example:-

 
Regression
→ With the help of regression, we can identify the extent of relationship among variables.
      Understanding how the "dependent" variable varies with respect to the variation in "independent" variable.

  
Who uses R?

1. FACEBOOK :- For behavior analysis related status updates and profile pictures.
2. GOOGLE :- For advertising effectiveness and economic forecasting.
3. TWITTER :- For data visualization and semantic clustering. 

Tasks to be performed

• Data Importing :- Import the "Houses for sale" dataset.
• Data Pre-processing :- Understand the structure of data and find correlation between different data entities.
• Data Mining :- Use Linear Regression to predict the rates of houses.
• Pattern Evaluation :- Evaluate which model fits better for the dataset.

Read More