Data Mining Process in R Language

Phases in a typical Data Mining effort:

1.  Discovery
     Frame business problem
     Identify analytics component
     Formulate initial hypotheses

2.  Data Preparation
     Obtain dataset form internal and external sources
     Data consistency checks in terms of definitions of fields, units of measurement, time periods etc.,
     Sample

3.  Data Exploration and Conditioning
     Missing data handling, Range reason ability, Outliers,
     Graphical or Visual Analysis
     Transformation, Creation of new variables, and Normalization
     Partitioning into Training, validation, and Test datasets

4.  Model Planning
    - Determine data mining task such as prediction, classification etc.
     - Select appropriate data mining methods and techniques such as regression, neural networks, clustering etc.
     
5.  Model Building
     Building different candidate models using selected techniques and their variants using training data
     Refine and select the final model using validation data
     Evaluate the final model on test data
  
6.  Results Interpretation
      Model evaluation using key performance metrics

7. Model Deployment
       Pilot project to integrate and run the model on operational systems

Similar data mining methodologies developed by SAS and IBM Modeler (SPSS Clementine) are called SEMAA and CRISP-DM respectively

Data mining techniques can be divided into Supervised Learning Methods and Unsupervised Learning Methods

Supervised Learning
-  In supervised learning, algorithms are used to learn the function 'f' that can map input variables (X) into output variables (Y)
                        Y = f(X)
- Idea is to approximate 'f' such that new data on input variables (X) can predict the output variables (Y) with minimum possible error (ε)


Supervised Learning problem can be grouped into prediction and classification problems

Unsupervised Learning
  -  In Unsupervised Learning, algorithms are used to learn the underlying structure or patterns hidden in the data

Unsupervised Learning problems can be grouped into clustering and association rule learning problems

Target Population
  - Subset of the population under study
  - Results are generalized to the target population

Sample 
  - Subset of the target population

Simple Random Sampling
  - A sampling method where in each observation has an equal chance of being selected.

Random Sampling
  - A sampling method where in each observation does not necessarily have an equal chance of being selected

Sampling with Replacement
  - Sample values are independent

Sampling without Replacement
  - Sample values aren't independent

Sampling results in less no. of observation than the no. of total observation in the dataset

Data Mining algorithms
  - Varying limitations on number of observation and variables

Limitation due to computing power and storage capacity

Limitations due to statistical being used

How many observation to build accurate models?

Rare Event, e.g., low response rate in advertising by traditional mail or email
 - Oversampling of 'success' cases
 - Arise mainly in classification tasks
 - Costs of misclassification
 - Costs of failing to identify 'success' cases are generally more than costs of detailed review of all cases
 - Prediction of 'success is likely to come at cost of misclassifying more 'failure' cases as 'success' cases than usual

Read More

Steps to write a programe with Examples of Programming in R Language

Steps to write a programme

• A programme is a set of instructions or commands which are written in a sequence of operations i.e., what comes first and what comes after that.

• The objective of a programme is to obtain a defined outcome based on input variables.

• The computer is instructed to perform the defined task.

• Computer is an obedient worker but it has its own language.

• We do not understand computer's language and computer does not understand our language.

• The software help us and work like an interpreter between us and computer.

• We say something in software's language and software informs it to computer.

• Computer does the task and informs back to software.

• The software translates it to our language and informs us.

• Programme in R is written as a function using function.

• Write down the objective, i.e., what we want to obtain as an outcome.

• Translate it in the language of R.

• Identify the input and output variables.

• Identify the nature of input and output, i.e., numeric string, factor, matrix etc.

• Input and output variables can be single variable, vector, matrix or even a function itself.

• The input variables are the component of function which are reported in the argument of function ( ) .

• The output of a function can also be input to another function.

• The output of an outcome can be formatted as per the need and requirement.   

Tips :

1. Loops usually slower the speed of programmes, so better is to use vectors and matrices.
2. Use # symbol to write comment to understand the syntax.
3. Use the variable names which are easy to understand.
4. Don't forget to initialize the variables.

Example 1

Input variables : x, y, n (if x and y have different number of observations, choose different numbers, say n1 and n2)

We need summation, so use sum function or alternatively computer it through vectors.

Read More

Numpy in Python

Numpy

• Numpy is the core library for scientific computing in Python.
• It provides a high-performance multidimensional array object, and tools for working with these arrays.

Multi-dimension array-
Example:-

Advantage of Numpy over List

• Less Memory
• Fast
• Convenient

Numpy Operations 

• Find the dimension of the array
• Find the byte size of each element
• Find the data type of the elements

Numpy Special Functions

Read More

R Programming vs Python Comparison

R Programming

R is a programming language and free software environment for statistical computing and graphics that is supported by the R Foundation for Statistical Computing.

Python


python is an interpreted high-level programming language for general-purpose programming. It has a design philosophy that notably using significant white-space.

 1. Ease of Learning :-
    R is not ease to Learn and Python is easy to learn.

2. Speed :-
    R is a Low Level Language Slower and Python is a High Level Language Faster.

3. Data Handling Capabilities :- 
    R is Convenient for large Datasets and Python is Progressing with new Releases.

4. Graphics and Visualization :-
      R is Easy and Better and Python is Complex and Tedious.

5. Deep Learning Support :-
     R is New to Deep Learning and Python is Works Amazingly (TensorFlow).

6. Flexibility :- 
     R is Statistical Tests and Models and Python is Websites and Applications.

7. Code Repository & Libraries
    R is Huge Repository more Libraries and Python is Lesser Libraries.

8. Popularity Index :-
  

9. Job Scenario :-
 

10. Community & Customer Support :-

Read More

Linear Layout in Android

Linear layout is a simple layout used in android for layout designing. In the Linear layout all the elements are displayed in linear fashion means all the childs/elements of a linear layout are displayed according to its orientation. The value for orientation property can be either horizontal or vertical.

Types Of Linear Layout Orientation

There are two types of linear layout orientation:

1. Vertical
2. Horizontal

As the name specified these two orientations are used to arrange there child one after the other, in a line, either vertically or horizontally. Let’s we describe these in detail.

1.Vertical:

In this all the child are arranged vertically in a line one after the other. In below code snippets we have specified orientation “vertical” so the childs/views of this layout are displayed vertically.

 

2. Horizontal:

In this all the child are arranged horizontally in a line one after the other. In below code snippets we have specified orientation “horizontal” so the childs/views of this layout are displayed horizontally.

Important Note: All of the layout managers can be nested. This means that you can put a Relative Layout or Frame Layout as a child to Linear Layout.

Main Attributes In Linear Layout:

Now let’s  we discuss about the attributes that helps us to configure a linear layout and its child controls. Some of the most important attributes you will use with linear layout include:

1. orientation: The orientation attribute used to set the childs/views horizontally or vertically. In Linear layout default orientation is vertical.

Example:  Orientation vertical:

Example: Orientation Horizontal:

2. gravity: The gravity attribute is an optional attribute which is used to control the alignment of the layout like left, right, center, top, bottom etc.

Example: We have set gravity right for linear layout. So the buttons gets align from right side in Horizontal orientation.

 

3. layout_weight: The layout weight attribute specify each child control’s relative importance within the parent linear layout.

Example: weight property for button in linear layout. In the below example one button is of weight 2 and other is of weight 1.

In the layout image you can notice Button with weight 2 gets more size related the other.

4. weightSum: weightSum is the sum up of all the child attributes weight. This attribute is required if we define weight property of the childs.

Example: In the same above example of weight, we can define weightSum value 3.

Example of Linear Layout:

Now lets design 2 linear layout UI. First we have designed using weight attribute and second without using it. So below layout output will clear the difference between them:

Example 1: First we will design Android Linear Layout without using weight property

In this example we have used one TextView and 4 Button. The orientation is set to vertical.

Below is the code of activity_main.xml

Output Screen:

Read More

Activity Lifecycle in Android

Activity Lifecycle: Activity is one of the building blocks of Android OS. In simple words Activity is a screen that user interact with. Every Activity in android has lifecycle like created, started, resumed, paused, stopped or destroyed. These different states are known as Activity Lifecycle. In other words we can say Activity is a class pre-written in Java Programming.

Below is Activity Lifecycle Table:

Short description of Activity Lifecycle example:

onCreate() – Called when the activity is first created

onStart() – Called just after it’s creation or by restart method after onStop(). Here Activity start becoming visible to user

onResume() – Called when Activity is visible to user and user can interact with it

onPause() – Called when Activity content is not visible because user resume previous activity

onStop() – Called when activity is not visible to user because some other activity takes place of it

onRestart() – Called when user comes on screen or resume the activity which was stopped

onDestroy() – Called when Activity is not in background

Below Activity Lifecycle Diagram Shows Different States:

Different Types of Activity Lifecycle States:

Activity have different states or it’s known as Activity life cycle. All life cycle methods aren’t required to override but it’s quite important to understand them. Lifecycles methods can be overridden according to requirements.

LIST OF ACTIVITY LIFECYCLE METHODS OR STATES:

Activity Created: onCreate(Bundle savedInstanceState):

onCreate() method is called when activity gets memory in the OS. To use create state we need to override onCreate(Bundle savedInstanceState) method. Now there will be question in mind what is Bundle here, so Bundle is a data repository object that can store any kind of primitive data and this object will be null until some data isn’t saved in that.

When an Activity first call or launched then onCreate(Bundle savedInstanceState) method is responsible to create the activity.

When ever orientation(i.e. from horizontal to vertical or vertical to horizontal) of activity gets changed or when an Activity gets forcefully terminated by any Operating System then savedInstanceState i.e. object of Bundle Class will save the state of an Activity.

It is best place to put initialization code.

Activity Started: onStart():

onStart() method is called just after it’s creation. In other case Activity can also be started by calling restart method i.e after activity stop. So this means onStart() gets called by Android OS when user switch between applications. For example, if a user was using Application A and then a notification comes and user clicked on notification and moved to Application B, in this case Application A will be paused. And again if a user again click on app icon of Application A then Application A which was stopped will again gets started.

Activity Resumed:.onResume():

Activity resumed is that situation when it is actually visible to user means the data displayed in the activity is visible to user. In lifecycle it always gets called after activity start and in most use case after activity paused (onPause).

Activity Paused: onPause():

Activity is called paused when it’s content is not visible to user, in most case onPause() method called by Android OS when user press Home button (Center Button on Device) to make hide.

Activity also gets paused before stop called in case user press the back navigation button. The activity will go in paused state for these reasons also if a notification or some other dialog is overlaying any part (top or bottom) of the activity (screen). Similarly, if the other screen or dialog is transparent then user can see the screen but cannot interact with it. For example, if a call or notification comes in, the user will get the opportunity to take the call or ignore it.

Activity Stopped: onStop():

Activity is called stopped when it’s not visible to user. Any activity gets stopped in case some other activity takes place of it. For example, if a user was on screen 1 and click on some button and moves to screen 2. In this case Activity displaying content for screen 1 will be stopped.

Every activity gets stopped before destroy in case of when user press back navigation button. So Activity will be in stopped state when hidden or replaced by other activities that have been launched or switched by user. In this case application will not present anything useful to the user directly as it’s going to stop.

Activity Restarted: onRestart():

Activity is called in restart state after stop state. So activity’s onRestart() function gets called when user comes on screen or resume the activity which was stopped. In other words, when Operating System starts the activity for the first time onRestart() never gets called. It gets called only in case when activity is resumes after stopped state.

Activity Destroyed: onDestroy():

Any activity is known as in destroyed state when it’s not in background. There can different cases at what time activity get destroyed.

First is if user pressed the back navigation button then activity will be destroyed after completing the lifecycle of pause and stop.

In case if user press the home button and app moves to background. User is not using it no more and it’s being shown in recent apps list. So in this case if system required resources need to use somewhere else then OS can destroy the Activity.

After the Activity is destroyed if user again click the app icon, in this case activity will be recreated and follow the same lifecycle again. Another use case is with Splash Screens if there is

call to finish() method from onCreate() of an activity then OS can directly call onDestroy() with calling onPause() and onStop().

Activity Lifecycle Example:

In the below example we have used the below JAVA and Android topics:

JAVA Topics Used: Method Overriding, static variable, package, Inheritance, method and class.

Android Topic Used: We have used Log class which is used to printout message in Logcat. One of the important use of Log is in debugging.

First we will create a new Android Project and name the activity as HomeActivity. In our case we have named our App project as Activity Lifecycle Example.

We will initialize a static String variable with the name of the underlying class using getSimpleName() method. In our case HOME_ACTIVITY_TAG is the name of the String variable which store class name HomeActivity.

Now we will create a new method which will print message in Logcat.

Now we will override all activity lifecycle method in Android and use showLog() method which we creating for printing message in Logcat.

Complete JAVA code of HomeActivity.java:

 

When creating an Activity we need to register this in AndroidManifest.xml file. Now question is why need to register? It's actually because manifest file has the information which Android OS read very first. When registering an activity other information can also be defined within manifest like Launcher Activity (An activity that should start when user click on app icon).

Here is declaration example in AndroidManifest.xml file

 

Output Of Activity Lifecycle:

When you will run the above program you will notice a blank white screen will open up in Emulator. You might be wondering where is default Hello world screen. Actually we have removed it by overriding onCreate() method. Below is the blank white screen that will pop up.

 

Now go to Logcat present inside Android Monitor: Scroll up and you will notice three methods which were called: Activity Created, Activity started and Activity resumed.

 So this clears:

first onCreate() method was called when activity was created

second onStart() method was called when activity start becoming visible to user

Finally onResume() method was called when activity is visible to user and user can interact with it

Now press the back button on the Emulator and exit the App:

Go to Logcat again and scroll down to bottom. You will see 3 more methods were called: Activity paused, Activity stopped and Activity is being destroyed.

 So this clears:

• onPause() method was called when user resume previous activity
• onStop() method was called when activity is not visible to user
• Last onDestroy() method was called when Activity is not in background

Important Note: In the above example onRestart() won’t be called because there was no situation when we can resume the onStart() method again. In future example we will show you onRestart() in action as well.

Importance Of Activity Life Cycle:

Activity is the main component of Android Application, as every screen is an activity so to create any simple app first we have to start with Activities. Every lifecycle method is quite important to implement according to requirements, However onCreate(Bundle state) is always needed to implement to show or display some content on screen

Read More