#run this before class
library(tidyverse)
library(repr)
theta <- seq(0,2*pi, length.out = 300)
circle <- tibble(X = sin(theta) + 0.75 * runif(300, min = 1, max = 2),
                 Y = cos(theta) + 0.75 * runif(300, min = 1, max = 2))
options(repr.plot.width = 10, repr.plot.height = 10)
circle_plot <- ggplot(circle, aes(x = X, y = Y)) +
    geom_point(alpha = 0.5, size = 2) +
    geom_smooth(method = "lm", se = FALSE) +
    xlim(c(-0.5, 2.5)) +
    ylim(c(-0.5, 2.5))

zigzag <- tibble(X = seq(0,3*pi, length.out = 200),
                Y = cos(X) + runif(200, min = 1, max = 2))
zigzag_plot <- ggplot(zigzag, aes(x = X, y = Y)) +
    geom_point(alpha = 0.5) +
    geom_smooth(method = "lm", se = FALSE)

DSCI 100 - Introduction to Data Science¶

Lecture 9 - Linear Regression¶

Artist: Allison Horst

Housekeeping¶

Nov 9: project proposal due
Nov 11: lecture cancelled (Remembrance Day)
Nov 16: Quiz 2!
- 11am - 12:30pm (both sections)
- Section 004: UBC Life 2201 (the usual classroom)
- Section 001: LSK201
- Topics: version control, classification, and regression

Review: Classification¶

In classification, we want to predict a categorical variable

What political party will someone vote for in the next US election?
Is a new patient diseased or healthy?
Is a new email spam or not?
Will a new, potential customer buy your product?

Review: Regression¶

In regression, we want to predict a quantitative variable

E.g. predict the price of a 2000 square foot home:

Linear Regression¶

Last time we used a nearest neighbours approach. Today we will find the line of "best-fit" through the data points:

Linear Regression¶

And then we "look up" the value we want to predict of off of the line.

Linear Regression¶

How do we choose the line of "best fit"? We can draw many lines through the data:

Linear Regression¶

Just like last time, we will evaluate our model using the error in our predictions.

We choose the line that minimzes the RMSE (square root of average squared vertical distance between prediction and training data response value).

Why did I say "RMSE" and not "RMSPE" above?

Linear Regression¶

The equation for a straight line is: $Y = \beta_0 + \beta_1X$

$\beta_0$ is the y-intercept of the line (the value where the line cuts the y-axis)
$\beta_1$ is the slope of the line

We can then write:

$(\text{house price}) = \beta_0 + \beta_1\times(\text{house size})$

And finally, fill in the values for $\beta_0$ and $\beta_1$:

$(\text{house price}) = 13268.08 + 139\times(\text{house size})$

Linear Regression vs Nearest Neighbours¶

Advantages of straight line:
- interpretable: coefficients are slopes, intercepts; K-NN not easy to interpret
- computation-efficient: can fit the model much faster than K-NN
- data-efficient: needs far fewer data to find a reasonable fit
Major disadvantage of straight line: cannot capture nonlinear patterns

Multiple Linear Regression¶

Similar to classification, we can do regression with > 1 predictor via k-nn or linear regression. But how do we fit a line with many predictors?

We don't fit a line - we fit a hyperplane!

$$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_m X_m$$

View examples from the textbook:¶

## # A tibble: 3 x 5
##   term        estimate std.error statistic  p.value

##   <chr>          <dbl>     <dbl>     <dbl>    <dbl>
## 1 (Intercept)   52690.  13745.        3.83 1.41e- 4
## 2 sqft            155.      6.72     23.0  4.46e-83
## 3 beds         -20209.   5734.       -3.52 4.59e- 4

$house\; price = 52690 + 155⋅(house \; size) − 20209 ⋅ (number\; of\; bedrooms)$

Why/why not linear/k-nn regression?¶

Models are not "one-size fits all."

circle_plot

Why/why not linear/k-nn regression?¶

Models are not "one-size fits all."

zigzag_plot

Worksheet time - go for it!¶

Break time!¶

Take a break! Lecture will resume at 2:30 PM (PT)