{"id":74,"date":"2020-08-18T00:19:00","date_gmt":"2020-08-18T00:19:00","guid":{"rendered":""},"modified":"2021-01-10T01:29:20","modified_gmt":"2021-01-10T01:29:20","slug":"linear-regression-you-should-always-try-it-first","status":"publish","type":"post","link":"https:\/\/therenaissanceraven.com\/?p=74","title":{"rendered":"Linear Regression – You should always try it first!"},"content":{"rendered":"

You should always try to use linear regression first when you are faced with an unknown data source for regression analysis, the main and simple reason behind this is:<\/p>\n

Linear Regression creates a stable and accurate model, unlikely to fail miserably with erratic data. <\/b><\/p>\n

If you are able to model the data correctly to fit a linear model, it is very likely that you have derived the true equation to solve the problem. Lets see an example with a very basic example with a very linear equation:<\/p>\n

Y = x1+x2<\/p>\n

<\/p>\n

\n
from<\/span> sklearn.linear_model<\/span> import<\/span> LinearRegression
from<\/span> sklearn.ensemble<\/span> import<\/span> RandomForestRegressor
from<\/span> sklearn.model_selection<\/span> import<\/span> train_test_split
import<\/span> numpy<\/span> as<\/span> np<\/span>
import<\/span> matplotlib.pyplot<\/span> as<\/span> plt<\/span>

# generating data<\/span>
x1 =<\/span> np.<\/span>random.<\/span>randint(100<\/span>,size=<\/span>100<\/span>)
x2 =<\/span> np.<\/span>random.<\/span>randint(100<\/span>,size=<\/span>100<\/span>)
X =<\/span> np.<\/span>stack((x1,x2),axis=<\/span>1<\/span>)

# purely linear equation<\/span>
y =<\/span> x1+<\/span>x2

# splitting data<\/span>
X_train, X_test, y_train, y_test =<\/span> train_test_split(X,y,test_size=<\/span>0.8<\/span>)

# linear regression<\/span>
linearRegressor =<\/span> LinearRegression()
linearRegressor.<\/span>fit(X_train,y_train)
y_pred_linear =<\/span> linearRegressor.<\/span>predict(X_test)
print<\/span>(linearRegressor.<\/span>intercept_)  # prints near zero<\/span>
print<\/span>(linearRegressor.<\/span>coef_)  # prints [1,1]<\/span>

# random forest regression<\/span>
randomForestRegressor =<\/span> RandomForestRegressor(max_depth=<\/span>100<\/span>,n_estimators=<\/span>100<\/span>)
randomForestRegressor.<\/span>fit(X_train,y_train)
y_pred_randomForest =<\/span> randomForestRegressor.<\/span>predict(X_test)

# plotting<\/span>
plt.<\/span>scatter(y_test,y_pred_linear,label=<\/span>'Linear'<\/span>)
plt.<\/span>scatter(y_test,y_pred_randomForest,label=<\/span>'Random Forest'<\/span>)
plt.<\/span>plot([np.<\/span>min(y_test),np.<\/span>max(y_test)],[np.<\/span>min(y_test),np.<\/span>max(y_test)],'k-'<\/span>,label=<\/span>'n=1'<\/span>)
plt.<\/span>legend()
plt.<\/span>show()
<\/pre>\n<\/div>\n
The results are shown below for degree of fit, additionally from printing the linear regression intercept and slope we can see that it derived the following equation:<\/p>\n
Y = 1*x1+1*x2+0<\/p>\n
\n
<\/a><\/div>\n
 <\/p>\n
Which is spot on.<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"
You should always try to use linear regression first when you are faced with an unknown data source for regression analysis, the main and simple reason behind this is: Linear Regression creates a stable and accurate model, unlikely to fail miserably with erratic data.  If you are able to model the data correctly to fit…<\/p>\n","protected":false},"author":2,"featured_media":109,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-74","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/posts\/74","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=74"}],"version-history":[{"count":1,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/posts\/74\/revisions"}],"predecessor-version":[{"id":110,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/posts\/74\/revisions\/110"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=\/wp\/v2\/media\/109"}],"wp:attachment":[{"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=74"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=74"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/therenaissanceraven.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=74"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}