Chapter 5 Interpreting Regression Results

Chapter description

A case study, on determining an individual’s characteristics that influence its health expenditures, illustrates the regression modeling process from start to finish. Subsequently, the chapter summarizes what we learn from the modeling process, underscoring the importance of variable selection.

5.1 Case study: MEPS health expenditures

5.1.1 Video

Video Overhead Details

Hide

A Details. MEPS health expenditures

This exercise considers data from the Medical Expenditure Panel Survey (MEPS), conducted by the U.S. Agency of Health Research and Quality. MEPS is a probability survey that provides nationally representative estimates of health care use, expenditures, sources of payment, and insurance coverage for the U.S. civilian population. This survey collects detailed information on individuals of each medical care episode by type of services including physician office visits, hospital emergency room visits, hospital outpatient visits, hospital inpatient stays, all other medical provider visits, and use of prescribed medicines. This detailed information allows one to develop models of health care utilization to predict future expenditures. You can learn more about MEPS at http://www.meps.ahrq.gov/mepsweb/.

We consider MEPS data from the panels 7 and 8 of 2003 that consists of 18,735 individuals between ages 18 and 65. From this sample, we took a random sample of 2,000 individuals that appear in the file HealthExpend. From this sample, there are 1,352 that had positive outpatient expenditures.

Our dependent variable is the amount of expenditures for outpatient visits, expendop. For MEPS, outpatient events include hospital outpatient department visits, office-based provider visits and emergency room visits excluding dental services. (Dental services, compared to other types of health care services, are more predictable and occur in a more regular basis.) Hospital stays with the same date of admission and discharge, known as “zero-night stays,” were included in outpatient counts and expenditures. (Payments associated with emergency room visits that immediately preceded an inpatient stay were included in the inpatient expenditures. Prescribed medicines that can be linked to hospital admissions were included in inpatient expenditures, not in outpatient utilization.)

Hide

B Details. Overhead MEPS health expenditures

Data from the Medical Expenditure Panel Survey (MEPS), conducted by the U.S. Agency of Health Research and Quality (AHRQ).

  • A probability survey that provides nationally representative estimates of health care use, expenditures, sources of payment, and insurance coverage for the U.S. civilian population.
  • Collects detailed information on individuals of each medical care episode by type of services including
    • physician office visits,
    • hospital emergency room visits,
    • hospital outpatient visits,
    • hospital inpatient stays,
    • all other medical provider visits, and
    • use of prescribed medicines.
  • This detailed information allows one to develop models of health care utilization to predict future expenditures.
  • We consider MEPS data from the first panel of 2003 and take a random sample of n = 2, 000 individuals between ages 18 and 65.

Hide

C Details. Outcome variable

Our dependent variable is expenditures for outpatient admissions.

  • For MEPS, inpatient admissions include persons who were admitted to a hospital and stayed overnight.
  • In contrast, outpatient events include hospital outpatient department visits, office-based provider visits and emergency room visits excluding dental services.
    • Hospital stays with the same date of admission and discharge, known as “zero-night stays,” were included in outpatient counts and expenditures.
    • Payments associated with emergency room visits that immediately preceded an inpatient stay were included in the inpatient expenditures.
    • Prescribed medicines that can be linked to hospital admissions were included in inpatient expenditures, not in outpatient utilization.

Hide

D Details. Explanatory variables

9 variables in the database. Here 13 most relevant.

\[ {\small \begin{array}{ll} expendop & \text{Amounts of expenditures for outpatient visits} \\ gender & \text{Indicate gender of patient (=1 if female, =0 if male)} \\ age & \text{Age in years between 18 and 65 }\\ race & \text{Race of patient described as Asian, Black, Native, White and other} \\ region & \text{Region of patient described as WEST, NORTHEAST, MIDWEST and SOUTH} \\ educ & \text{Level of education received described by words (LHIGHSC, HIGHSCH and COLLEGE)} \\ phstat & \text{Self-rated physical health status described as EXCE, VGOO, GOOD, FAIR and POOR} \\ mpoor & \text{Self-rated mental health (=1 if poor or fair, =0 if good to excellent mental health)} \\ anylimit & \text{Any activity limitation (=1 if any functional/activity limitation, =0 if otherwise)} \\ income & \text{Income compared to poverty line described as POOR, NPOOR, LINCOME, MINCOME and HINCOME} \\ insure & \text{Insurance coverage (=1 if covered by public/private health insurance in any month of 1996, =0 otherwise)} \\ usc & \text{1 if dissatisfied with one's usual source of care} \\ unemploy & \text{Employment status of patients} \\ managedcare & \text{1 if enrolled in an HMO or gatekeeper plan} \\ \end{array}} \]

Hide

E Details. Case study outline

The next series of exercises leads you through an analysis of the steps for understanding a complex data set. Because of the complexity of the data, in each step only a sample of procedures will be executed.

The outline consists of:

  • Summary statistics
  • Splitting the data into training and testing portions with initial model fits
  • Selecting variables to be included in the model

5.1.2 Exercise. Summarizing data

Assignment Text

With a complex dataset, you will probably want to take a look at the structure of the data. You are already familiar with taking a [summary()] of a dataframe which provides summary statistics for many variables. You will see that several variables in this dataframe are categorical, or factor, variables. We can use the table() function to summarize them.

After getting a sense of the distributions of explanatory variables, we want to take a deeper dive into the distribution of the outcome variable, expendop. We will do this by comparing the histograms of the variable to that of its logarithmic version.

To examine relationships of the outcome variable visually, we look to scatterplots for continuous variables (such as age) and boxplots for categorical variables (such as phstat).

Instructions

  • Examine the structure of the meps dataframe using the str() function. Also, get a [summary()] of the dataframe.
  • Examine the distribution of the race variable using the table() function.
  • Compare the expenditures distribution to its logarithmic version visually via histograms plotted next to another. par(mfrow = c(1, 2)) is used to organize the plots you create.
  • Examine the distribution of logarithmic expenditures in terms of levels of phstat visually using the boxplot() function.
  • Examine the relationship of age versus logarithmic expenditures using a scatter plot. Superimpose a local fitting line using the lines() and lowess() functions.
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiNtZXBzIDwtIHJlYWQuY3N2KFwiQ1NWRGF0YVxcXFxIZWFsdGhNZXBzLmNzdlwiLCBoZWFkZXIgPSBUUlVFKVxubWVwcyA8LSByZWFkLmNzdihcImh0dHBzOi8vYXNzZXRzLmRhdGFjYW1wLmNvbS9wcm9kdWN0aW9uL3JlcG9zaXRvcmllcy8yNjEwL2RhdGFzZXRzLzdiN2RhYjZkMGM1MjhlNGNkMmY4ZDBlMGZjNzgyNGEyNTQ0MjliZjgvSGVhbHRoTWVwcy5jc3ZcIiwgaGVhZGVyID0gVFJVRSkiLCJzYW1wbGUiOiIjIEV4YW1pbmUgdGhlIHN0cnVjdHVyZSBhbmQgZ2V0IGEgc3VtbWFyeSBvZiB0aGUgYG1lcHNgIGRhdGFmcmFtZSBcbnN0cihfX18pXG5zdW1tYXJ5KF9fXylcblxuIyBFeGFtaW5lIHRoZSBkaXN0cmlidXRpb24gb2YgdGhlIGByYWNlYCB2YXJpYWJsZSBcbnRhYmxlKF9fXylcblxuIyBDb21wYXJlIHRoZSBleHBlbmRpdHVyZXMgZGlzdHJpYnV0aW9uIHRvIGl0cyBsb2dhcml0aG1pYyB2ZXJzaW9uIHZpc3VhbGx5XG5wYXIobWZyb3cgPSBjKDEsIDIpKVxuaGlzdChfX18sIG1haW4gPSBcIlwiLCB4bGFiID0gXCJvdXRwYXRpZW50IGV4cGVuZGl0dXJlc1wiKVxuaGlzdChsb2coX19fKSwgbWFpbiA9IFwiXCIsIHhsYWIgPSBcImxvZyBleHBlbmRpdHVyZXNcIilcblxuIyBFeGFtaW5lIHRoZSBkaXN0cmlidXRpb24gb2YgbG9nYXJpdGhtaWMgZXhwZW5kaXR1cmVzIGluIHRlcm1zIG9mIGxldmVscyBvZiBgcGhzdGF0YCBcbnBhcihtZnJvdyA9IGMoMSwgMSkpXG5tZXBzJGxvZ2V4cGVuZCA8LSBsb2cobWVwcyRleHBlbmRvcClcbmJveHBsb3QobG9nZXhwZW5kIH4gX19fLCBkYXRhID0gbWVwcywgbWFpbiA9IFwiYm94cGxvdCBvZiBsb2cgZXhwZW5kXCIpXG5cbiMgRXhhbWluZSB0aGUgcmVsYXRpb25zaGlwIG9mIGFnZSB2ZXJzdXMgbG9nYXJpdGhtaWMgZXhwZW5kaXR1cmVzLiBTdXBlcmltcG9zZSBhIGxvY2FsIGZpdHRpbmcgbGluZS5cbnBsb3QoX19fLF9fXywgeGxhYiA9IFwiYWdlXCIsIHlsYWIgPSBcImxvZyBleHBlbmRcIilcbmxpbmVzKGxvd2VzcyhfX18sIF9fXyksIGNvbD1cInJlZFwiKSIsInNvbHV0aW9uIjoic3RyKG1lcHMpXG5zdW1tYXJ5KG1lcHMpXG50YWJsZShtZXBzJHJhY2UpXG5wYXIobWZyb3cgPSBjKDEsIDIpKVxuaGlzdChtZXBzJGV4cGVuZG9wLCBtYWluID0gXCJcIiwgeGxhYiA9IFwib3V0cGF0aWVudCBleHBlbmRpdHVyZXNcIilcbmhpc3QobG9nKG1lcHMkZXhwZW5kb3ApLCBtYWluID0gXCJcIiwgeGxhYiA9IFwibG9nIGV4cGVuZGl0dXJlc1wiKVxucGFyKG1mcm93ID0gYygxLCAxKSlcbm1lcHMkbG9nZXhwZW5kIDwtIGxvZyhtZXBzJGV4cGVuZG9wKVxuYm94cGxvdChsb2dleHBlbmQgfiBwaHN0YXQsIGRhdGEgPSBtZXBzLCBtYWluID0gXCJib3hwbG90IG9mIGxvZyBleHBlbmRcIilcbnBsb3QobWVwcyRhZ2UsbWVwcyRsb2dleHBlbmQsIHhsYWIgPSBcImFnZVwiLCB5bGFiID0gXCJsb2cgZXhwZW5kXCIpXG5saW5lcyhsb3dlc3MobWVwcyRhZ2UsIG1lcHMkbG9nZXhwZW5kKSwgY29sPVwicmVkXCIpIiwic2N0IjoiIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInN0clwiLCBub3RfY2FsbGVkX21zZz1cIlVzZSB0aGUgYHN0cmAgY29tbWFuZCB0byB2aWV3IHRoZSBzdHJ1Y3R1cmUgb2YgdGhlIGRhdGEuIFwiKSAlPiUgY2hlY2tfYXJnKC4sIFwib2JqZWN0XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIHRoZSBzdHJ1Y3R1cmUgb2YgdGhlIGRhdGEgYG1lcHNgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInN1bW1hcnlcIiwgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBzdW1tYXJ5YCBmdW5jdGlvbiB0byB2aWV3IGEgc3VtbWFyeSBvZiB0aGUgZGF0YS4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJvYmplY3RcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gc3BlY2lmeSB0aGF0IHdlIHdvdWxkIGxpa2UgdG8gdmlldyBhIHN1bW1hcnkgb2YgYG1lcHNgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInRhYmxlXCIsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgdGFibGVgIGZ1bmN0aW9uIHRvIGZpbmQgdGhlIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgb2YgZWFjaCByYWNlLiBcIikgJT4lIGNoZWNrX3Jlc3VsdCgpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIHRvIGNyZWF0ZSBhIHRhYmxlIGZyb20gYG1lcHNgIHRoYXQgc2VwYXJhdGVzIGJhc2VkIG9uIGByYWNlYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwYXJcIixpbmRleD0xLG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgcGFyYCBmdW5jdGlvbiB0byBhbHRlciB0aGUgcGxvdHRpbmcgZGV2aWNlIHRvIGhhdmUgMiBzaWRlLWJ5LXNpZGUgZ3JhcGhzLiBcIikgJT4lIGNoZWNrX2FyZyguLCBcIm1mcm93XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiVG8gY3JlYXRlIHR3byBzaWRlLWJ5LXNpZGUgZ3JhcGhzLCBzZXQgYG1mcm93YCBlcXVhbCB0byBgYygyLDEpYC4gXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwiaGlzdFwiLGluZGV4PTEsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgaGlzdGAgZnVuY3Rpb24gdG8gY3JlYXRlIGEgaGlzdG9ncmFtIG9mIGBleHBlbmRvcGAuXCIpICU+JSBjaGVja19hcmcoLiwgXCJ4XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3YW50IHRoZSBmaXJzdCBoaXN0b2dyYW0gdG8gYmUgbWFkZSB1c2luZyB0aGUgZGF0YSBmb3VuZCBpbiBgZXhwZW5kb3BgLCB3aGljaCBpcyBhIGNvbHVtbiBpbiBgbWVwc2AuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwiaGlzdFwiLGluZGV4PTIgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBoaXN0YCBmdW5jdGlvbiB0byBjcmVhdGUgYSBoaXN0b2dyYW0gb2YgdGhlIGxvZyBvZiBgZXhwZW5kb3BgLlwiKSAlPiUgY2hlY2tfYXJnKC4sIFwieFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgd2Ugd2FudCB0aGUgc2Vjb25kIGhpc3RvZ3JhbSB0byBiZSBtYWRlIHVzaW5nIHRoZSBsb2cgb2YgdGhlIGRhdGEgZm91bmQgaW4gYGV4cGVuZG9wYCwgd2hpY2ggaXMgYSBjb2x1bW4gaW4gYG1lcHNgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInBhclwiLGluZGV4PTIsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIGBwYXJgIHRvIHJlc2V0IHRoZSBwbG90dGluZyBkZXZpY2UgdG8gaG9sZCBhIHNpbmdsZSBncmFwaC4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJtZnJvd1wiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cInRvIHJlc2V0IHRoZSBwbG90dGluZyBkZXZpY2UsIHNldCBgbWZyb3dgIHRvIGJlIGVxdWFsIHRvIGBjKDEsMSkuIFwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtZXBzXCIpICU+JSBjaGVja19jb2x1bW4oXCJsb2dleHBlbmRcIiwgY29sX21pc3NpbmdfbXNnPVwiTWFrZSBzdXJlIHRvIGNyZWF0ZSBhIG5ldyBjb2x1bW4gaW4gYG1lcHNgIG5hbWVkIGBsb2dleHBlbmRgLlwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIlRoZSB2YWx1ZXMgaW4gYGxvZ2V4cGVuZGAgc2hvdWxkIGJlIGVxdWFsIHRvIHRoZSBsb2cgb2YgdGhlIHZhbHVlcyBpbiBgbWVwcyRleHBlbmRvcGAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwiYm94cGxvdFwiLCBub3RfY2FsbGVkX21zZz1cIlVzZSB0aGUgYGJveHBsb3RgIGZ1bmN0aW9uIHRvIGNyZWF0ZSBhIGJveHBsb3Qgc2hvd2luZyBgbG9nZXhwZW5kYCBiYXNlZCBvbiBgcGhzdGF0YC5cIikgJT4lIHtcbiMgICBjaGVja19hcmcoLiwgXCJmb3JtdWxhXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3YW50IGJveHBsb3RzIHRoYXQgc2hvdyBgbG9nZXhwZW5kYCBmb3IgZWFjaCB2YWx1ZSBvZiBgcGhzdGF0YC5cIilcbiMgICBjaGVja19hcmcoLiwgXCJkYXRhXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCBvdXIgZGF0YSBjb21lcyBmcm9tIHRoZSBkYXRhZnJhbWUgYG1lcHNgLlwiKVxuIyB9XG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwicGxvdFwiLCBub3RfY2FsbGVkX21zZz1cIlVzZSBgcGxvdGAgdG8gY3JlYXRlIGEgcGxvdCB0aGF0IHNob3dzIGBsb2dleHBlbmRgIGJhc2VkIG9uIGBhZ2VgLlwiKSAlPiUge1xuIyAgIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgaW5kZXBlbmRlbnQgdmFyaWFibGUgc2hvdWxkIGJlIGBhZ2VgLlwiKVxuIyAgIGNoZWNrX2FyZyguLCBcInlcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgZGVwZW5kZW50IHZhcmlhYmxlIHNob3VsZCBiZSBgbG9nZXhwZW5kYC5cIilcbiMgfVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcImxpbmVzXCIsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgbGluZXNgIGZ1bmN0aW9uIHRvIGFkZCBhIGxpbmUgdG8gb3VyIHBsb3QuIFwiKSBcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJsb3dlc3NcIiwgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBsb3dlc3NgIGZ1bmN0aW9uIHRvIGFkZCBhIHNtb290aGluZyBlZmZlY3QgdG8gb3VyIGxpbmUuIFwiKSAlPiUge1xuIyAgIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgZmlyc3QgYXJndW1lbnQgaW4gYGxvd2Vzc2Agc2hvdWxkIGJlIG91ciBgeGAgdmFyaWFibGUsIHdoaWNoIGlzIGBhZ2VgLlwiKVxuIyAgIGNoZWNrX2FyZyguLCBcInlcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgc2Vjb25kIGFyZ3VtZW50IGluIGBsb3dlc3NgIHNob3VsZCBiZSBvdXIgYHlgIHZhcmlhYmxlLCB3aGljaCBpcyBgbG9nZXhwZW5kYC5cIilcbiMgfVxuc3VjY2Vzc19tc2coXCJFeGNlbGxlbnQhIFN1bW1hcml6aW5nIGRhdGEsIHdpdGhvdXQgcmVmZXJlbmNlIHRvIGEgbW9kZWwsIGlzIHByb2JhYmx5IHRoZSBtb3N0IHRpbWUtY29uc3VtaW5nIHBhcnQgb2YgYW55IHByZWRpY3RpdmUgbW9kZWxpbmcgZXhlcmNpc2UuIFN1bW1hcnkgc3RhdGlzdGljcyBhcmUgYWxzbyBhIGtleSBwYXJ0IG9mIGFueSByZXBvcnQgYXMgdGhleSBpbGx1c3RyYXRlIGZlYXR1cmVzIG9mIHRoZSBkYXRhIHRoYXQgYXJlIGFjY2Vzc2libGUgdG8gYSBicm9hZCBhdWRpZW5jZS5cIikiLCJoaW50IjoiT25lIG9mIHRoZSBtb3N0IGltcG9ydGFudCBzdGVwcyBpbiBsaW5lYXIgcmVncmVzc2lvbiBpcyB1bmRlcnN0YW5kaW5nIHlvdXIgZGF0YS4gVXNpbmcgYSBjb21iaW5hdGlvbiBvZiA8Y29kZT5zdHIoKTwvY29kZT4gYW5kICA8Y29kZT5zdW1tYXJ5KCk8L2NvZGU+IGxldHMgeW91IHZpZXcgcXVhbnRpdGF0aXZlIHN0YXRpc3RpY3MgYWJvdXQgeW91ciBkYXRhLCB3aGlsZSA8Y29kZT5ib3hwbG90KCk8L2NvZGU+IGFuZCA8Y29kZT5oaXN0KCk8L2NvZGU+IGdpdmUgeW91IGEgbW9yZSB2aXN1YWwgcmVwcmVzZW50YXRpb24gb2YgeW91ciBkYXRhLiJ9

5.1.3 Exercise. Fit a benchmark multiple linear regression model

Assignment Text

As part of the pre-processing for the model fitting, we will split the data into training and test subsamples. For this exercise, we use a 75/25 split although other choices are certainly suitable. Some analysts prefer to do this splitting before looking at the data. Another approach, adopted here, is that the final report typically contains summary statistcs of the entire data set and so it makes sense to do so when examining summary statistics.

We start by fitting a benchmark model. It is common to use all available explanatory variables with the outcome on the original scale and so we use this as our benchmark model. This exercise shows that when you plot() a fitted linear regression model in R, the result provides four graphs that you have seen before. These can be useful for identifying an appropriate model.

Instructions

  • Randomly split the data into a training and a testing data sets. Use 75% for the training, 25% for the testing.
  • Fit a full model using expendop as the outcome and all explanatory variables. Summarize the results of this model fitting.
  • You can plot() the fitted model to view several diagnostic plots. These plots provide evidence that expenditures may not be the best scale for linear regression.
  • Fit a full model using logexpend as the outcome and all explanatory variables and summarize the fit. Use the plot() function for evidence that this variable is more suited for linear regression methods than expenditures on the original scale.

Hint. A plot of a regression object such as plot(mlr) provides four diagnostic plots. These can be organized as a 2 by 2 array using par(mfrow = c(2, 2)).

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiNtZXBzIDwtIHJlYWQuY3N2KFwiQ1NWRGF0YVxcXFxIZWFsdGhNZXBzLmNzdlwiLCBoZWFkZXIgPSBUUlVFKVxubWVwcyA8LSByZWFkLmNzdihcImh0dHBzOi8vYXNzZXRzLmRhdGFjYW1wLmNvbS9wcm9kdWN0aW9uL3JlcG9zaXRvcmllcy8yNjEwL2RhdGFzZXRzLzdiN2RhYjZkMGM1MjhlNGNkMmY4ZDBlMGZjNzgyNGEyNTQ0MjliZjgvSGVhbHRoTWVwcy5jc3ZcIiwgaGVhZGVyID0gVFJVRSlcbm1lcHMkbG9nZXhwZW5kIDwtIGxvZyhtZXBzJGV4cGVuZG9wKSIsInNhbXBsZSI6IiMgUmFuZG9tbHkgc3BsaXQgdGhlIGRhdGEgaW50byBhIHRyYWluaW5nIGFuZCBhIHRlc3RpbmcgZGF0YSBzZXRzLiBVc2UgNzVcXCUgZm9yIHRoZSB0cmFpbmluZywgMjVcXCUgZm9yIHRoZSB0ZXN0aW5nLlxubiA8LSBucm93KG1lcHMpXG5zZXQuc2VlZCgxMjM0NylcbnNodWZmbGVkX21lcHMgPC0gbWVwc1tzYW1wbGUobiksIF1cbnRyYWluX2luZGljZXMgPC0gMTpyb3VuZCgwLjc1ICogbilcbnRyYWluX21lcHMgICAgPC0gc2h1ZmZsZWRfbWVwc1t0cmFpbl9pbmRpY2VzLCBdXG50ZXN0X2luZGljZXMgIDwtIChyb3VuZCgwLjI1ICogbikgKyAxKTpuXG50ZXN0X21lcHMgICAgIDwtIHNodWZmbGVkX21lcHNbdGVzdF9pbmRpY2VzLCBdXG5cbiMgRml0IGEgZnVsbCBtb2RlbCB1c2luZyBgZXhwZW5kb3BgIGFzIHRoZSBvdXRjb21lIGFuZCBhbGwgZXhwbGFuYXRvcnkgdmFyaWFibGVzLiBTdW1tYXJpemUgdGhlIHJlc3VsdHMgb2YgdGhpcyBtb2RlbCBmaXR0aW5nLlxubWVwc19tbHIxIDwtIGxtKF9fXyB+IGdlbmRlciArIGFnZSArIHJhY2UgKyByZWdpb24gKyBlZHVjICsgcGhzdGF0ICsgbXBvb3IgKyBhbnlsaW1pdCArIGluY29tZSArIGluc3VyZSArIHVzYyArIHVuZW1wbG95ICsgbWFuYWdlZGNhcmUsIGRhdGEgPSBfX18pXG5zdW1tYXJ5KG1lcHNfbWxyMSlcblxuIyBQcm92aWRlIGRpYWdub3N0aWMgcGxvdHMgb2YgdGhlIGZpdHRlZCBtb2RlbC4gXG5wYXIobWZyb3cgPSBjKDIsIDIpKVxucGxvdChfX18pXG5cbiMgRml0IGEgZnVsbCBtb2RlbCB1c2luZyBgbG9nZXhwZW5kYCBhcyB0aGUgb3V0Y29tZSBhbmQgYWxsIGV4cGxhbmF0b3J5IHZhcmlhYmxlcy4gU3VtbWFyaXplIHRoZSBmaXQgYW5kIGV4YW1pbmUgZGlhZ25vc3RpYyBwbG90cyBvZiB0aGUgZml0dGVkIG1vZGVsLiBcbm1lcHNfbWxyMiA8LSBsbShfX18gfiBnZW5kZXIgKyBhZ2UgKyByYWNlICsgcmVnaW9uICsgZWR1YyArIHBoc3RhdCArIG1wb29yICsgYW55bGltaXQgKyBpbmNvbWUgKyBpbnN1cmUgKyB1c2MgKyB1bmVtcGxveSArIG1hbmFnZWRjYXJlLCBkYXRhID0gX19fKVxuc3VtbWFyeShtZXBzX21scjIpXG5wbG90KG1lcHNfbWxyMikiLCJzb2x1dGlvbiI6IiMgU3BsaXQgdGhlIHNhbXBsZSBpbnRvIGEgYHRyYWluaW5nYCBhbmQgYHRlc3RgIGRhdGFcbm4gPC0gbnJvdyhtZXBzKVxuc2V0LnNlZWQoMTIzNDcpXG5zaHVmZmxlZF9tZXBzIDwtIG1lcHNbc2FtcGxlKG4pLCBdXG50cmFpbl9pbmRpY2VzIDwtIDE6cm91bmQoMC43NSAqIG4pXG50cmFpbl9tZXBzICAgIDwtIHNodWZmbGVkX21lcHNbdHJhaW5faW5kaWNlcywgXVxudGVzdF9pbmRpY2VzICA8LSAocm91bmQoMC4yNSAqIG4pICsgMSk6blxudGVzdF9tZXBzICAgICA8LSBzaHVmZmxlZF9tZXBzW3Rlc3RfaW5kaWNlcywgXVxuXG5tZXBzX21scjEgPC0gbG0oZXhwZW5kb3AgfiBnZW5kZXIgKyBhZ2UgKyByYWNlICsgcmVnaW9uICsgZWR1YyArIHBoc3RhdCArIG1wb29yICsgYW55bGltaXQgKyBpbmNvbWUgKyBpbnN1cmUgKyB1c2MgKyB1bmVtcGxveSArIG1hbmFnZWRjYXJlLCBkYXRhID0gdHJhaW5fbWVwcylcbnN1bW1hcnkobWVwc19tbHIxKVxucGFyKG1mcm93ID0gYygyLCAyKSlcbnBsb3QoeD1tZXBzX21scjEpXG5cbm1lcHNfbWxyMiA8LSBsbShsb2dleHBlbmQgfiBnZW5kZXIgKyBhZ2UgKyByYWNlICsgcmVnaW9uICsgZWR1YyArIHBoc3RhdCArIG1wb29yICsgYW55bGltaXQgKyBpbmNvbWUgKyBpbnN1cmUgKyB1c2MgKyB1bmVtcGxveSArIG1hbmFnZWRjYXJlLCBkYXRhID0gdHJhaW5fbWVwcylcbnN1bW1hcnkobWVwc19tbHIyKVxucGxvdChtZXBzX21scjIpIiwic2N0IjoiIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtZXBzX21scjFcIiwgdW5kZWZpbmVkX21zZz1cIlVzZSBgbG1gIHRvIGNyZWF0ZSBhIGxpbmVhciBtb2RlbCBuYW1lZCBgbWVwc19tbHIxYFwiLikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gY3JlYXRlIHRoaXMgbW9kZWwgc28gdGhhdCBgZXhwZW5kb3BgIGlzIGJhc2VkIG9uIGBnZW5kZXJgLCBgYWdlYCwgYHJhY2VgLCBgcmVnaW9uYCwgYGVkdWNgLCBgcGhzdGF0YCwgYG1wb29yYCwgYGFueWxpbWl0YCwgYGluY29tZWAsIGBpbnN1cmVgLCBgdXNjYCwgYHVuZW1wbG95YCwgYW5kIGBtYW5hZ2VkY2FyZWAgZnJvbSB0aGUgZGF0YWZyYW1lIGBtZXBzYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJzdW1tYXJ5XCIsaW5kZXg9MSxub3RfY2FsbGVkX21zZz1cIlVzZSB0aGUgYHN1bW1hcnlgIGZ1bmN0aW9uIHRvIHZpZXcgYSBzdW1tYXJ5IG9mIG91ciBsaW5lYXIgbW9kZWwuIFwiKSAlPiUgY2hlY2tfYXJnKC4sIFwib2JqZWN0XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIGEgc3VtbWFyeSBvZiBgbWVwc19tbHIxYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwYXJcIiwgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBwYXJgIGZ1bmN0aW9uIHRvIGFsdGVyIHRoZSBwbG90dGluZyBkZXZpY2UgdG8gc2hvdyBhIDIgYnkgMiBncmlkIG9mIGdyYXBocy4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJtZnJvd1wiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIlRvIGNyZWF0ZSBhIDIgYnkgMiBncmlkIG9mIGdyYXBocywgc2V0IGBtZnJvd2AgZXF1YWwgdG8gYGMoMiwyKWAuIFwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInBsb3RcIixpbmRleD0xLCBub3RfY2FsbGVkX21zZz1cIlVzZSBgcGxvdGAgdG8gY3JlYXRlIHRoZSA0IHBsb3RzIGFzc29jaWF0ZWQgd2l0aCBhIGxpbmVhciBtb2RlbC4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJ4XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiSWYgeW91IHNldCB0aGUgcGFyYW1ldGVyIGB4YCB0byBiZSBlcXVhbCB0byBhIGxpbmVhciBtb2RlbCwgaW4gdGhpcyBjYXNlIGBtZXBzX21scjFgLCB5b3Ugd2lsbCBnZXQgNCB1c2VmdWwgZ3JhcGhzIVwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtZXBzX21scjJcIiwgdW5kZWZpbmVkX21zZz1cIlVzZSBgbG1gIHRvIGNyZWF0ZSBhIGxpbmVhciBtb2RlbCBuYW1lZCBgbWVwc19tbHIyYC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gY3JlYXRlIHRoaXMgbW9kZWwgc28gdGhhdCBgbG9nZXhwZW5kYCBpcyBiYXNlZCBvbiBgZ2VuZGVyYCwgYGFnZWAsIGByYWNlYCwgYHJlZ2lvbmAsIGBlZHVjYCwgYHBoc3RhdGAsIGBtcG9vcmAsIGBhbnlsaW1pdGAsIGBpbmNvbWVgLCBgaW5zdXJlYCwgYHVzY2AsIGB1bmVtcGxveWAsIGFuZCBgbWFuYWdlZGNhcmVgIGZyb20gdGhlIGRhdGFmcmFtZSBgbWVwc2AuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwic3VtbWFyeVwiLGluZGV4PTIsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgc3VtbWFyeWAgZnVuY3Rpb24gdG8gdmlldyBhIHN1bW1hcnkgb2Ygb3VyIG5ldyBsaW5lYXIgbW9kZWwsIGBtZXBzX21scjJgLlwiKSAlPiUgY2hlY2tfYXJnKC4sIFwib2JqZWN0XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIGEgc3VtbWFyeSBvZiBgbWVwc19tbHIyYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwbG90XCIsaW5kZXg9Miwgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBwbG90YCBmdW5jdGlvbiB0byBjcmVhdGUgdGhlIDQgcGxvdHMgYXNzb2NpYXRlZCB3aXRoIGEgbGluZWFyIG1vZGVsLiBcIikgJT4lIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJJZiB5b3Ugc2V0IHRoZSBwYXJhbWV0ZXIgYHhgIHRvIGJlIGVxdWFsIHRvIGEgbGluZWFyIG1vZGVsLCBpbiB0aGlzIGNhc2UgYG1lcHNfbWxyMmAsIHlvdSB3aWxsIGdldCA0IHVzZWZ1bCBncmFwaHMhXCIpXG5zdWNjZXNzX21zZyhcIkV4Y2VsbGVudCEgWW91IG1heSBoYXZlIGNvbXBhcmVkIHRoZSBmb3VyIGRpYWdub3N0aWMgZ3JhcGhzIGZyb20gdGhlIE1MUiBtb2RlbCBmaXQgb2YgJ2V4cGVuZCcgdG8gdGhvc2UgY3JlYXRlZCB1c2luZyB0aGUgc2FtZSBwcm9jZWR1cmUgYnV0IHdpdGggbG9nYXJpdGhtaWMgZXhwZW5kaXR1cmVzIGFzIHRoZSBvdXRjb21lLiBUaGlzIHByb3ZpZGVzIGFub3RoZXIgcGllY2Ugb2YgZXZpZGVuY2UgdGhhdCBsb2cgZXhwZW5kaXR1cmVzIGFyZSBtb3JlIHN1aXRhYmxlIGZvciByZWdyZXNzaW9uIG1vZGVsaW5nLiBVc2luZyBsb2dhcml0aG1pYyBvdXRjb21lcyBpcyBhIGNvbW1vbiBmZWF0dXJlIG9mIGFjdHVhcmlhbCBhcHBsaWNhdGlvbnMgYnV0IGNhbiBiZSBkaWZmaWN1bHQgdG8gZGlhZ25vc2UgYW5kIGludGVycHJldCB3aXRob3V0IHByYWN0aWNlLlwiKSIsImhpbnQiOiJBIDxjb2RlPnBsb3Q8L2NvZGU+IG9mIGEgcmVncmVzc2lvbiBvYmplY3Qgc3VjaCBhcyBwbG90KG1scikgcHJvdmlkZXMgZm91ciBkaWFnbm9zdGljIHBsb3RzLiBUaGVzZSBjYW4gYmUgb3JnYW5pemVkIGFzIGEgMiBieSAyIGFycmF5IHVzaW5nIDxjb2RlPnBhcihtZnJvdyA9IGMoMiwgMikpPC9jb2RlPi4ifQ==

5.1.4 Exercise. Variable selection

Assignment Text

Modeling building can be approached using a “ground-up” strategy, where the analyst introduces a variable, examines residuls from a regression fit, and then seeks to understand the relationship between these residuals and other available variables so that these variables might be added to the model.

Another approach is a “top-down” strategy where all available variables are entered into a model and unnecessary variables are pruned from the model. Both approaches are helpful when using data to specify models. This exercise illustrates the latter approach, using the [step()] function to help narrow our search for the best fitting model.

Instructions

From our prior work, the training dataframe train_meps has already been loaded in. A multiple linear regression model fit object meps_mlr2 is available that summarizes a fit of logexpend as the outcome variable using all 13 explanatory variables.

  • Use the step() function function to drop unnecessary variables from the full fitted model summarized in the object meps_mlr2 and summarize this recommended model.
  • As an alternative, use the explanatory variables in the recommended model and add the varibles phstat. Summarize the fit and note that statistical significance of the new variable.
  • You have been reminded by your boss that use of the variable gender is unsuitable for actuarial pricing purposes. As an another alternative, drop gender from the recommended model (still keeping phstat). Note the statistical significance of the variable uscwith this fitted model.
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiNtZXBzIDwtIHJlYWQuY3N2KFwiQ1NWRGF0YVxcXFxIZWFsdGhNZXBzLmNzdlwiLCBoZWFkZXIgPSBUUlVFKVxubWVwcyA8LSByZWFkLmNzdihcImh0dHBzOi8vYXNzZXRzLmRhdGFjYW1wLmNvbS9wcm9kdWN0aW9uL3JlcG9zaXRvcmllcy8yNjEwL2RhdGFzZXRzLzdiN2RhYjZkMGM1MjhlNGNkMmY4ZDBlMGZjNzgyNGEyNTQ0MjliZjgvSGVhbHRoTWVwcy5jc3ZcIiwgaGVhZGVyID0gVFJVRSlcbm1lcHMkbG9nZXhwZW5kIDwtIGxvZyhtZXBzJGV4cGVuZG9wKVxuIyBTcGxpdCB0aGUgc2FtcGxlIGludG8gYSBgdHJhaW5pbmdgIGFuZCBgdGVzdGAgZGF0YVxubiA8LSBucm93KG1lcHMpXG5zZXQuc2VlZCgxMjM0NylcbnNodWZmbGVkX21lcHMgPC0gbWVwc1tzYW1wbGUobiksIF1cbnRyYWluX2luZGljZXMgPC0gMTpyb3VuZCgwLjc1ICogbilcbnRyYWluX21lcHMgICAgPC0gc2h1ZmZsZWRfbWVwc1t0cmFpbl9pbmRpY2VzLCBdXG50ZXN0X2luZGljZXMgIDwtIChyb3VuZCgwLjI1ICogbikgKyAxKTpuXG50ZXN0X21lcHMgICAgIDwtIHNodWZmbGVkX21lcHNbdGVzdF9pbmRpY2VzLCBdIiwic2FtcGxlIjoibWVwc19tbHIyIDwtIGxtKGxvZ2V4cGVuZCB+IGdlbmRlciArIGFnZSArIHJhY2UgKyByZWdpb24gKyBlZHVjICsgcGhzdGF0ICsgbXBvb3IgKyBhbnlsaW1pdCArIGluY29tZSArIGluc3VyZSArIHVzYyArIHVuZW1wbG95ICsgbWFuYWdlZGNhcmUsIGRhdGEgPSB0cmFpbl9tZXBzKVxuIyBVc2UgdGhlIHN0ZXAoKSB0byBkcm9wIHVubmVjZXNzYXJ5IHZhcmlhYmxlcyBmcm9tIHRoZSBmdWxsIGZpdHRlZCBtb2RlbCBzdW1tYXJpemVkIGluIHRoZSBvYmplY3QgYG1lcHNfbWxyMmAgYW5kIHN1bW1hcml6ZSB0aGlzIHJlY29tbWVuZGVkIG1vZGVsLlxubW9kZWxfc3RlcHdpc2UgPC0gc3RlcChfX18sIGRpcmVjdGlvbj0gXCJib3RoXCIsIGsgPSBsb2cobnJvdyh0cmFpbl9tZXBzKSksIHRyYWNlID0gMClcbnN1bW1hcnkobW9kZWxfc3RlcHdpc2UpXG5cbiMgQXMgYW4gYWx0ZXJuYXRpdmUsIHVzZSB0aGUgZXhwbGFuYXRvcnkgdmFyaWFibGVzIGluIHRoZSByZWNvbW1lbmRlZCBtb2RlbCBhbmQgYWRkIHRoZSB2YXJpYmxlcyBgbXBvb3JgLiBTdW1tYXJpemUgdGhlIGZpdCAgYW5kIG5vdGUgdGhhdCBzdGF0aXN0aWNhbCBzaWduaWZpY2FuY2Ugb2YgdGhlIG5ldyB2YXJpYWJsZS5cbm1lcHNfbWxyNCA8LSBsbShfX18gfiBnZW5kZXIgKyBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCArIGluc3VyZSAgKyBfX18sIGRhdGEgPSB0cmFpbl9tZXBzKVxuc3VtbWFyeShtZXBzX21scjQpXG5cbiMgWW91IGhhdmUgYmVlbiByZW1pbmRlZCBieSB5b3VyIGJvc3MgdGhhdCB1c2Ugb2YgdGhlIHZhcmlhYmxlIGBnZW5kZXJgIGlzIHVuc3VpdGFibGUgZm9yIGFjdHVhcmlhbCBwcmljaW5nIHB1cnBvc2VzLiBBcyBhbiBhbm90aGVyIGFsdGVybmF0aXZlLCBkcm9wIGBnZW5kZXJgIGZyb20gdGhlIHJlY29tbWVuZGVkIG1vZGVsIChzdGlsbCBrZWVwaW5nIGBtcG9vcmApLiBOb3RlIHRoZSBzdGF0aXN0aWNhbCBzaWduaWZpY2FuY2Ugb2YgdGhlIHZhcmlhYmxlIGB1c2Ngd2l0aCB0aGlzIGZpdHRlZCBtb2RlbC5cbm1lcHNfbWxyNSA8LSBsbShsb2dleHBlbmQgfiBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCArIGluc3VyZSAgKyBfX18sIGRhdGEgPSB0cmFpbl9tZXBzKVxuc3VtbWFyeShfX18pIiwic29sdXRpb24iOiJtZXBzX21scjIgPC0gbG0obG9nZXhwZW5kIH4gZ2VuZGVyICsgYWdlICsgcmFjZSArIHJlZ2lvbiArIGVkdWMgKyBwaHN0YXQgKyBtcG9vciArIGFueWxpbWl0ICsgaW5jb21lICsgaW5zdXJlICsgdXNjICsgdW5lbXBsb3kgKyBtYW5hZ2VkY2FyZSwgZGF0YSA9IHRyYWluX21lcHMpXG4jbGlicmFyeShSY21kcilcbiN0ZW1wIDwtIHN0ZXB3aXNlKG1lcHNfbWxyMiwgZGlyZWN0aW9uID0gJ2JhY2t3YXJkL2ZvcndhcmQnKVxubW9kZWxfc3RlcHdpc2UgPC0gc3RlcChtZXBzX21scjIsIGRpcmVjdGlvbj0gXCJib3RoXCIsIGsgPSBsb2cobnJvdyh0cmFpbl9tZXBzKSksIHRyYWNlID0gMCkgXG5zdW1tYXJ5KG1vZGVsX3N0ZXB3aXNlKVxubWVwc19tbHIzIDwtIGxtKGxvZ2V4cGVuZCB+IGdlbmRlciArIGFnZSArIHBoc3RhdCArIGFueWxpbWl0ICsgaW5zdXJlICwgZGF0YSA9IHRyYWluX21lcHMpXG5zdW1tYXJ5KG1lcHNfbWxyMylcbm1lcHNfbWxyNCA8LSBsbShsb2dleHBlbmQgfiBnZW5kZXIgKyBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCArIGluc3VyZSAgKyBtcG9vciwgZGF0YSA9IHRyYWluX21lcHMpXG5zdW1tYXJ5KG1lcHNfbWxyNClcbm1lcHNfbWxyNSA8LSBsbShsb2dleHBlbmQgfiBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCArIGluc3VyZSAgKyBtcG9vciwgZGF0YSA9IHRyYWluX21lcHMpXG5zdW1tYXJ5KG1lcHNfbWxyNSlcblxuIyBwYXIobWZyb3cgPSBjKDIsIDIpKVxuIyBwbG90KG1lcHNfbWxyMylcbiMgXG4jIG1lcHNfbWxyNCA8LSBsbShsb2dleHBlbmQgfiBnZW5kZXIgKyBhZ2UgKyBtcG9vciArIGFueWxpbWl0ICsgaW5zdXJlICsgdXNjICArIHBoc3RhdCwgZGF0YSA9IHRyYWluX21lcHMpXG4jIHN1bW1hcnkobWVwc19tbHI0KVxuIyBcbiMgXG4jIG1lcHNfbWxyNSA8LSBsbShsb2dleHBlbmQgfiBhZ2UgICsgYW55bGltaXQgKyBtcG9vciArIGluc3VyZSAgKyB1c2MgICsgcGhzdGF0LCBkYXRhID0gdHJhaW5fbWVwcylcbiMgc3VtbWFyeShtZXBzX21scjUpXG4jIGFub3ZhKG1lcHNfbWxyNCwgbWVwc19tbHI1KVxuIyBcbiMgI2JveHBsb3QodHJhaW5fbWVwcyRsb2dleHBlbmQgfiB0cmFpbl9tZXBzJHBoc3RhdCp0cmFpbl9tZXBzJHVzYykiLCJzY3QiOiIjIGV4KCkgJT4lIGNoZWNrX29iamVjdChcIm1lcHNfbWxyMlwiLCB1bmRlZmluZWRfbXNnPVwiVXNlIGBsbWAgdG8gY3JlYXRlIGEgbGluZWFyIG1vZGVsIG5hbWVkIGBtZXBzX21scjJgLlwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBjcmVhdGUgdGhlIGxpbmVhciBtb2RlbCBzbyB0aGF0IGBsb2dleHBlbmRgIGlzIGJhc2VkIG9uIGBnZW5kZXJgLCBgYWdlYCwgYHJhY2VgLCBgcmVnaW9uYCwgYGVkdWNgLCBgcGhzdGF0YCwgYG1wb29yYCwgYGFueWxpbWl0YCwgYGluY29tZWAsIGBpbnN1cmVgLCBgdXNjYCwgYHVuZW1wbG95YCwgYW5kIGBtYW5hZ2VkY2FyZWAgZnJvbSB0aGUgZGF0YSBzZXQgYHRyYWluX21lcHNgLlwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtb2RlbF9zdGVwd2lzZVwiLCB1bmRlZmluZWRfbXNnPVwiVXRpbGl6ZSB0aGUgYHN0ZXBgIGZ1bmN0aW9uIHRvIGNyZWF0ZSBhIHNpbXBsaWZpZWQgbW9kZWwgbmFtZWQgYG1vZGVsX3N0ZXB3aXNlYC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gcnVuIGBzdGVwYCBvbiBgbWVwc19tbHIyYCBpbiBib3RoIGRpcmVjdGlvbnMsIGFuZCB3aXRoIGsgc2V0IHRvIGJlIHRoZSBCSUMgZXN0aW1hdGUuIFwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInN1bW1hcnlcIixpbmRleD0xLCBub3RfY2FsbGVkX21zZz1cIlVzZSBgc3VtbWFyeWAgdG8gY3JlYXRlIGFuZCB2aWV3IGEgc3VtbWFyeSBvZiBgbW9kZWxfc3RlcHdpc2VgLlwiKSAlPiUgY2hlY2tfYXJnKC4sIFwib2JqZWN0XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIGEgc3VtbWFyeSBvZiBgbW9kZWxfc3RlcHdpc2VgLlwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtZXBzX21scjNcIiwgdW5kZWZpbmVkX21zZz1cIkNyZWF0ZSBhIG5ldyBsaW5lYXIgbW9kZWwgb2ZmIGEgc21hbGwgbnVtYmVyIG9mIGV4cGxhbmF0b3J5IHZhcmlhYmxlcyBuYW1lZCBgbWVwc19tbHIzYC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJXaGVuIHlvdSBjcmVhdGUgeW91ciBsaW5lYXIgbW9kZWwsIG1ha2Ugc3VyZSB0byBtb2RlbCBgbG9nZXhwZW5kYCBiYXNlZCBvbiBgZ2VuZGVyYCwgYGFnZWAsIGBwaHN0YXRgLCBgYW55bGltaXRgLCBhbmQgYGluc3VyZWAgZnJvbSB0aGUgZGF0YWZyYW1lIGB0cmFpbl9tZXBzYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJzdW1tYXJ5XCIsaW5kZXg9Miwgbm90X2NhbGxlZF9tc2c9XCJVc2UgdGhlIGBzdW1tYXJ5YCBmdW5jdGlvbiB0byBjcmVhdGUgYW5kIHZpZXcgYSBzdW1tYXJ5IG9mIGBtZXBzX21scjNgLlwiKSAlPiUgY2hlY2tfYXJnKC4sIFwib2JqZWN0XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIGEgc3VtbWFyeSBvZiBgbWVwc19tbHIzYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfb2JqZWN0KFwibWVwc19tbHI0XCIsIHVuZGVmaW5lZF9tc2c9XCJDcmVhdGUgYSBuZXcgbGluZWFyIG1vZGVsIG9mZiBhIHNtYWxsIG51bWJlciBvZiBleHBsYW5hdG9yeSB2YXJpYWJsZXMgbmFtZWQgYG1lcHNfbWxyNGAuXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiV2hlbiB5b3UgY3JlYXRlIHlvdXIgbGluZWFyIG1vZGVsLCBtYWtlIHN1cmUgdG8gbW9kZWwgYGxvZ2V4cGVuZGAgYmFzZWQgb24gYGdlbmRlcmAsIGBhZ2VgLCBgcGhzdGF0YCwgYGFueWxpbWl0YCwgYGluc3VyZWAsIGFuZCBgbXBvb3JgIGZyb20gdGhlIGRhdGFmcmFtZSBgdHJhaW5fbWVwc2AuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwic3VtbWFyeVwiLGluZGV4PTMsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgc3VtbWFyeWAgZnVuY3Rpb24gdG8gY3JlYXRlIGFuZCB2aWV3IGEgc3VtbWFyeSBvZiBgbWVwc19tbHI0YC5cIikgJT4lIGNoZWNrX2FyZyguLCBcIm9iamVjdFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgd2Ugd291bGQgbGlrZSBhIHN1bW1hcnkgb2YgYG1lcHNfbWxyNGAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX29iamVjdChcIm1lcHNfbWxyNVwiLCB1bmRlZmluZWRfbXNnPVwiQ3JlYXRlIGEgbmV3IGxpbmVhciBtb2RlbCBvZmYgYSBzbWFsbCBudW1iZXIgb2YgZXhwbGFuYXRvcnkgdmFyaWFibGVzIG5hbWVkIGBtZXBzX21scjVgLlwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIldoZW4geW91IGNyZWF0ZSB5b3VyIGxpbmVhciBtb2RlbCwgbWFrZSBzdXJlIHRvIG1vZGVsIGBsb2dleHBlbmRgIGJhc2VkIG9uIGBhZ2VgLCBgcGhzdGF0YCwgYGFueWxpbWl0YCwgYW5kIGBpbnN1cmVgIGZyb20gdGhlIGRhdGFmcmFtZSBgdHJhaW5fbWVwc2AuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwic3VtbWFyeVwiLGluZGV4PTQsIG5vdF9jYWxsZWRfbXNnPVwiVXNlIHRoZSBgc3VtbWFyeWAgZnVuY3Rpb24gdG8gY3JlYXRlIGFuZCB2aWV3IGEgc3VtbWFyeSBvZiBgbWVwc19tbHI1YC5cIikgJT4lIGNoZWNrX2FyZyguLCBcIm9iamVjdFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgd2Ugd291bGQgbGlrZSBhIHN1bW1hcnkgb2YgYG1lcHNfbWxyNWAuXCIpXG5zdWNjZXNzX21zZyhcIkV4Y2VsbGVudCEgU29tZXRpbWVzIHZhcmlhYmxlcyBtYXkgaGF2ZSBnb29kIHByZWRpY3RpdmUgcG93ZXIgYnV0IGFyZSB1bmFjY2VwdGFibGUgZm9yIHBvbGljeSBwdXJwb3NlcyAtIGluIGluc3VyYW5jZSwgZXRobmljaXR5IGFuZCBzb21ldGltZXMgc2V4IGFyZSBnb29kIGV4YW1wbGVzLiBUaGlzIGltcGxpZXMgdGhhdCBtb2RlbCBpbnRlcnByZXRhdGlvbiBjYW4gYmUganVzdCBhcyBpbXBvcnRhbnQgYXMgdGhlIGFiaWxpdHkgdG8gcHJlZGljdC5cIikiLCJoaW50IjoiU3RhcnRpbmcgd2l0aCBhIG1vZGVsIHRoYXQgY29uc2lzdHMgb2YgYWxsIHlvdXIgZXhwbGFuYXRvcnkgdmFyaWFibGVzIGFuZCB1c2luZyA8Y29kZT5zdGVwKCk8L2NvZGU+IGlzIGEgZ29vZCB3YXkgdG8gY3JlYXRlIGEgbW9kZWwgdGhhdCBmaXRzIHRoZSBkYXRhIHdlbGwuIE9uIHRoZSBvdGhlciBoYW5kLCB0aGlzIGlzIHNpbXBseSBzdGF0aXN0aWNzLCB3aGljaCBpcyB1bmFibGUgdG8gY2FwdHVyZSAmbGRxdW87cmVhbCZyZHF1bzsgZXhwbGFuYXRpb25zLCBvciBmb2xsb3cgcmVndWxhdG9yeSBydWxlcy4ifQ==

5.1.5 Exercise. Model comparisons using cross-validation

Assignment Text

To compare alternative models, you decide to utilize cross-validation. For this exercise, you split the training sample into six subsamples of approximately equal size.

In the sample code, the cross-validation procedure has been summarized into a function that you can call. The input to the function is a list of variables that you select as your model explanatory variables. With this function, you can readily test several candidate models.

Instructions

  • Run the cross validation (crossvalfct) function using the explanatory variables suggested by the stepwise function.
  • Run the function again but adding the mpoor variable
  • Run the function again but omitting the gender variable

Note which model is suggested by the cross validation function.

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiNtZXBzIDwtIHJlYWQuY3N2KFwiQ1NWRGF0YVxcXFxIZWFsdGhNZXBzLmNzdlwiLCBoZWFkZXIgPSBUUlVFKVxubWVwcyA8LSByZWFkLmNzdihcImh0dHBzOi8vYXNzZXRzLmRhdGFjYW1wLmNvbS9wcm9kdWN0aW9uL3JlcG9zaXRvcmllcy8yNjEwL2RhdGFzZXRzLzdiN2RhYjZkMGM1MjhlNGNkMmY4ZDBlMGZjNzgyNGEyNTQ0MjliZjgvSGVhbHRoTWVwcy5jc3ZcIiwgaGVhZGVyID0gVFJVRSlcbm1lcHMkbG9nZXhwZW5kIDwtIGxvZyhtZXBzJGV4cGVuZG9wKVxuIyBTcGxpdCB0aGUgc2FtcGxlIGludG8gYSBgdHJhaW5pbmdgIGFuZCBgdGVzdGAgZGF0YVxubiA8LSBucm93KG1lcHMpXG5zZXQuc2VlZCgxMjM0NylcbnNodWZmbGVkX21lcHMgPC0gbWVwc1tzYW1wbGUobiksIF1cbnRyYWluX2luZGljZXMgPC0gMTpyb3VuZCgwLjc1ICogbilcbnRyYWluX21lcHMgICAgPC0gc2h1ZmZsZWRfbWVwc1t0cmFpbl9pbmRpY2VzLCBdXG50ZXN0X2luZGljZXMgIDwtIChyb3VuZCgwLjI1ICogbikgKyAxKTpuXG50ZXN0X21lcHMgICAgIDwtIHNodWZmbGVkX21lcHNbdGVzdF9pbmRpY2VzLCBdXG5cbiMjIENyb3NzIC0gVmFsaWRhdGlvblxuXG5jcm9zc3ZhbGZjdCA8LSBmdW5jdGlvbihleHBsdmFycyl7XG4gIGN2ZGF0YSAgIDwtIHRyYWluX21lcHNbLCBjKFwibG9nZXhwZW5kXCIsIGV4cGx2YXJzKV1cbiAgY3Jvc3N2YWwgPC0gMFxuICBmb3IgKGkgaW4gMTo2KSB7XG4gICAgaW5kaWNlcyA8LSAoKChpLTEpICogcm91bmQoKDEvNikqbnJvdyhjdmRhdGEpKSkgKyAxKTooKGkqcm91bmQoKDEvNikgKiBucm93KGN2ZGF0YSkpKSlcbiAgICAjIEV4Y2x1ZGUgdGhlbSBmcm9tIHRoZSB0cmFpbiBzZXRcbiAgICB0cmFpbl9tbHIgPC0gbG0obG9nZXhwZW5kIH4gLiwgZGF0YSA9IGN2ZGF0YVstaW5kaWNlcyxdKVxuICAgICMgSW5jbHVkZSB0aGVtIGluIHRoZSB0ZXN0IHNldFxuICAgIHRlc3QgIDwtIGRhdGEuZnJhbWUoY3ZkYXRhW2luZGljZXMsIGV4cGx2YXJzXSlcbiAgICBuYW1lcyh0ZXN0KSAgPC0gZXhwbHZhcnNcbiAgICBwcmVkaWN0X3Rlc3QgPC0gZXhwKHByZWRpY3QodHJhaW5fbWxyLCB0ZXN0KSlcbiAgICAjIENvbXBhcmUgcHJlZGljdGVkIHRvIGhlbGQtb3V0IGFuZCBzdW1tYXJpemVcbiAgICBwcmVkaWN0X2VyciAgPC0gZXhwKGN2ZGF0YVtpbmRpY2VzLCBcImxvZ2V4cGVuZFwiXSkgLSBwcmVkaWN0X3Rlc3RcbiAgICBjcm9zc3ZhbCA8LSBjcm9zc3ZhbCArIHN1bShhYnMocHJlZGljdF9lcnIpKVxuICB9XG4gIGNyb3NzdmFsLzEwMDAwMDBcbn0iLCJzYW1wbGUiOiIjIFJ1biB0aGUgY3Jvc3MgdmFsaWRhdGlvbiAoYGNyb3NzdmFsZmN0YCkgZnVuY3Rpb24gdXNpbmcgdGhlIGV4cGxhbmF0b3J5IHZhcmlhYmxlcyBzdWdnZXN0ZWQgYnkgdGhlIHN0ZXB3aXNlIGZ1bmN0aW9uLlxuZXhwbHZhcnMuMSA8LSBjKFwiZ2VuZGVyXCIsIFwiYWdlXCIsIFwicGhzdGF0XCIsIFwiYW55bGltaXRcIiwgXCJpbnN1cmVcIilcbmNyb3NzdmFsZmN0KGV4cGx2YXJzKVxuXG4jIFJ1biB0aGUgZnVuY3Rpb24gYWdhaW4gYnV0IGFkZGluZyB0aGUgYG1wb29yYCB2YXJpYWJsZVxuZXhwbHZhcnMuMiA8LSBjKF9fXylcbmNyb3NzdmFsZmN0KGV4cGx2YXJzLjIpXG5cbiMgUnVuIHRoZSBmdW5jdGlvbiBhZ2FpbiBidXQgb21pdHRpbmcgdGhlIGBnZW5kZXJgIHZhcmlhYmxlXG5leHBsdmFycy4zIDwtIGMoIF9fXylcbmNyb3NzdmFsZmN0KGV4cGx2YXJzLjMpIiwic29sdXRpb24iOiJleHBsdmFycy4xIDwtIGMoXCJnZW5kZXJcIiwgXCJhZ2VcIiwgXCJwaHN0YXRcIiwgXCJhbnlsaW1pdFwiLCBcImluc3VyZVwiKVxuY3Jvc3N2YWxmY3QoZXhwbHZhcnMuMSlcbmV4cGx2YXJzLjIgPC0gYyhcImdlbmRlclwiLCBcImFnZVwiLCBcInBoc3RhdFwiLCBcImFueWxpbWl0XCIsIFwiaW5zdXJlXCIsIFwibXBvb3JcIilcbmNyb3NzdmFsZmN0KGV4cGx2YXJzLjIpXG5leHBsdmFycy4zIDwtIGMoIFwiYWdlXCIsIFwicGhzdGF0XCIsIFwiYW55bGltaXRcIiwgXCJpbnN1cmVcIiwgXCJtcG9vclwiKVxuY3Jvc3N2YWxmY3QoZXhwbHZhcnMuMykiLCJzY3QiOiIjIGV4KCkgJT4lIGNoZWNrX29iamVjdChcImV4cGx2YXJzLjFcIiwgdW5kZWZpbmVkX21zZz1cIkNyZWF0ZSBhIHZhcmlhYmxlIG5hbWVkIGBleHBsdmFycy4xYCB0aGF0IGNvbnRhaW5zIHRoZSBmb2xsb3dpbmcgYXMgY2hhcmFjdGVyIHN0cmluZ3M6IGBnZW5kZXJgLCBgYWdlYCwgYHBoc3RhdGAsIGBhbnlsaW1pdGAsIGFuZCBgaW5zdXJlYC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gc3BlY2lmeSB0aGF0IHRoZSBmb2xsb3dpbmcgYXJlIGV4cHJlc3NlZCBhcyBjaGFyYWN0ZXIgc3RyaW5ncywgYW5kIG5vdCBhcyBudW1iZXJzOiA6IGBnZW5kZXJgLCBgYWdlYCwgYHBoc3RhdGAsIGBhbnlsaW1pdGAsIGFuZCBgaW5zdXJlYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJjcm9zc3ZhbGZjdFwiLGluZGV4PTEsIG5vdF9jYWxsZWRfbXNnPVwiVXRpbGl6ZSB0aGUgY3VzdG9tIGJ1aWx0IGZ1bmN0aW9uIGBjcm9zc3ZhbGZjdGAgdG8gZmluZCB0aGUgY3Jvc3MtdmFsaWRhdGlvbiBzdGF0aXN0aWMgZm9yIGBleHBsdmFycy4xYC5cIikgJT4lIGNoZWNrX2FyZyguLCBcImV4cGx2YXJzXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3YW50IHRoZSBjcm9zcy12YWxpZGF0aW9uIHN0YXRpc3RpYyBmb3IgYGV4cGx2YXNyLjFgLlwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJleHBsdmFycy4yXCIsIHVuZGVmaW5lZF9tc2c9XCJDcmVhdGUgYSB2YXJpYWJsZSBuYW1lZCBgZXhwbHZhcnMuMmAgdGhhdCBjb250YWlucyB0aGUgZm9sbG93aW5nIGFzIGNoYXJhY3RlciBzdHJpbmdzOiBgZ2VuZGVyYCwgYGFnZWAsIGBwaHN0YXRgLCBgYW55bGltaXRgLCBgaW5zdXJlYCwgYW5kIGBtcG9vcmAuXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB0aGUgZm9sbG93aW5nIGFyZSBleHByZXNzZWQgYXMgY2hhcmFjdGVyIHN0cmluZ3MsIGFuZCBub3QgYXMgbnVtYmVyczogOiBgZ2VuZGVyYCwgYGFnZWAsIGBwaHN0YXRgLCBgYW55bGltaXRgLCBgaW5zdXJlYCwgYW5kIGBtcG9vcmAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwiY3Jvc3N2YWxmY3RcIixpbmRleD0yLCBub3RfY2FsbGVkX21zZz1cIlV0aWxpemUgdGhlIGN1c3RvbSBidWlsdCBmdW5jdGlvbiBgY3Jvc3N2YWxmY3RgIHRvIGZpbmQgdGhlIGNyb3NzLXZhbGlkYXRpb24gc3RhdGlzdGljIGZvciBgZXhwbHZhcnMuMmAuXCIpICU+JSBjaGVja19hcmcoLiwgXCJleHBsdmFyc1wiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgd2Ugd2FudCB0aGUgY3Jvc3MtdmFsaWRhdGlvbiBzdGF0aXN0aWMgZm9yIGBleHBsdmFzci4yYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfb2JqZWN0KFwiZXhwbHZhcnMuM1wiLCB1bmRlZmluZWRfbXNnPVwiQ3JlYXRlIGEgdmFyaWFibGUgbmFtZWQgYGV4cGx2YXJzLjNgIHRoYXQgY29udGFpbnMgdGhlIGZvbGxvd2luZyBhcyBjaGFyYWN0ZXIgc3RyaW5nczogYGFnZWAsIGBwaHN0YXRgLCBgYW55bGltaXRgLCBgaW5zdXJlYCwgYW5kIGBtcG9vcmAuXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB0aGUgZm9sbG93aW5nIGFyZSBleHByZXNzZWQgYXMgY2hhcmFjdGVyIHN0cmluZ3MsIGFuZCBub3QgYXMgbnVtYmVyczogOmBhZ2VgLCBgcGhzdGF0YCwgYGFueWxpbWl0YCwgYGluc3VyZWAsIGFuZCBgbXBvb3JgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcImNyb3NzdmFsZmN0XCIsaW5kZXg9Mywgbm90X2NhbGxlZF9tc2c9XCJVdGlsaXplIHRoZSBjdXN0b20gYnVpbHQgZnVuY3Rpb24gYGNyb3NzdmFsZmN0YCB0byBmaW5kIHRoZSBjcm9zcy12YWxpZGF0aW9uIHN0YXRpc3RpYyBmb3IgYGV4cGx2YXJzLjNgLlwiKSAlPiUgY2hlY2tfYXJnKC4sIFwiZXhwbHZhcnNcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gc3BlY2lmeSB0aGF0IHdlIHdhbnQgdGhlIGNyb3NzLXZhbGlkYXRpb24gc3RhdGlzdGljIGZvciBgZXhwbHZhc3IuM2AuXCIpXG5zdWNjZXNzX21zZyhcIkV4Y2VsbGVudCEgQ3Jvc3MtdmFsaWRhdGlvbiBoYXMgYmVjb21lIGFuIGVzc2VudGlhbCBwaWVjZSBvZiB0aGUgZGF0YSBhbmFseXN0cyB0b29sa2l0LiBHb29kIHRoYXQgeW91IG5vdyBoYXZlIGFkZGl0aW9uYWwgZXhwZXJpZW5jZSB3aXRoIGl0LlwiKSIsImhpbnQiOiJQZXJmb3JtaW5nIGNyb3NzLXZhbGlkYXRpb24gbWVhc3VyZXMgb24geW91ciBtb2RlbCBhbGxvd3MgeW91IHRvIHNlZSBob3cgd2VsbCBpdCBkb2VzIHdoZW4gaXQgY29tZXMgdG8gcHJlZGljdGluZyAmbGRxdW87b3V0IG9mIHNhbXBsZSZyZHF1bzsgZGF0YSBwb2ludHMuIn0=

5.1.6 Exercise. Out of sample validation

Assignment Text

From our prior work, the training train_meps and test test_meps dataframes have already been loaded in. We think our best model is based on logarithmic expenditures as the outcome and the following explanatory variables:

explvars3 <- c("gender", "age", "phstat", "anylimit", "insure", "mpoor")

We will compare this to a benchmark model that is based on expenditures as the outcome and all 13 explanatory variables

explvars4 <- c(explvars3, "race", "income", "region", "educ", "unemploy", "managedcare", "usc")

The comparisons will be based on expenditures in dollars using the held-out validation sample.

Instructions

  • Use the training sample to fit a linear model with logexpend and explanatory variables listed in explvars3
  • Predict expenditures (not logged) for the test data and summarize the fit using the sum of absolute prediction errors.
  • Use the training sample to fit a benchmark linear model with expendop and explanatory variables listed in explvars4
  • Predict expenditures for the test data and summarize the fit for the benchmark model using the sum of absolute prediction errors.
  • Compare the predictions of the models graphically.
eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiNtZXBzIDwtIHJlYWQuY3N2KFwiQ1NWRGF0YVxcXFxIZWFsdGhNZXBzLmNzdlwiLCBoZWFkZXIgPSBUUlVFKVxubWVwcyA8LSByZWFkLmNzdihcImh0dHBzOi8vYXNzZXRzLmRhdGFjYW1wLmNvbS9wcm9kdWN0aW9uL3JlcG9zaXRvcmllcy8yNjEwL2RhdGFzZXRzLzdiN2RhYjZkMGM1MjhlNGNkMmY4ZDBlMGZjNzgyNGEyNTQ0MjliZjgvSGVhbHRoTWVwcy5jc3ZcIiwgaGVhZGVyID0gVFJVRSlcbm1lcHMkbG9nZXhwZW5kIDwtIGxvZyhtZXBzJGV4cGVuZG9wKVxuIyBTcGxpdCB0aGUgc2FtcGxlIGludG8gYSBgdHJhaW5pbmdgIGFuZCBgdGVzdGAgZGF0YVxubiA8LSBucm93KG1lcHMpXG5zZXQuc2VlZCgxMjM0NylcbnNodWZmbGVkX21lcHMgPC0gbWVwc1tzYW1wbGUobiksIF1cbnRyYWluX2luZGljZXMgPC0gMTpyb3VuZCgwLjc1ICogbilcbnRyYWluX21lcHMgICAgPC0gc2h1ZmZsZWRfbWVwc1t0cmFpbl9pbmRpY2VzLCBdXG50ZXN0X2luZGljZXMgIDwtIChyb3VuZCgwLjI1ICogbikgKyAxKTpuXG50ZXN0X21lcHMgICAgIDwtIHNodWZmbGVkX21lcHNbdGVzdF9pbmRpY2VzLCBdXG5leHBsdmFyczMgPC0gYyhcImdlbmRlclwiLCBcImFnZVwiLCBcInJhY2VcIiwgXCJtcG9vclwiLCBcImFueWxpbWl0XCIsIFwiaW5jb21lXCIsIFwiaW5zdXJlXCIsIFwidXNjXCIpXG5leHBsdmFyczQgPC0gYyhleHBsdmFyczMsIFwicmVnaW9uXCIsIFwiZWR1Y1wiLCBcInBoc3RhdFwiLCBcInVuZW1wbG95XCIsIFwibWFuYWdlZGNhcmVcIikiLCJzYW1wbGUiOiIjIFJlZ3Jlc3MgYGxvZ2V4cGVuZGAgb24gdGhlIGV4cGxhbmF0b3J5IHZhcmlhYmxlcyBsaXN0ZWQgaW4gYGV4cGx2YXJzM2Bcbm1lcHNfbWxyMyA8LSBsbShsb2dleHBlbmQgfiBnZW5kZXIgKyBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCAgKyBpbnN1cmUgKyBtcG9vciwgZGF0YSA9IHRyYWluX21lcHMpXG5cbiMgUHJlZGljdCBleHBlbmRpdHVyZXMgKG5vdCBsb2dnZWQpIGFuZCBzdW1tYXJpemUgdXNpbmcgdGhlIHN1bSBvZiBhYnNvbHV0ZSBwcmVkaWN0aW9uIGVycm9ycy5cbmV4cGx2YXJzMyA8LSBjKFwiZ2VuZGVyXCIsIFwiYWdlXCIsIFwicGhzdGF0XCIsIFwiYW55bGltaXRcIiwgXCJpbnN1cmVcIiwgXCJtcG9vclwiKVxucHJlZGljdF9tZXBzMyA8LSB0ZXN0X21lcHNbLGV4cGx2YXJzM11cbnByZWRpY3RfbWxyMyAgPC0gZXhwKHByZWRpY3QobWVwc19tbHIzLCBwcmVkaWN0X21lcHMzKSlcbnByZWRpY3RfZXJyX21scjMgPC0gdGVzdF9tZXBzJGV4cGVuZG9wIC0gcHJlZGljdF9tbHIzXG5zYXBlMyAgICAgPC0gc3VtKGFicyhwcmVkaWN0X2Vycl9tbHIzKSkvMTAwMFxuXG4jIFJlZ3Jlc3MgYGV4cGVuZG9wYCBvbiBhbGwgMTMgZXhwbGFuYXRvcnkgdmFyaWFibGVzXG5tZXBzX21scjQgPC0gbG0oX19ffiBnZW5kZXIgKyBhZ2UgKyByYWNlICsgcmVnaW9uICsgZWR1YyArIHBoc3RhdCArIG1wb29yICsgYW55bGltaXQgKyBpbmNvbWUgKyBpbnN1cmUgKyB1c2MgKyB1bmVtcGxveSArIG1hbmFnZWRjYXJlLCBkYXRhID0gdHJhaW5fbWVwcylcblxuIyBQcmVkaWN0IGV4cGVuZGl0dXJlcyBhbmQgc3VtbWFyaXplIHVzaW5nIHRoZSBzdW0gb2YgYWJzb2x1dGUgcHJlZGljdGlvbiBlcnJvcnMuXG5leHBsdmFyczQgPC0gYyhcImdlbmRlclwiLFwiYWdlXCIsXCJyYWNlXCIsXCJyZWdpb25cIixcImVkdWNcIixcInBoc3RhdFwiLFwibXBvb3JcIixcImFueWxpbWl0XCIsXCJpbmNvbWVcIixcImluc3VyZVwiLFwidXNjXCIsXCJ1bmVtcGxveVwiLFwibWFuYWdlZGNhcmVcIilcbnByZWRpY3RfbWVwczQgPC0gdGVzdF9tZXBzWyxleHBsdmFyczRdXG5wcmVkaWN0X21scjQgIDwtIHByZWRpY3QobWVwc19tbHI0LCBwcmVkaWN0X21lcHM0KVxucHJlZGljdF9lcnJfbWxyNCA8LSB0ZXN0X21lcHMkZXhwZW5kb3AgLSBwcmVkaWN0X21scjRcbnNhcGU0ICAgICA8LSBzdW0oYWJzKHByZWRpY3RfZXJyX21scjQpKS8xMDAwXG5zYXBlMztzYXBlNFxuXG4jIENvbXBhcmUgdGhlIHByZWRpY3Rpb25zIG9mIHRoZSBtb2RlbHMgZ3JhcGhpY2FsbHkuXG5wYXIobWZyb3cgPSBjKDEsIDIpKVxucGxvdChwcmVkaWN0X2Vycl9tbHI0LCBwcmVkaWN0X2Vycl9tbHIzLCB4bGFiID0gXCJCZW5jaG1hcmsgUHJlZGljdCBFcnJvclwiLCB5bGFiID0gXCJNTFIgUHJlZGljdCBFcnJvclwiKVxucGxvdChwcmVkaWN0X21scjMsIHRlc3RfbWVwcyRleHBlbmRvcCwgeGxhYiA9IFwiTUxSIFByZWRpY3RzXCIsIHlsYWIgPSBcIkhlbGQgT3V0IEV4cGVuZHNcIikiLCJzb2x1dGlvbiI6Im1lcHNfbWxyMyA8LSBsbShsb2dleHBlbmQgfiBnZW5kZXIgKyBhZ2UgKyBwaHN0YXQgKyBhbnlsaW1pdCAgKyBpbnN1cmUgKyBtcG9vciwgZGF0YSA9IHRyYWluX21lcHMpXG5leHBsdmFyczMgPC0gYyhcImdlbmRlclwiLCBcImFnZVwiLCBcInBoc3RhdFwiLCBcImFueWxpbWl0XCIsIFwiaW5zdXJlXCIsIFwibXBvb3JcIilcbnByZWRpY3RfbWVwczMgPC0gdGVzdF9tZXBzWyxleHBsdmFyczNdXG5wcmVkaWN0X21scjMgIDwtIGV4cChwcmVkaWN0KG1lcHNfbWxyMywgcHJlZGljdF9tZXBzMykpXG5wcmVkaWN0X2Vycl9tbHIzIDwtIHRlc3RfbWVwcyRleHBlbmRvcCAtIHByZWRpY3RfbWxyM1xuc2FwZTMgICAgIDwtIHN1bShhYnMocHJlZGljdF9lcnJfbWxyMykpLzEwMDBcblxubWVwc19tbHI0IDwtIGxtKGV4cGVuZG9wIH4gZ2VuZGVyICsgYWdlICsgcmFjZSArIHJlZ2lvbiArIGVkdWMgKyBwaHN0YXQgKyBtcG9vciArIGFueWxpbWl0ICsgaW5jb21lICsgaW5zdXJlICsgdXNjICsgdW5lbXBsb3kgKyBtYW5hZ2VkY2FyZSwgZGF0YSA9IHRyYWluX21lcHMpXG5leHBsdmFyczQgPC0gYyhcImdlbmRlclwiLFwiYWdlXCIsXCJyYWNlXCIsXCJyZWdpb25cIixcImVkdWNcIixcInBoc3RhdFwiLFwibXBvb3JcIixcImFueWxpbWl0XCIsXCJpbmNvbWVcIixcImluc3VyZVwiLFwidXNjXCIsXCJ1bmVtcGxveVwiLFwibWFuYWdlZGNhcmVcIilcbnByZWRpY3RfbWVwczQgPC0gdGVzdF9tZXBzWyxleHBsdmFyczRdXG5wcmVkaWN0X21scjQgIDwtIHByZWRpY3QobWVwc19tbHI0LCBwcmVkaWN0X21lcHM0KVxucHJlZGljdF9lcnJfbWxyNCA8LSB0ZXN0X21lcHMkZXhwZW5kb3AgLSBwcmVkaWN0X21scjRcbnNhcGU0ICAgICA8LSBzdW0oYWJzKHByZWRpY3RfZXJyX21scjQpKS8xMDAwXG5cbnNhcGUzO3NhcGU0XG5cbnBhcihtZnJvdyA9IGMoMSwgMikpXG5wbG90KHByZWRpY3RfZXJyX21scjQsIHByZWRpY3RfZXJyX21scjMsIHhsYWIgPSBcIkJlbmNobWFyayBQcmVkaWN0IEVycm9yXCIsIHlsYWIgPSBcIk1MUiBQcmVkaWN0IEVycm9yXCIpXG5wbG90KHByZWRpY3RfbWxyMywgdGVzdF9tZXBzJGV4cGVuZG9wLCB4bGFiID0gXCJNTFIgUHJlZGljdHNcIiwgeWxhYiA9IFwiSGVsZCBPdXQgRXhwZW5kc1wiKSIsInNjdCI6IiMgZXgoKSAlPiUgY2hlY2tfb2JqZWN0KFwibWVwc19tbHIzXCIsIHVuZGVmaW5lZF9tc2c9XCJVc2UgYGxtYCB0byBjcmVhdGUgYSBsaW5lYXIgbW9kZWwgbmFtZWQgYG1lcHNfbWxyM2AuXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIGNyZWF0ZSB5b3VyIG1vZGVsIHNvIHRoYXQgYGxvZ2V4cGVuZGAgaXMgYmFzZWQgb24gYGdlbmRlcmAsIGBhZ2VgLCBgcGhzdGF0YCwgYGFueWxpbWl0YCwgYGluc3VyZWAsIGFuZCBgbXBvb3JgIGZyb20gdGhlIGRhdGEgaW4gYHRyYWluX21lcHNgLlwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJleHBsdmFyczNcIiwgdW5kZWZpbmVkX21zZz1cIkNyZWF0ZSBhIHZhcmlhYmxlIG5hbWVkIGBleHBsdmFyczNgIHRoYXQgaGFzIGBnZW5kZXJgLCBgYWdlYCwgYHBoc3RhdGAsIGBhbnlsaW1pdGAsIGBpbnN1cmVgLCBhbmQgYG1wb29yYCBhcyBjaGFyYWN0ZXIgc3RyaW5ncy4gXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIGhhdmUgdGhlIGZvbGxvd2luZyBhcyBhIGNoYXJhY3RlciBzdHJpbmcgaW5zdGVhZCBvZiB0aGUgdmFsdWVzIGNvbnRhaW5lZCBpbiBlYWNoOiBgZ2VuZGVyYCwgYGFnZWAsIGBwaHN0YXRgLCBgYW55bGltaXRgLCBgaW5zdXJlYCwgYW5kIGBtcG9vcmAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX29iamVjdChcInByZWRpY3RfbWVwczNcIiwgdW5kZWZpbmVkX21zZz1cIk1ha2UgYSBuZXcgZGF0YWZyYW1lIG5hbWVkIGBwcmVkaWN0X21lcHMzYCB0aGF0IGEgc3Vic2V0IG9mIHRoZSBkYXRhIG5vdCB1c2VkIHRvIGNyZWF0ZSB0aGUgbW9kZWwuIFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIlRoZSBzdWJzZXQgb2YgZGF0YSBub3QgdXNlZCBpcyBzdG9yZWQgaW4gYHRlc3RfbWVwc2AsIGFuZCB0aGUgY29sdW1ucyB3ZSB3YW50IGFyZSBub3cgc3RvcmVkIGluIGBleHBsdmFyczNgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcImV4cFwiLGluZGV4PTEsbm90X2NhbGxlZF9tc2c9XCJNYWtlIHN1cmUgdG8gdGFrZSB0aGUgZXhwb25lbnRpYWwgb2YgdGhlIHByZWRpY3RlZCB2YWx1ZXMgdG8gcmVtb3ZlIHRoZSBsb2cgZWZmZWN0LiBcIikgJT4lIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gcnVuIGBleHBgIG9uIHRoZSB2YWx1ZXMgeW91IGdldCBmcm9tIHlvdXIgYHByZWRpY3RgIGNhbGwuIFwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInByZWRpY3RcIixpbmRleD0xLCBub3RfY2FsbGVkX21zZz1cIlJ1biBgcHJlZGljdGAgb24gdGhlIGRhdGEgeW91IGRpZCBub3QgdXNlIHRvIGNyZWF0ZSB0aGUgbW9kZWwuIFwiKSAlPiUge1xuIyAgIGNoZWNrX2FyZyguLCBcIm9iamVjdFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgdGhlIG1vZGVsIHdlIGFyZSB1c2luZyB0byBtYWtlIHByZWRpY3Rpb25zIGlzIGBtZXBzX21scjNgLlwiKVxuIyAgIGNoZWNrX2FyZyguLCBcIm5ld2RhdGFcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gc3BlY2lmeSB0aGF0IHdlIHdvdWxkIGxpa2UgcHJlZGljdGlvbnMgZm9yIHRoZSB2YWx1ZXMgZm91bmQgaW4gYHByZWRpY3RfbWVwczNgLlwiKVxuIyB9XG4jIGV4KCkgJT4lIGNoZWNrX29iamVjdChcInByZWRpY3RfbWxyM1wiLCB1bmRlZmluZWRfbXNnPVwiTWFrZSBzdXJlIHRvIHNhdmUgdGhlIGV4cG9uZW50aWFsIG9mIHRoZSBwcmVkaWN0ZWQgdmFsdWVzIHRvIGBwcmVkaWN0X21scjNgLlwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byB0YWtlIHRoZSBleHBvbmVudGlhbCBvZiB0aGUgdmFsdWVzIGZvdW5kIHVzaW5nIGBwcmVkaWN0YC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfb2JqZWN0KFwicHJlZGljdF9lcnJfbWxyM1wiLCB1bmRlZmluZWRfbXNnPVwiU2F2ZSB0aGUgcHJlZGljdGlvbiBlcnJvcnMgdG8gYSB2YXJpYWJsZSBuYW1lZCBgcHJlZGljdF9lcnJfbWxyM2ApICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiVG8gZmluZCB0aGUgcHJlZGljdGlvbiBlcnJvcnMsIHN1YnRyYWN0IHRoZSBwcmVkaWN0ZWQgdmFsdWVzIGluIGBwcmVkaWN0X21scjNgIGZyb20gdGhlIGFjdHVhbCB2YWx1ZXMgZm91bmQgaW4gYHRlc3RfbWVwcyRleHBlbmRvcGAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwiYWJzXCIsaW5kZXg9MSwgdW5kZWZpbmVkX21zZz1cIk1ha2Ugc3VyZSB0byB0YWtlIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMuIFwiKSAlPiUgY2hlY2tfYXJnKC4sIFwieFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byB0YWtlIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMsIHdoaWNoIGNhbiBiZSBmb3VuZCBpbiBgcHJlZGljdF9lcnJfbWxyM2AuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX2Z1bmN0aW9uKFwic3VtXCIsaW5kZXg9MSwgbm90X2NhbGxlZF9tc2c9XCJUYWtlIHRoZSBzdW0gb2YgdGhlIGFic29sdXRlIHZhbHVlIG9mIHRoZSBwcmVkaWN0aW9uIGVycm9ycy4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJ4XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIHRoZSBzdW0gb2YgdGhlIGFic29sdXRlIHZhbHVlIG9mIHRoZSBwcmVkaWN0aW9uIGVycm9ycy4gXCIpXG4jIGV4KCkgJT4lIGNoZWNrX29iamVjdChcInNhcGUzXCIsIHVuZGVmaW5lZF9tc2c9XCJNYWtlIHN1cmUgdG8gc2V0IHRoZSBzdW0gb2YgdGhlIGFic29sdXRlIHZhbHVlIG9mIHRoZSBwcmVkaWN0aW9uIGVycm9ycyBkaXZpZGVkIGJ5IDEwMDAgZXF1YWwgdG8gYHNhcGUzYC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdGhhdCB5b3UgaGF2ZSB0YWtlbiB0aGUgc3VtIG9mIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMsIGFuZCBkaXZpZGVkIHRoYXQgbnVtYmVyIGJ5IDEwMDAuIFwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJtZXBzX21scjRcIiwgdW5kZWZpbmVkX21zZz1cIlVzZSBgbG1gIHRvIGNyZWF0ZSBhIGxpbmVhciBtb2RlbCBuYW1lZCBgbWVwc19tbHI0YC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gY3JlYXRlIHlvdXIgbW9kZWwgc28gdGhhdCBgZXhwZW5kb3BgIGlzIGJhc2VkIG9uIGBnZW5kZXJgLGAgYWdlYCwgYHJhY2VgLCBgcmVnaW9uYCwgYGVkdWNgLCBgcGhzdGF0YCwgYG1wb29yYCwgYGFueWxpbWl0YCwgYGluY29tZWAsIGB1c2NgLCBgdW5lbXBsb3lgLCBhbmQgYG1hbmFnZWRjYXJlYCBmcm9tIHRoZSBkYXRhIGluIGB0cmFpbl9tZXBzYC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfb2JqZWN0KFwiZXhwbHZhcnM0XCIsIHVuZGVmaW5lZF9tc2c9XCJDcmVhdGUgYSB2YXJpYWJsZSBuYW1lZCBgZXhwbHZhcnM0YCB0aGF0IGhhcyBgZ2VuZGVyYCxgIGFnZWAsIGByYWNlYCwgYHJlZ2lvbmAsIGBlZHVjYCwgYHBoc3RhdGAsIGBtcG9vcmAsIGBhbnlsaW1pdGAsIGBpbmNvbWVgLCBgdXNjYCwgYHVuZW1wbG95YCwgYW5kIGBtYW5hZ2VkY2FyZWAgYXMgY2hhcmFjdGVyIHN0cmluZ3MuIFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBoYXZlIHRoZSBmb2xsb3dpbmcgYXMgYSBjaGFyYWN0ZXIgc3RyaW5nIGluc3RlYWQgb2YgdGhlIHZhbHVlcyBjb250YWluZWQgaW4gZWFjaDogYGdlbmRlcmAsYCBhZ2VgLCBgcmFjZWAsIGByZWdpb25gLCBgZWR1Y2AsIGBwaHN0YXRgLCBgbXBvb3JgLCBgYW55bGltaXRgLCBgaW5jb21lYCwgYHVzY2AsIGB1bmVtcGxveWAsIGFuZCBgbWFuYWdlZGNhcmVgLlwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJwcmVkaWN0X21lcHM0XCIsIHVuZGVmaW5lZF9tc2c9XCJNYWtlIGEgbmV3IGRhdGFmcmFtZSBuYW1lZCBgcHJlZGljdF9tZXBzNGAgdGhhdCBhIHN1YnNldCBvZiB0aGUgZGF0YSBub3QgdXNlZCB0byBjcmVhdGUgdGhlIG1vZGVsLiBcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgc3Vic2V0IG9mIGRhdGEgbm90IHVzZWQgaXMgc3RvcmVkIGluIGB0ZXN0X21lcHNgLCBhbmQgdGhlIGNvbHVtbnMgd2Ugd2FudCBhcmUgbm93IHN0b3JlZCBpbiBgZXhwbHZhcnM0YC5cIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJleHBcIixpbmRleD0yLG5vdF9jYWxsZWRfbXNnPVwiTWFrZSBzdXJlIHRvIHRha2UgdGhlIGV4cG9uZW50aWFsIG9mIHRoZSBwcmVkaWN0ZWQgdmFsdWVzIHRvIHJlbW92ZSB0aGUgbG9nIGVmZmVjdC4gXCIpICU+JSBjaGVja19hcmcoLiwgXCJ4XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHJ1biBgZXhwYCBvbiB0aGUgdmFsdWVzIHlvdSBnZXQgZnJvbSB5b3VyIGBwcmVkaWN0YCBjYWxsLiBcIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwcmVkaWN0XCIsaW5kZXg9Miwgbm90X2NhbGxlZF9tc2c9XCJSdW4gYHByZWRpY3RgIG9uIHRoZSBkYXRhIHlvdSBkaWQgbm90IHVzZSB0byBjcmVhdGUgdGhlIG1vZGVsLiBcIikgJT4lIHtcbiMgICBjaGVja19hcmcoLiwgXCJvYmplY3RcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gc3BlY2lmeSB0aGF0IHRoZSBtb2RlbCB3ZSBhcmUgdXNpbmcgdG8gbWFrZSBwcmVkaWN0aW9ucyBpcyBgbWVwc19tbHI0YC5cIilcbiMgICBjaGVja19hcmcoLiwgXCJuZXdkYXRhXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRvIHNwZWNpZnkgdGhhdCB3ZSB3b3VsZCBsaWtlIHByZWRpY3Rpb25zIGZvciB0aGUgdmFsdWVzIGZvdW5kIGluIGBwcmVkaWN0X21lcHM0YC5cIilcbiMgfVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJwcmVkaWN0X21scjRcIiwgdW5kZWZpbmVkX21zZz1cIk1ha2Ugc3VyZSB0byBzYXZlIHRoZSBleHBvbmVudGlhbCBvZiB0aGUgcHJlZGljdGVkIHZhbHVlcyB0byBgcHJlZGljdF9tbHI0YC5cIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gdGFrZSB0aGUgZXhwb25lbnRpYWwgb2YgdGhlIHZhbHVlcyBmb3VuZCB1c2luZyBgcHJlZGljdGAuXCIpXG4jIGV4KCkgJT4lIGNoZWNrX29iamVjdChcInByZWRpY3RfZXJyX21scjRcIiwgdW5kZWZpbmVkX21zZz1cIlNhdmUgdGhlIHByZWRpY3Rpb24gZXJyb3JzIHRvIGEgdmFyaWFibGUgbmFtZWQgYHByZWRpY3RfZXJyX21scjRgKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIlRvIGZpbmQgdGhlIHByZWRpY3Rpb24gZXJyb3JzLCBzdWJ0cmFjdCB0aGUgcHJlZGljdGVkIHZhbHVlcyBpbiBgcHJlZGljdF9tbHI0YCBmcm9tIHRoZSBhY3R1YWwgdmFsdWVzIGZvdW5kIGluIGB0ZXN0X21lcHMkZXhwZW5kb3BgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcImFic1wiLGluZGV4PTIsIHVuZGVmaW5lZF9tc2c9XCJNYWtlIHN1cmUgdG8gdGFrZSB0aGUgYWJzb2x1dGUgdmFsdWUgb2YgdGhlIHByZWRpY3Rpb24gZXJyb3JzLiBcIikgJT4lIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJNYWtlIHN1cmUgdG8gdGFrZSB0aGUgYWJzb2x1dGUgdmFsdWUgb2YgdGhlIHByZWRpY3Rpb24gZXJyb3JzLCB3aGljaCBjYW4gYmUgZm91bmQgaW4gYHByZWRpY3RfZXJyX21scjRgLlwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInN1bVwiLGluZGV4PTIsIG5vdF9jYWxsZWRfbXNnPVwiVGFrZSB0aGUgc3VtIG9mIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMuIFwiKSAlPiUgY2hlY2tfYXJnKC4sIFwieFwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIk1ha2Ugc3VyZSB0byBzcGVjaWZ5IHRoYXQgd2Ugd291bGQgbGlrZSB0aGUgc3VtIG9mIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMuIFwiKVxuIyBleCgpICU+JSBjaGVja19vYmplY3QoXCJzYXBlNFwiLCB1bmRlZmluZWRfbXNnPVwiTWFrZSBzdXJlIHRvIHNldCB0aGUgc3VtIG9mIHRoZSBhYnNvbHV0ZSB2YWx1ZSBvZiB0aGUgcHJlZGljdGlvbiBlcnJvcnMgZGl2aWRlZCBieSAxMDAwIGVxdWFsIHRvIGBzYXBlNGAuXCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiTWFrZSBzdXJlIHRoYXQgeW91IGhhdmUgdGFrZW4gdGhlIHN1bSBvZiB0aGUgYWJzb2x1dGUgdmFsdWUgb2YgdGhlIHByZWRpY3Rpb24gZXJyb3JzLCBhbmQgZGl2aWRlZCB0aGF0IG51bWJlciBieSAxMDAwLiBcIilcbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwYXJcIixub3RfY2FsbGVkX21zZz1cIlVzZSBgcGFyYCB0byBhbHRlciB0aGUgZ3JhcGhpbmcgZGV2aWNlIHRvIGhhdmUgMiBzaWRlIGJ5IHNpZGUgZ3JhcGhzLiBcIikgJT4lIGNoZWNrX2FyZyguLCBcIm1mcm93XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiVG8gZG8gdGhpcywgc2V0IGBtZnJvd2AgZXF1YWwgdG8gYGMoMiwxKWAuIFwiKVxuIyBleCgpICU+JSBjaGVja19mdW5jdGlvbihcInBsb3RcIixpbmRleD0xLCBub3RfY2FsbGVkX21zZz1cIkNyZWF0ZSBhIHBsb3Qgb2YgdGhlIHR3byBwcmVkaWN0aW9uIGVycm9ycyBhcyB0aGUgZmlyc3QgZ3JhcGguIFwiKSAlPiUge1xuIyAgIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgeCBheGlzIHNob3VsZCBoYXZlIHRoZSBwcmVkaWN0aW9uIGVycm9ycyBmcm9tIGBwcmVkaWN0X2Vycl9tbHI0YC5cIilcbiMgICBjaGVja19hcmcoLiwgXCJ5XCIpICU+JSBjaGVja19lcXVhbChpbmNvcnJlY3RfbXNnPVwiVGhlIHkgYXhpcyBzaG91bGQgaGF2ZSB0aGUgcHJlZGljdGlvbiBlcnJvcnMgZnJvbSBgcHJlZGljdF9lcnJfbWxyM2AuXCIpXG4jIH1cbiMgZXgoKSAlPiUgY2hlY2tfZnVuY3Rpb24oXCJwbG90XCIsaW5kZXg9Miwgbm90X2NhbGxlZF9tc2c9XCJUaGUgc2Vjb25kIGdyYXBoIHNob3VsZCBwbG90IHRoZSBwcmVkaWN0aW9ucyBmcm9tIG1scjMgYWdhaW5zdCB0aGUgYWN0dWFsIHZhbHVlcyBvZiBleHBlbmRpdHVyZXMuIFwiKSAlPiUge1xuIyAgIGNoZWNrX2FyZyguLCBcInhcIikgJT4lIGNoZWNrX2VxdWFsKGluY29ycmVjdF9tc2c9XCJUaGUgWCBheGlzIHNob3VsZCBoYXZlIHRoZSBwcmVkaWN0aW9ucywgZm91bmQgaW4gYHByZWRpY3RfbWxyM2AuXCIpXG4jICAgY2hlY2tfYXJnKC4sIFwieVwiKSAlPiUgY2hlY2tfZXF1YWwoaW5jb3JyZWN0X21zZz1cIlRoZSBZIGF4aXMgc2hvdWxkIGhhdmUgdGhlIGFjdHVhbCBleHBlbmRpdHVyZSB2YWx1ZXMsIGZvdW5kIGluIGB0ZXN0X21lcHMkZXhwZW5kb3BgLlwiKVxuIyB9XG5zdWNjZXNzX21zZyhcIkV4Y2VsbGVudCEgV2UgZm91bmQgdGhhdCB0aGUgbW9kZWwgb2YgbG9nIGV4cGVuZGl0dXJlcyBvdXRwZXJmb3JtcyB0aGUgYmVuY2htYXJrIHRoYXQgbW9kZWxzIGV4cGVuZGl0dXJlcywgZXZlbiB3aGVuIHRoZSBvdXQgb2Ygc2FtcGxlIGNyaXRlcmlvbiB3YXMgaW4gdGhlIG9yaWdpbmFsICdkb2xsYXInIHVuaXRzLiBJdCBpcyBjb21vZm9ydGluZyB0byBrbm93IHRoYXQgYSBzZWFyY2ggZm9yIGEgZ29vZCBtb2RlbCBkb2VzIHdlbGwgd2hlbiB1c2luZyBkaWZmZXJlbnQgb3V0IG9mIHNhbXBsZSBjcml0ZXJpYS5cIikiLCJoaW50IjoiVGhlcmUgaXMgbm90aGluZyBpbiB0aGlzIGNvZGUgdG8gY2hhbmdlLiBzaW1wbHkgZXhhbWluZSB0aGUgY29kZSwgcGxheSBhcm91bmQgd2l0aCBpdCwgYW5kIGZpZ3VyZSBvdXQgd2hhdCBlYWNoIGluZGl2aWR1YWwgbGluZSBkb2VzLCBhbmQgaG93IHRoZXkgd29yayB0b2dldGhlci4ifQ==

5.2 What the modeling procedure tells us

In this section, you learn how to:

  • Interpret individual effects, based on their substantive and statistical significance
  • Describe other purposes of regression modeling, including regression function for pricing, benchmarking studies, and predicting future observations.

5.2.1 Video

Video Overhead Details

Hide

A Details. Interpreting individual effects

  • Substantive Effect
    • Does a 1 unit change in \(x\) imply an economically meaningful change in \(y\)?
    • Example: Looking at urban and rural claims experience, is there a big enough difference to warrant differentiating prices by location?
  • Statistical Significance
    • We have standards for deciding whether or not a variable is statistically significant.
    • A “statistically significant effect” is the result of a regression coefficient that is large relative to its standard error.
  • Statistical significance is driven by
    • precision of \(s\),
    • collinearity (\(VIF\)) and
    • sample size
  • Causal Effects
    • If we change \(x\), would \(y\) change?

Hide

B Details. Other Interpretations

  • Regression function and pricing
    • The regression function is \(\mathrm{E~}y = \beta_0 + \beta_1 x_1 + \cdots +\beta _k x_k\).
    • Think about expected claims as our baseline price for short-term insurance coverages.
  • Benchmarking studies
    • In studies of CEO’s salaries, who is making a lot (or a little), controlled for industry, years of experience and so forth?
    • In studies of medical claims, who are the high-cost patients?
  • Prediction
    • A new patient comes in with a given set of characteristics, what can I say about his or her future medical claims?


MC Exercise. Which of the following are not important when interpreting the effects of individual variables?
Substantive significance
Statistical significance
The amount of effort that it took to gather the data and do the analysis
Role of causality



MC Exercise. Which of the following is not a potential explanation for the lack of statistical significance of an explanatory variable?
Large variation of the disturbance term
High collinearity, so that the variable may be confounded with other variables
The coefficient of determination, \(R^2\), is not sufficiently large



MC Exercise. Which of the following is not an important purpose of regression modeling?
Pricing of risks such as insurance contracts
Benchmarking studies, to compare an observation to others
Prediction
Keeping a computer occupied with work

5.3 The importance of variable selection

In this section, you learn how to:

  • Describe the bias that can occur when omitting important variables
  • Describe the principle of parsimony and reasons for adopting this approach

5.3.1 Video

Video Overhead Details

<button class=“tablinks” onclick=“openTab(event, ‘Over5.3C’)”C Details. Principle of parsimony

Hide

A Details. The importance of variable selection

  • With too many or too few variables, \(s\) is too large an estimate of \(\sigma\).
    • Prediction intervals are too large
    • Standard errors for the partial slopes are too large
  • With too few or incorrect variables, we produce biased estimates of the slopes \(\beta\). Thus, our predictions are biased and hence inaccurate.

Hide

B Details. Example. Regression using one explanatory variable

  • Too Many Variables
    • The “true” model is \(y_i = \beta_0+ \varepsilon_i\)
    • We mistakenly use \(y_i = \beta_0+ \beta_1 x_i^* + \varepsilon_i\)
    • The prediction at a generic level \(x\) is \(b_0^* + b_1^* x\).
    • It is not to hard to confirm that \(Bias = \mathrm{E} (b_0^* + b_1^* x) - \mathrm{E } y= 0\).
  • Too Few Variables
    • The “true” model is \(y_i = \beta_0 + \beta_1 x_i + \varepsilon_i\).
    • We mistakenly use \(y_i = \beta_0^* \varepsilon_i\).
    • Under the true model, \(\overline{y} = \beta_0 + \beta_1 \overline{x} + \overline{\varepsilon}\)
    • Thus, the bias is

\[ Bias = \text{E }\bar{y} - \text{E }(\beta_0 + \beta_1 x + \varepsilon) \\ = \text{E }(\beta_0 + \beta_1 \bar{x}+\bar{\varepsilon})-(\beta_0+\beta_1 x)=\beta_1 (\bar{x}-x). \]

There is a persistent, long-term error in omitting the explanatory variable \(x\).

Hide

C Details. Principle of parsimony

  • The principle of parsimony, also known as Occam’s Razor, states that when there are several possible explanations for a phenomenon, use the simplest.
    • A simpler explanation is easier to interpret.
    • Simpler models, also known as ``more parsimonious’’ models, often do well on fitting out-of-sample data
    • Extraneous variables can cause problems of collinearity, leading to difficulty in interpreting individual coefficients.
  • In contrast, in a quote often attributed to Albert Einstein, we should use “the simplest model possible, but no simpler.”
    • Omitting important variables can lead to biased results, a potentially serious error.
    • Including extraneous variables decreases the degrees of freedom and increases the estimate of variability, typically of less concern in actuarial applications.


MC Exercise. Which of the following is true about under- and over-fitting a model?
When we over-fit a model, estimates of regression coefficients are over-biased as is \(s^2\), the estimate of model variance \(\sigma^2\).
When we over-fit a model, estimates of regression coefficients remain unbiased whereas \(s^2\), the estimate of model variance \(\sigma^2\), is over-biased.
When we over-fit a model, estimates of regression coefficients remain under-biased as is \(s^2\), the estimate of model variance \(\sigma^2\).
When we under-fit a model, estimates of regression coefficients remain unbiased whereas \(s^2\), the estimate of model variance \(\sigma^2\), is over-biased.



MC Exercise. Which of the following is not true of Occam’s Razor?
When there are several possible explanations for a phenomenon, use the simplest one.
Simpler models are easier to interpret.
Variables can be statistically significant but practically unimportant.
Simpler models often do better for predicting out-of-sample data