How much data is missing?<\/h2>\r\nThe overall percentage of data that is missing is important. Generally, if less than 5% of values are missing then it is acceptable to ignore them (REF). However, the overall percentage missing alone is not enough; you also need to pay attention to which<\/em> data is missing. Often you may need to consider deleting cases (participants) or individual variables that are missing a ton of values. This step alone can drastically improve the integrity of your data and reduce the overall percentage of missing values in your dataset.\r\n\r\n
\r\n\r\nTypes of Missing Data<\/h1>\r\nThere are several types of missing data, as we will discuss here.\u00a0 Some types are easy to consider and account for, while others are confusing and may be less obvious to the novice researcher.\r\nData Missing at Random<\/h3>\r\nIn this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:\r\n\r\nExample 1: <\/strong>\r\n\r\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).\r\n\r\nExample 2: <\/strong>\r\n\r\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)\r\n\r\nExample 3:<\/strong>\r\n\r\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.\r\n\r\nNote that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.\r\nMissing Completely at Random (MCAR)<\/h3>\r\nMissing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:\r\n\r\nExample 1:<\/strong>\r\n\r\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.\r\n\r\nExample 2:<\/strong>\r\n\r\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.\r\n\r\nThere is NO pattern causing the missing data.\r\nNot Missing at Random (NMAR)<\/h3>\r\nThe last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.\r\n\r\nExample 1:<\/strong>\r\n\r\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).\r\n\r\nExample 2:<\/strong>\r\n\r\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).\r\n\r\n
\r\n\r\nAnalyzing Missing Values<\/h3>\r\nThe default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.\r\n\r\nOne of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.\r\n\r\n<\/code>\r\n\r\nThe following code uses the NIMPUTE=0 option to create the \"Missing Data Patterns\" table for the specified variables.<\/p>\r\n\r\n<\/header>\r\n
\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/code>\r\n\r\nLet\u2019s do an example together:<\/strong>\r\n\r\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.\r\n\r\nThe first step of course is to set up your data file in SAS:<\/p>\r\n\r\n<\/header>\r\n
\r\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\r\n\r\n
\r\n\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.\r\n\r\nThe next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.\r\n\r\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.\r\nOutput table showing patterns of missing values<\/h6>\r\n![\"\"](\"http:\/\/pressbooks.library.upei.ca\/montelpare\/wp-content\/uploads\/sites\/49\/2020\/04\/missingData.png\")
\r\n\r\n
\r\n\r\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.","rendered":"
In this section, we will work through the concepts of dealing with missing data using specific examples that are demonstrated with SAS coding, and which are based on the SAS Studio Education Analytic Suite.<\/p>\n
\nMissing Values<\/h2>\n
Missing data are observations that we intended to record but did not. Values can be missing for different reasons and most of the time we don\u2019t know the exact reason why people didn\u2019t answer certain questions. However, we can look at how much data is missing as well as the patterns of missing values and determine whether missingness is related to the variable itself, other variables in the dataset, or has no apparent pattern. In the following sections, we will go through three categories of missing data that are commonly used in research to explain why data is missing.<\/p>\n
How much data is missing?<\/h2>\n
The overall percentage of data that is missing is important. Generally, if less than 5% of values are missing then it is acceptable to ignore them (REF). However, the overall percentage missing alone is not enough; you also need to pay attention to which<\/em> data is missing. Often you may need to consider deleting cases (participants) or individual variables that are missing a ton of values. This step alone can drastically improve the integrity of your data and reduce the overall percentage of missing values in your dataset.<\/p>\n
\nTypes of Missing Data<\/h1>\n
There are several types of missing data, as we will discuss here.\u00a0 Some types are easy to consider and account for, while others are confusing and may be less obvious to the novice researcher.<\/p>\n
Data Missing at Random<\/h3>\n
In this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:<\/p>\n
Example 1: <\/strong><\/p>\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).<\/p>\nExample 2: <\/strong><\/p>\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)<\/p>\nExample 3:<\/strong><\/p>\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.<\/p>\n
Note that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.<\/p>\n
Missing Completely at Random (MCAR)<\/h3>\n
Missing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:<\/p>\n
Example 1:<\/strong><\/p>\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.<\/p>\n
Example 2:<\/strong><\/p>\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.<\/p>\n
There is NO pattern causing the missing data.<\/p>\n
Not Missing at Random (NMAR)<\/h3>\n
The last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.<\/p>\n
Example 1:<\/strong><\/p>\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).<\/p>\n
Example 2:<\/strong><\/p>\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).<\/p>\n
\nAnalyzing Missing Values<\/h3>\n
The default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.<\/p>\n
One of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.<\/p>\n
<\/code><\/p>\n\n\nThe following code uses the NIMPUTE=0 option to create the “Missing Data Patterns” table for the specified variables.<\/p>\n<\/header>\n
\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/code><\/p>\nLet\u2019s do an example together:<\/strong><\/p>\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.<\/p>\n
\n\nThe first step of course is to set up your data file in SAS:<\/p>\n<\/header>\n
\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\n
\n\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.<\/p>\n
The next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.<\/p>\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.<\/p>\n
Output table showing patterns of missing values<\/h6>\n
<\/p>\n
\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.<\/p>\n","protected":false},"author":56,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[47],"contributor":[],"license":[],"class_list":["post-283","chapter","type-chapter","status-publish","hentry","chapter-type-standard"],"part":180,"_links":{"self":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/users\/56"}],"version-history":[{"count":9,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions"}],"predecessor-version":[{"id":285,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions\/285"}],"part":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/parts\/180"}],"metadata":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/media?parent=283"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapter-type?post=283"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/contributor?post=283"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/license?post=283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Data Missing at Random<\/h3>\r\nIn this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:\r\n\r\nExample 1: <\/strong>\r\n\r\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).\r\n\r\nExample 2: <\/strong>\r\n\r\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)\r\n\r\nExample 3:<\/strong>\r\n\r\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.\r\n\r\nNote that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.\r\nMissing Completely at Random (MCAR)<\/h3>\r\nMissing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:\r\n\r\nExample 1:<\/strong>\r\n\r\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.\r\n\r\nExample 2:<\/strong>\r\n\r\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.\r\n\r\nThere is NO pattern causing the missing data.\r\nNot Missing at Random (NMAR)<\/h3>\r\nThe last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.\r\n\r\nExample 1:<\/strong>\r\n\r\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).\r\n\r\nExample 2:<\/strong>\r\n\r\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).\r\n\r\n
\r\n\r\nAnalyzing Missing Values<\/h3>\r\nThe default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.\r\n\r\nOne of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.\r\n\r\n<\/code>\r\n\r\nThe following code uses the NIMPUTE=0 option to create the \"Missing Data Patterns\" table for the specified variables.<\/p>\r\n\r\n<\/header>\r\n
\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/code>\r\n\r\nLet\u2019s do an example together:<\/strong>\r\n\r\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.\r\n\r\nThe first step of course is to set up your data file in SAS:<\/p>\r\n\r\n<\/header>\r\n
\r\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\r\n\r\n
\r\n\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.\r\n\r\nThe next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.\r\n\r\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.\r\nOutput table showing patterns of missing values<\/h6>\r\n\r\n\r\n
\r\n\r\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.","rendered":"
In this section, we will work through the concepts of dealing with missing data using specific examples that are demonstrated with SAS coding, and which are based on the SAS Studio Education Analytic Suite.<\/p>\n
\nMissing Values<\/h2>\n
Missing data are observations that we intended to record but did not. Values can be missing for different reasons and most of the time we don\u2019t know the exact reason why people didn\u2019t answer certain questions. However, we can look at how much data is missing as well as the patterns of missing values and determine whether missingness is related to the variable itself, other variables in the dataset, or has no apparent pattern. In the following sections, we will go through three categories of missing data that are commonly used in research to explain why data is missing.<\/p>\n
How much data is missing?<\/h2>\n
The overall percentage of data that is missing is important. Generally, if less than 5% of values are missing then it is acceptable to ignore them (REF). However, the overall percentage missing alone is not enough; you also need to pay attention to which<\/em> data is missing. Often you may need to consider deleting cases (participants) or individual variables that are missing a ton of values. This step alone can drastically improve the integrity of your data and reduce the overall percentage of missing values in your dataset.<\/p>\n
\nTypes of Missing Data<\/h1>\n
There are several types of missing data, as we will discuss here.\u00a0 Some types are easy to consider and account for, while others are confusing and may be less obvious to the novice researcher.<\/p>\n
Data Missing at Random<\/h3>\n
In this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:<\/p>\n
Example 1: <\/strong><\/p>\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).<\/p>\nExample 2: <\/strong><\/p>\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)<\/p>\nExample 3:<\/strong><\/p>\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.<\/p>\n
Note that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.<\/p>\n
Missing Completely at Random (MCAR)<\/h3>\n
Missing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:<\/p>\n
Example 1:<\/strong><\/p>\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.<\/p>\n
Example 2:<\/strong><\/p>\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.<\/p>\n
There is NO pattern causing the missing data.<\/p>\n
Not Missing at Random (NMAR)<\/h3>\n
The last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.<\/p>\n
Example 1:<\/strong><\/p>\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).<\/p>\n
Example 2:<\/strong><\/p>\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).<\/p>\n
\nAnalyzing Missing Values<\/h3>\n
The default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.<\/p>\n
One of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.<\/p>\n
<\/code><\/p>\n\n\nThe following code uses the NIMPUTE=0 option to create the “Missing Data Patterns” table for the specified variables.<\/p>\n<\/header>\n
\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/code><\/p>\nLet\u2019s do an example together:<\/strong><\/p>\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.<\/p>\n
\n\nThe first step of course is to set up your data file in SAS:<\/p>\n<\/header>\n
\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\n
\n\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.<\/p>\n
The next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.<\/p>\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.<\/p>\n
Output table showing patterns of missing values<\/h6>\n
<\/p>\n
\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.<\/p>\n","protected":false},"author":56,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[47],"contributor":[],"license":[],"class_list":["post-283","chapter","type-chapter","status-publish","hentry","chapter-type-standard"],"part":180,"_links":{"self":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/users\/56"}],"version-history":[{"count":9,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions"}],"predecessor-version":[{"id":285,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions\/285"}],"part":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/parts\/180"}],"metadata":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/media?parent=283"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapter-type?post=283"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/contributor?post=283"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/license?post=283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Not Missing at Random (NMAR)<\/h3>\r\nThe last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.\r\n\r\nExample 1:<\/strong>\r\n\r\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).\r\n\r\nExample 2:<\/strong>\r\n\r\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).\r\n\r\n
\r\n\r\nAnalyzing Missing Values<\/h3>\r\nThe default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.\r\n\r\nOne of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.\r\n\r\n<\/code>\r\n\r\nThe following code uses the NIMPUTE=0 option to create the \"Missing Data Patterns\" table for the specified variables.<\/p>\r\n\r\n<\/header>\r\n
\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/code>\r\n\r\nLet\u2019s do an example together:<\/strong>\r\n\r\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.\r\n\r\nThe first step of course is to set up your data file in SAS:<\/p>\r\n\r\n<\/header>\r\n
\r\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\r\n\r\n
\r\n\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.\r\n\r\nThe next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.\r\n\r\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.\r\nOutput table showing patterns of missing values<\/h6>\r\n\r\n\r\n
\r\n\r\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.","rendered":"
In this section, we will work through the concepts of dealing with missing data using specific examples that are demonstrated with SAS coding, and which are based on the SAS Studio Education Analytic Suite.<\/p>\n
\nMissing Values<\/h2>\n
Missing data are observations that we intended to record but did not. Values can be missing for different reasons and most of the time we don\u2019t know the exact reason why people didn\u2019t answer certain questions. However, we can look at how much data is missing as well as the patterns of missing values and determine whether missingness is related to the variable itself, other variables in the dataset, or has no apparent pattern. In the following sections, we will go through three categories of missing data that are commonly used in research to explain why data is missing.<\/p>\n
How much data is missing?<\/h2>\n
The overall percentage of data that is missing is important. Generally, if less than 5% of values are missing then it is acceptable to ignore them (REF). However, the overall percentage missing alone is not enough; you also need to pay attention to which<\/em> data is missing. Often you may need to consider deleting cases (participants) or individual variables that are missing a ton of values. This step alone can drastically improve the integrity of your data and reduce the overall percentage of missing values in your dataset.<\/p>\n
\nTypes of Missing Data<\/h1>\n
There are several types of missing data, as we will discuss here.\u00a0 Some types are easy to consider and account for, while others are confusing and may be less obvious to the novice researcher.<\/p>\n
Data Missing at Random<\/h3>\n
In this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:<\/p>\n
Example 1: <\/strong><\/p>\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).<\/p>\nExample 2: <\/strong><\/p>\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)<\/p>\nExample 3:<\/strong><\/p>\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.<\/p>\n
Note that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.<\/p>\n
Missing Completely at Random (MCAR)<\/h3>\n
Missing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:<\/p>\n
Example 1:<\/strong><\/p>\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.<\/p>\n
Example 2:<\/strong><\/p>\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.<\/p>\n
There is NO pattern causing the missing data.<\/p>\n
Not Missing at Random (NMAR)<\/h3>\n
The last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.<\/p>\n
Example 1:<\/strong><\/p>\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).<\/p>\n
Example 2:<\/strong><\/p>\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).<\/p>\n
\nAnalyzing Missing Values<\/h3>\n
The default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.<\/p>\n
One of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.<\/p>\n
<\/code><\/p>\n\n\nThe following code uses the NIMPUTE=0 option to create the “Missing Data Patterns” table for the specified variables.<\/p>\n<\/header>\n
\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/code><\/p>\nLet\u2019s do an example together:<\/strong><\/p>\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.<\/p>\n
\n\nThe first step of course is to set up your data file in SAS:<\/p>\n<\/header>\n
\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\n
\n\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.<\/p>\n
The next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.<\/p>\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.<\/p>\n
Output table showing patterns of missing values<\/h6>\n
<\/p>\n
\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.<\/p>\n","protected":false},"author":56,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[47],"contributor":[],"license":[],"class_list":["post-283","chapter","type-chapter","status-publish","hentry","chapter-type-standard"],"part":180,"_links":{"self":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/users\/56"}],"version-history":[{"count":9,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions"}],"predecessor-version":[{"id":285,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions\/285"}],"part":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/parts\/180"}],"metadata":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/media?parent=283"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapter-type?post=283"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/contributor?post=283"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/license?post=283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
<\/code>\r\n\r\nThe following code uses the NIMPUTE=0 option to create the \"Missing Data Patterns\" table for the specified variables.<\/p>\r\n\r\n<\/header>\r\n
\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/code>\r\n\r\nLet\u2019s do an example together:<\/strong>\r\n\r\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.\r\n\r\nThe first step of course is to set up your data file in SAS:<\/p>\r\n\r\n<\/header>\r\n
\r\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\r\n\r\n
\r\n\r\n\r\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.\r\n\r\nThe next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.\r\n\r\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.\r\nOutput table showing patterns of missing values<\/h6>\r\n\r\n\r\n
\r\n\r\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.","rendered":"
In this section, we will work through the concepts of dealing with missing data using specific examples that are demonstrated with SAS coding, and which are based on the SAS Studio Education Analytic Suite.<\/p>\n
\nMissing Values<\/h2>\n
Missing data are observations that we intended to record but did not. Values can be missing for different reasons and most of the time we don\u2019t know the exact reason why people didn\u2019t answer certain questions. However, we can look at how much data is missing as well as the patterns of missing values and determine whether missingness is related to the variable itself, other variables in the dataset, or has no apparent pattern. In the following sections, we will go through three categories of missing data that are commonly used in research to explain why data is missing.<\/p>\n
How much data is missing?<\/h2>\n
The overall percentage of data that is missing is important. Generally, if less than 5% of values are missing then it is acceptable to ignore them (REF). However, the overall percentage missing alone is not enough; you also need to pay attention to which<\/em> data is missing. Often you may need to consider deleting cases (participants) or individual variables that are missing a ton of values. This step alone can drastically improve the integrity of your data and reduce the overall percentage of missing values in your dataset.<\/p>\n
\nTypes of Missing Data<\/h1>\n
There are several types of missing data, as we will discuss here.\u00a0 Some types are easy to consider and account for, while others are confusing and may be less obvious to the novice researcher.<\/p>\n
Data Missing at Random<\/h3>\n
In this situation, data is not actually missing at random which makes the name of the category very confusing! \u00a0MAR data happen when missing values are related to another variable in the data set. That is, the missing value (y) depends on x, but not y (itself). Here are some examples:<\/p>\n
Example 1: <\/strong><\/p>\nIn a survey of health care professionals, nurses<\/strong> do not report their age<\/strong>. In this case, being a nurse (x) predicts the missing data for age (y).<\/p>\nExample 2: <\/strong><\/p>\nIn a family survey, single parents <\/strong>do not report their income. In this case, being a single parent (x) predicts the missing data for income (y)<\/p>\nExample 3:<\/strong><\/p>\nEmployees who fear their manager do not report their job satisfaction. In this case, employees might be afraid to report their job satisfaction for fear of reprisal.<\/p>\n
Note that in real life you might find MAR patterns in your data but the rationale behind them is still speculative. Unless you go back and check with the participants it is impossible to prove.<\/p>\n
Missing Completely at Random (MCAR)<\/h3>\n
Missing data that doesn\u2019t have a pattern of missingness is referred to as data missing completely at random (MCAR). This is the ideal situation when you have missing data because missing values are random so any influence they have on your analysis is also random. Here are some examples of MCAR situations:<\/p>\n
Example 1:<\/strong><\/p>\nYou conduct a study on heart transplant patients and discover that 10 patients did not answer 2-3 questions on your survey.\u00a0 The questions that are missing are different for each person and there is no pattern.<\/p>\n
Example 2:<\/strong><\/p>\nYou conduct an RCT comparing the effects of fish oil supplementation versus placebo on anxiety levels in nursing students.\u00a0 1 patient in the control group forgot to take their supplement on Sept 10th because they were busy. 2 patients in the experimental group missed their dose on October 3rd and Nov 19th, respectively.\u00a0 One woke up late. The other one burnt their breakfast and got distracted.<\/p>\n
There is NO pattern causing the missing data.<\/p>\n
Not Missing at Random (NMAR)<\/h3>\n
The last category is data not missing at random. In this situation, missingness is because of the variable itself. In other words, there is a reason why people don\u2019t want to answer that particular question. Usually this happens with sensitive questions.<\/p>\n
Example 1:<\/strong><\/p>\nPeople who are overweight do not report their weight. In this case, being overweight (x) predicts the missing data for weight (x).<\/p>\n
Example 2:<\/strong><\/p>\nSingle parents do not report their marital status. In this case, being a single parent (x) predicts the missing data for marital status (x).<\/p>\n
\nAnalyzing Missing Values<\/h3>\n
The default in SAS is to delete missing values from your analysis. The effect this has on your results depends on how much of your data is missing. SAS offers a number of robust options for dealing with missing data but the focus of this section is on being able to see how much of your data is missing and examine patterns.<\/p>\n
One of the easiest ways to examine missing data patterns is to use the PROC MI command which is the multiple imputation (MI) procedure in SAS.<\/p>\n
<\/code><\/p>\n\n\nThe following code uses the NIMPUTE=0 option to create the “Missing Data Patterns” table for the specified variables.<\/p>\n<\/header>\n
\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = NAMEOFDATASET NIMPUTE=0;\r\nVAR VAR_1 VAR_2 VAR_3;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/code><\/p>\nLet\u2019s do an example together:<\/strong><\/p>\nIn this example you are interested in knowing more about stress levels of caregivers of older adults with dementia. You send out a pilot survey and get an initial sample of 16 people to answer it.\u00a0 The questionnaire includes demographic variables and a five-item questionnaire to measure stress rated on a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of stress.<\/p>\n
\n\nThe first step of course is to set up your data file in SAS:<\/p>\n<\/header>\n
\nOPTIONS PAGESIZE=60 LINESIZE=80 CENTER DATE;\r\nDATA CAREGIVER;\r\nLABEL\r\nID = \u2018PARTICIPANT ID\u2019\r\nSTRESS1 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 1\u2019\r\nSTRESS2 =\u2018CAREGIVER STRESS QUESTIONNAIRE ITEM 2\u2019\r\nSTRESS3 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM3\u2019\r\nSTRESS4 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM4\u2019\r\nSTRESS5 = \u2018CAREGIVER STRESS QUESTIONNAIRE ITEM5\u2019\r\nSEX = \u2018SEX\u2019;\r\nINPUT ID 1-2 SEX 4 STRESS1 6 STRESS2 8 STRESS3 10 STRESS4 12 STRESS5 14;\r\nDATALINES;\r\n01 0 4 3 4 5 4\r\n02 1 3 2 3 4 4\r\n03 1\u00a0\u00a0 3 3\u00a0\u00a0 3\r\n04 0 1 2 1 2 3\r\n05 1 4 4\u00a0\u00a0 5 3\r\n06 1 2 3 3 4 4\r\n07 0 3 3 5\u00a0\u00a0 5\r\n08 1 3 5 4 5 3\r\n09 1 4 4 5 4 4\r\n10 1\u00a0\u00a0 2 4 4 4\r\n11 0 4 3 4 5 5\r\n12 1 2 1 2 3 4\r\n13 0 1 2 4 4 2\r\n14 1 3 4 4 5 4\r\n15 1 3 4 5 4 3\r\n16 1\u00a0\u00a0 4 3 2 1\r\n;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\nNext we use the MI code template but we replace NAMEOFDATASET with the actual name of our dataset (CAREGIVER) and replace the VARIABLE NAMES with the actual names of the variables in our dataset:<\/p>\n
\n\n\nODS SELECT MISSPATTERN;\r\nPROC MI DATA = CAREGIVER NIMPUTE=0;\r\nVAR ID SEX STRESS1 STRESS2 STRESS3 STRESS4 STRESS5;\r\nRUN;<\/pre>\n<\/div>\n<\/div>\n<\/div>\nAs you can see in the figure below, SAS uses this code to produce a table showing the number of cases with each pattern of missing data. First we look at the lefthand side of the table to examine the patterns of missingness in our dataset. In this example, you can see that Group 1 has no missing data because there is an \u201cX\u201d in each of the variables in the dataset. This means that there is data for each variable for participants in this group. The frequency of participants in this group is in the column labelled \u201cfreq\u201d and you can see that there are 11 people with no missing values. By looking at the next column over which is labelled \u201cpercent\u201d, we can see that this represents 68.75% of the sample.<\/p>\n
The next pattern of missing data is Group 2. Looking across the columns, we can see that there is no \u201cX\u201d for stress4<\/strong>. This means that participants in Group 2 answered all the questions except that one. There is only 1 person in this group and they represent 6.25% of the data. We can continue doing the same interpretation for Groups 4-6 in the table.<\/p>\nThis table also provides the means for each of the variables. Again, don\u2019t forget that \u201cmeans\u201d for some variables are not meaningful. For example, the mean values provided for Participant ID and sex should be ignored here. What is valuable though is that for continuous variables you can compare their means for participants with different patterns of missing data. For example, for the variable Stress2, you can see that the mean is the same for Groups 1, 2, 4, and 5 but it is higher for Group 3. Although this is a small sample for illustrative purposes, you can hopefully see how the information in this table can help you understand the patterns of missing values in your data better.<\/p>\n
Output table showing patterns of missing values<\/h6>\n
<\/p>\n
\nFinally, consider that in research we can always expect to have missing data for a variety of reasons. The SAS program provides a powerful platform for calculating data while recognizing strategies to handle missing data.<\/p>\n","protected":false},"author":56,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[47],"contributor":[],"license":[],"class_list":["post-283","chapter","type-chapter","status-publish","hentry","chapter-type-standard"],"part":180,"_links":{"self":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/users\/56"}],"version-history":[{"count":9,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions"}],"predecessor-version":[{"id":285,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/revisions\/285"}],"part":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/parts\/180"}],"metadata":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapters\/283\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/media?parent=283"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/pressbooks\/v2\/chapter-type?post=283"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/contributor?post=283"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.library.upei.ca\/montelpare\/wp-json\/wp\/v2\/license?post=283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}