The strategic objectives for the S-CAFH study were two-fold: (1) to create a study sample of 1,000 households representative of residential areas within New Jersey exposed to Hurricane Sandy, and (2) to have sufficient numbers of cases within the sample for sub-group analyses that can be conducted of “high” damage versus “not high damage” areas, “northern” versus “southern” regions, and households with low income versus all other income levels.    Addressing the first objective enables us to estimate population-level impacts and needs across the hardest-hit areas of the state.    Addressing the second objective enables us to examine the extent to which New Jersey residents’ decisions, needs, health effects, and recovery may be explained by the damage they were exposed to, by regional differences, and by access to economic resources.    To accomplish these objectives, we defined an area within New Jersey that was exposed to the storm (referred to as the “S-CAFH Disaster Footprint”), and developed a multi-stage stratified sampling design to yield sufficient numbers of cases for sub-group analyses.    Sampling and post-stratification weights were developed and applied to the data once sampling and data collection were complete.   The various elements of this approach are described in more detail in this appendix.

• Disaster Footprint:

  The S-CAFH Study was designed to examine the impact of Hurricane Sandy on the Disaster Footprint presented in Appendix A Figure 3.    Approximately 1,047,000 people—including about 411,000 households—live within this geographical area.   The Disaster Footprint covers an area approximately 14% of the state, and that the population represents about 12% of the state.   The disaster footprint was created based on three criteria:

  • The Hurricane Sandy Impact Analysis by the FEMA Modeling Task Force (MOTF) was used to identify the nine counties in New Jersey with a “Very High Impact” rating.   The FEMA MOTF impact model is a composite of storm surge, wind, and precipitation.   These very high impact counties had a population of over 10,000 persons exposed to storm survey in addition to more than 8 inches of precipitation during the storm and an estimate of over $100M in wind-related damages.   The counties that met these criteria included Atlantic, Bergen, Cape May, Essex, Hudson, Middlesex, Monmouth, Ocean and Union.
  • Once these nine counties were selected, the study team developed a sampling frame using a geographic information system (GIS) based procedure.   Storm surge within the nine counties was identified using FEMA storm surge raster data based on satellite imagery and further filtered to include all areas with storm surge of greater than or equal to one foot.   Housing damage data was acquired based on FEMA damage assessments.   These data were available for the majority of housing lots in high impact zones.   Lots which were classified by FEMA as minor (Full Verified Loss of $5,000-$17,000), major (Full Verified Loss of more than $17,000), or destroyed (indicated by an Individual Assistance (IA) inspector) were aggregated at the census block group level.   Block groups with at least 20% of all assessed units having one of the prior three classifications were then selected for inclusion in the study.   FEMA Individual Assistance data were acquired at the ZIP code level.   Valid registrations were summed and standardized (z-score) for the ZIP codes in the nine counties and those which summed to greater than the mean (a z-score of >0) were selected to be part of the footprint.
  • Finally, these three resultant geographic layers were superimposed upon one another and any census block group, which intersected any one of the three layers was selected to be included in the final Disaster Footprint.
  • In summary, then, the Disaster Footprint within the nine high impact counties is composed of:
    • Census block groups which experienced a storm surge of at least one foot, OR
    • Census block groups in which at least 20% of all housing units sustained “Minor Damage,” “Major Damage,” or were “Destroyed,” per FEMA assessments, OR
    • ZIP codes which reported a greater than average number (z-score >0) of valid FEMA Housing Assistance registrations.

• Sampling:

  When conducting a household survey, researchers often use a random sample, which is a subset of individuals that have been randomly selected from the population. Sometimes, because researchers cannot ask survey questions of every member of the population—at least in heavily populated areas such as the one where we were working—a smaller subset of people is drawn at random that is intended to be representative of the larger population. We first determined the target number of New Jersey residents to be sampled by calculating the number necessary to have sufficient power in the sample, which would allow us to detect meaningful differences on key characteristics. In other words, there had to be enough people randomly sampled who could potentially exhibit a given characteristic to detect statistically significant differences between groups. Therefore, the research team determined that we needed a target sample size of 1,075 respondents.

  • One approach to selecting study respondents is to conduct a simple random sample, in which all the households within a given area of interest, in this case the Disaster Footprint, would be enumerated and then 1,075 of them, would be “picked out of a hat.” Although this selection strategy does provide the basis for estimating the characteristics of the entire population within the Disaster Footprint, it would not have guaranteed that there would be enough cases in the sub-groups of research interest – particularly those households that suffered varying degrees of damage or that were living in lower socio-economic neighborhoods. Thus, it also would not allow our team to make estimates that were reliably representative of these smaller populations.
  • An alternative approach, which our team ultimately employed, was to first group the “neighborhoods” (census block groups) into different strata, such as neighborhoods in the north, or neighborhoods that suffered considerable housing damage, or neighborhoods that were composed of households living at or below a poverty threshold. Once this grouping was completed, we could then randomly select households within these strata and make sure that there would be enough households to be representative.

• Field Effort:

  S-CAFH Field Team members conducted face-to-face and phone surveys with residents living in the Disaster Footprint between August 2014 and April 2015.   Interviewers were rigorously trained over the course of five days on field protocols and on how to utilize mobile technology to conduct the survey.   Team members were assigned to work certain census block groups and led by one of three team captains who were primarily responsible for managing the field effort.

  • The field team started working each census block group with a list of ordered addresses per block group.   To be eligible to participate in S-CAFH, sampled respondents had to be the primary resident of the household at the time of the storm.    The field team attempted to survey the first 25-50 addresses on that list.   Any given visit to a household could result in a variety of outcomes that the team member documented through a status code for the rest of the staff.   These status codes included the following:
    • Complete: Respondent has completed the entire interview.
    • Incomplete: Respondent has completed portions of the interview but not the entire interview.
    • Not Available: Respondent answers the door but does not have time to complete the interview.    Interviewer should attempt to schedule future appointment with respondent to complete the interview.
    • Soft Refusal: Respondent answers the door but has low interest in completing the survey. Interviewers should attempt to persuade respondent and flip the case.
    • Hard Refusal: Respondent answers the door and it is clear that he or she does not have any interest in participating in the study.
    • No Answer: Respondent does not answer the door.
    • Ineligible (needs follow-up from captain): Respondent was not primary resident at the time of Hurricane Sandy.   No contact information is given so interviewer should return the case to the team captain for tracking and tracing.
    • Ineligible (has contact information): Respondent was not primary resident at the time of Hurricane Sandy.    Interviewer is able to obtain contact information on primary resident/owner at the time of Sandy.
    • Bad Address: Address given to interviewer does not exist.    Please note that this is different from finding a vacant home/lot.
    • Vacant (needs follow-up): Interviewer arrives at sampled address to find a slab or uninhabitable/vacant home.    This case should be returned to the team captain for tracking and tracing.
    • No access: Interviewer arrives at sampled address to find a gated area or other barrier to physically obtaining entrance to the property.    This case should be returned to the team captain for tracking and tracing.
• If there was no answer or the respondent was not available, the interviewer returned to the household to complete five expected visits before the address was retired by the captain and the interviewer proceeded with working another randomly sampled household.    A new, randomly sampled address within the block group was then put into the address list to be worked by the interview team.

• Weighting:

  Even when random sampling has been used, it is important to compare the resulting survey data to population data, to see whether it is representative of the population.   When the resulting data is different from the population level estimates, weights are often applied in order to allow researchers to generalize the results of that data to the population as a whole.   Surveys often have imperfections due to various real-world conditions which can bias population-level estimates, so these sampling weights are also used to refine such imperfections within reasonable margins of error.

  • The S-CAFH weighting protocol used sampling weights that (1) compensate for unequal probabilities of selection such as damage (see above), (2) compensate for non-response, and (3) adjust for weighted sample distribution among key variables of interest.   Specifically, base weights were calculated to map S-CAFH respondents to the total footprint population; subsequently, adjustments to the strata (geography, damage, and poverty) were made to reflect proportional distributions in relation to census block group characteristics.   In addition, potential bias due to non-response was compensated by examining differences between target and sampled households in the strata; hard-to-reach housing units were adjusted by applying a correction for areas with high prevalence of vacant rental housing units.   Adjustments were also made for gender, age, and households with children so that they reflect population distributions.   Standard guidelines and techniques for constructing weights were applied in making these adjustments.  The overall 95% sampling error based on these adjustments is about 4%.