Things to Consider When Finalizing Your Methodology

Here are the ingredients a methodologist can help you use to construct a good methodology section.

What is your population?

  • Your population is the group of people to whom you intend to generalize the results of your study. 
  • Your exact population will depend on the scope of your study. 
  • Think of your population as being those people to whom your results will apply.  If the purpose of your study is to determine the perceptions of teachers about bullying among middle school students, your population is middle school teachers in Grades 6 through 8.  You can narrow the sampling frame more by delimiting the population to middle school teachers in Grades 6 through 8 in inner-city school districts.

What is the setting?

  • Describe the setting of your study in as much detail as possible. 
  • Use a table to provide an instant graphic. 
  • For instance, you plan to survey middle school teachers in a large inner-city school district.  Provide information on the number of schools, teachers, and students in the entire district, and then the number of middle schools, middle school teachers, and students in the district.  Include breakouts of ethnicity, gender, and socioeconomic status of the schools and students. 
  • Your committee cannot determine if your study is viable if it does not know how many potential participants are available and if those potential participants fit the profile of your population. 

What is the sample size?

  • The sample size is based on the number of variables and the type of analyses planned to answer the research questions. 
  • Which comes first, the sample size or the analysis plan?  You should have a general idea of what variables you want to study, how you will measure those variables, and what type of analysis you plan to use.  With those parameters, you can get a good idea of the sample size you need, using a statistical power program.  However, as adjustments are made to number and type of variables and analyses you want to use, the sample size must be revisited.
  • Included in this section is a description of the type of sampling technique you intend to use.  A nonprobability convenience sampling method is most common in social science research.  Data collected using this method is useful but may be prone to significant bias.  The use of convenience sampling is always part of the limitations of a social science study.
  • Survey research is susceptible to missing data, incomplete responses, and low response rates.  Plan to lose at least 10% of data collected due to missing variables.  A survey conducted with care can anticipate a response rate approaching 50%.  Therefore, a researcher hoping for a sample size of 100 must contact at least 220.  If you cannot access a population with at least 300 potential participants to provide 100 complete responses for analysis, you should reconsider the scope of your study.

What instruments will you use to measure the variables in your research questions?

  • Use instruments/questionnaires already in the field.  Do NOT develop a new instrument.  Determining the validity and reliability of a new instrument is a study of its own.
  • If the instrument is more than 5 years old, cite current research that shows it is still of value in your field of study.
  • Cite validity and reliability information provided by authors.  If possible, cite current reliability data obtained by researchers other than the instrument’s authors.
  • Describe the scale(s) in the instrument.  Does the instrument contain more than one scale? 
  • Do you plan to use only a few scales instead of the entire instrument?  If so, describe the entire instrument, but then focus ONLY on those scales you plan to use in your study.
  • Reference the appendix where a copy of the instrument is located or create a table to list the items in each scale.  Do not make your committee members look for your instrument and its items.  (Note: Some instruments are proprietary.  Although you have been given permission by the authors to use the instrument, they may not give you permission to include all the items in your dissertation.  Instead, provide examples, with the authors’ permission, of course.)
  • Describe how you plan to operationalize each variable in your research questions.  For instance, you want to measure depression using the depression scale from XYZ instrument.  It has 10 items, with a Likert-style response range from 1 (extremely unhappy) to 6 (extremely happy).  The responses to each of the 10 items are averaged and the score for your variable (depression) will range from 1 to 6, with higher scores indicating less depression.  Or, the responses are summed, and the score will range from 10 to 60, with higher scores indicating less depression.
  • Be specific about which items are reversed.  If the scale discussed in the previous bullet point has 3 items that must be reversed scored, indicate which items are reversed in the table, mention it in the text, and continue with the rest of the discussion about scale and range of scores.

What demographic variables will you collect?

  • Will some of the demographic characteristics of the participants be some of your research variables?
  • If demographic characteristics such as sex, age, years of experience, or region of the country are variables you plan to use to answer your research questions, be sure to operationalize them to minimize the number of groups.  For example, sex is usually dichotomous and is simple to use in t tests, analyses of variance, and regression/correlation.  However, age and years of experience should be collected as a continuous variable (see Top Three Mistakes People Make When Writing Questionnaire) to facilitate correlation and regression analyses.  Age and years of experience can be recoded into categories, but age and experience categories cannot be recoded into ranges.  
  • Be careful using a research variable that many categories.  The United States, for example, has a lot of regions.  Conducting an analysis using all those regions can be clumsy and difficult to interpret.  However, if you plan to isolate one or two of those regions for specific analysis, go for it!
  • Will the demographics only be used to describe your sample?  In that case, ask questions that provide information you can use to compare your sample to samples in other research or to the information your provided in your description of the setting.  Only ask demographic questions you plan to use in the study.
  • You do not have to collect the same demographic information collected by the authors of your instrument(s).  You are only interested in the scale(s) in the instrument.  The demographic questions used in the original instrument are not part of your study.

How do you plan to protect the participants in your study?

  • Archival student data does not require permission from students or their guardians.  Nor do secondary databases.
  • Institutional reviews boards require that adults informed about the study—usually through a form that provides information about the study and who to contact with questions.  However, implied consent is provided when adults respond to a questionnaire; therefore, no signature is required on a consent form.
  • Cite the Belmont Report (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1976) and report how you plan to meet its basic ethical principles.

What procedures will you use to collect your data?

  • The procedure section of your methodology is a step-by-step plan that describes how you intend to collect the data to answer your research questions. 
  • It is important that you make inquiries to determine if you can pull off your plan.  Clients have told me that they have been admonished to not collect data until they have permission.  Yes, that is correct!  However, you must discuss your proposed study with relevant people at your study’s setting before you make the decision to conduct your study.  You must rethink your study if you cannot be assured that the district will allow your study, or the way you plan to collect the data is not feasible, or there are not enough potential participants in your sampling frame.  If you feel good about the feasibility of your study, then proceed.
  • If the data you want to use is archival, who do you intend to work with to download the relevant data?  What parameters do you need to ensure their database queries provide the appropriate data for you?  In what format do you want the data downloaded?
  • If you want to conduct a survey, how do you plan to access your potential participants?  (See How to Improve Your Response Rate When You Conduct a Survey)
  • If you want to reach the middle school teachers in your inner-city school district, how will you do that?  Is there a listserv, a Facebook group, or a Twitter Twibe for middle school teachers in your school district?  Will someone in human resources provide you with a list of the email addresses of all middle school teachers in the district.  Probably not (for a host of reasons).  If they do, can you be assured that the district’s email security system will not consider your Survey Monkey or Qualtrics questionnaire spam and refuse to send it to your potential respondents’ email inboxes?
  • Does your district have large meetings for middle school teachers?  Can you get on the agenda and make a personal appeal for the teachers’ participation?  Will you have to seek permission from each middle school principal to attend a faculty meeting and personally make your appeal and hand out paper questionnaires?  How many weeks will it take to survey the sampling frame you selected and collect the number of responses specified in your power analysis?  Be realistic, double the amount of time you think it will take. 
  • How will the data be inputted into a database for analysis?
  • The procedure section must contain a detailed description of the points presented above.  Your committee must be provided with the details so they can consider if the study is viable.  The readers of your finished dissertation want to know what procedures you used to collect your data.

What analyses will you use?

  • This section is at the heart of what a quantitative methodologist provides to doctoral students.  She reads the draft of your proposal, matches the research questions with your variables of interest, determines the viability of your study, and then asks you questions.
  • Based on the answers to your questions, the methodologist can determine the complexity of the statistical analysis you need to answer the research questions. 
    • Do your research questions require only descriptive statistics? 
    • Do the research questions require a type of analysis that you or your committee will find too hard to interpret? 
    • If a simpler analysis can be used, do the research questions need to be reworded? 
    • Will you feel comfortable defending the results using the type of analysis she suggests?

A good methodologist helps you finalize the methodology section of your dissertation.  She then stays with you until the data are analyzed, the results are written, you have interpreted the findings, and you are ready to defend.