Factor Analysis of the Protective Factor Index (PFI) SEL Assessment Instrument
This webpage discusses an article entitled “Development and Factor Analysis of the Protective Factors Index: A Report Card Section Related to the Work of School Counselors” by Gwen Bass, Ji Hee Lee, Craig Wells, John C. Carey and Sangmin Lee. The article was published in The Professional Counselor, Volume 5, Issue 4, pages 516–528. It can be accessed online at:
Purpose of Study
The article describes a factor analysis conducted on a Protective Factors Index” (PFI) used in a pilot program in elementary schools in an urban-suburban school district in the northeastern United States. The PFI was developed to identify and evaluate socio-emotional factors relating to students’ academic performance and emotional health, and use the data to inform the work of school counselors. The factor analysis sought to determine the potential of the PFI to become an efficient and accurate way for school counselors to collect data from teachers about student performance and improve school counselors’ ability to support students’ academic achievement and well-being.
Factor analysis is a statistical procedure for examining the relationship of correlated variables in observed data in order to determine which variables, or factors, are most useful in interpreting and using the information in data-based decision making.
The variables focused on in this analysis consisted of the 13-item Protective Analysis Index scale used by teachers to record their observations regarding students’ social-emotional development. Typically, such data is not collected and available to school counselors to make informed decisions regarding students’ social-emotional issues, and to select appropriate interventions for effectively addressing these issues.
The PFI consisted of 13 items that are organized into four groups based on the CBA constructs: motivation (4 items), self-direction (2 items), self-knowledge (3 items) and relationships (4 items). The items address developmentally appropriate skills in each of these domains. The format for teachers to rate their students consisted of dichotomous response options: “on target” and “struggling.”
This article describes both the development of the teacher-rated Protective Factor Index (PFI) scale used to assess students’ social-emotional development, and the results of an exploratory and confirmatory factor analysis to determine which variables were most useful in interpreting the data. The results of the factor analysis are important because: a) they validate the items used in the PFI, confirm which factors have the greatest potential for providing school counselors with meaningful data to more accurately profile students’ social-emotional needs, and c) provides school counselors with a more simplified process for interpreting and applying the PFI results in both prevention and intervention school counseling contexts. The analysis is also important as it helps to validate the use of the CBA’s four foundational constructs (motivation, self-direction, self-knowledge, relationships) as a useful conceptual framework for the PFI.
The data generated by the PFI significantly contributes to school counselors being able to use data-based decision making and become leaders in data discussions in their schools, particularly in the area of enabling school counselors to become proactively involved in interpreting students’ academic, social-emotional and behavioral data and determining appropriate interventions in collaboration with teachers and other professional support personnel.
The research team grouped the 13 items into two, three and four factor measurement models to determine which worked best to explain the data in terms of efficiency and interpretability. The models were compared and it was determined that the three factor model had the best fit with the data. The three factors were: a) academic temperament, b) self-knowledge and c) motivation. These are factors that are demonstrated to be foundational to school success.
Although there is not a perfect alignment, the PFI factor structure is consistent with the CBA model for clustering student competencies. In addition, it corresponds to previous research on the links between construct-based skills and academic achievement. As stated in the article, “These results indicate that the PFI may be a useful instrument for identifying elementary students’ strengths and needs in terms of exhibiting developmentally-appropriate skills that are known to influence academic achievement and personal well-being.”
The PFI is Useful
The results of the factor analysis suggest that the PFI generates meaningful data that can be used in data-based decision making and evaluation. The data can be used to provide targeted academic and social-emotional support for elementary students. The PFI can be completed in conjunction with the academic report card and it is minimally time-intensive for teachers.
The analysis showed that the PFI results, reported along with academic data on the students’ report cards, could be a very useful source of information for parents—helping to support school-home interactions and increasing family-engagement in interventions. The study showed that the PFI could have considerable impact on school counselors’ abilities to identify students’ social-emotional needs, and provide group-based interventions to address specific student needs.
In addition, using the PFI can significantly impact counselor-teacher interactions by enabling counselors to work with teachers to identify and address challenges students face in the classroom. Further, counselors can help teachers evaluate the impact of their interventions and advocate for increased support for student learning from school and district leaders. Since the PFI is based on constructs that are strongly related to students’ academic achievement and well-being, and is generated by school counselors, the counselor-teacher collaboration significantly increases the school’s ability to meet the developmental and social-emotional needs of its students by improving their ability to learn and their learning outcomes.
The PFI is designed to gather social-emotional data in order to assess students’ development of skills that correspond to the CBA’s achievement-related constructs. These data help counselors understand the competencies that underlie achievement as they provide interventions that seek to increase learner outcomes through development of these competencies.
School counselors are trained and have expertise in addressing skills that fall within the domains of the three-factor model: academic temperament, self-knowledge and motivation. This information could also be used to measure correlations between PFI competencies and achievement to demonstrate the relationship between academic and social-emotional data, and how school counselors impact students’ academic outcomes.
As stated in the article, “The findings offer great promise in terms of practical implications for school personnel and parents. This analysis quite clearly illustrates “academic temperament,” “self-knowledge” and “motivation” as factors that are demonstrated to be foundational to school success. The results indicate that the teachers’ ratings of students’ behavior align with findings of existing research and, thus, that the instrument is evaluating appropriate skills and constructs.”
Further Study is Required
The results of this factor analysis are indeed promising, but as is the case with any new approach in the initial phases of its implementation, the PFI will require additional research to confirm the results and address limitations of this study. Additional research would include studies to confirm the reliability of the PFI teacher rating scale and inter-rater reliability.
Subsequent research can also explore how the current rating scale might be improved, such as whether reliable sub-scales could be developed to increase the validity and utility of the PFI. The purpose of the PFI, after all, is to help determine students’ social-emotional needs, decide on appropriate interventions and monitor the progress of students in this critical area of learning. How PFI data relates to results from other forms of measurement, such as academic performance, is important in trying to determine the efficacy of the PFI as a universal screening tool for social-emotional development. The impact of the PFI on school counselors’ use of data-based decision making should also be explored.
Establishing the validity and reliability of the PFI is a multi-dimensional process that requires the evaluation of each dimension. Among these dimensions, the following are noted:
- Teacher ratings in the pilot were set up as dichotomous options. Changing the format to a Likert scale could be explored to evaluate whether that would allow teachers to evaluate students with greater specificity. Such a change could potentially help in the development of subscales.
- A developmentally-appropriate rubric was developed for the pilot to assist teachers in rating their students. The criteria could be evaluated in terms of the relevance to their assigned development level and usefulness in helping all teachers consistently make valid ratings.
- The current PFI used in the pilot focused on elementary schools. Research needs to be conducted that examines the impact of using a PFI customized for middle and high schools. Research can also be conducted in a variety of contexts beyond the urban-suburban setting of the pilot, and also with a variety of demographics.
The bottom line for all research studies is to determine whether the PFI, in its current or modified form, fulfills its intended purpose of providing a universal assessment tool for generating meaningful data related to students’ social-emotional needs, deciding on appropriate prevention and intervention activities, and evaluating the impact of these activities on students’ academic achievement and well-being. Studies will also need to establish that the PFI is an efficient, cost-effective tool for schools to use.