INTRODUCTION
The decision to return an athlete to sport is a complex clinical judgement requiring careful and deliberate consideration of both physical and psychological factors. Physical factors (e.g., range of motion, strength, coordination, and agility) are objectively quantified, and often can be compared to the individual’s contralateral side as well as the norms generated from standard objective measures. All this clinical information is used to judge physical readiness from a purely physiological perspective. However, there is increasing interest in the psychological aspects of rehabilitation within the physical therapy community and throughout sports rehabilitation, most notably among these, fear and self-confidence.1–4 Moreover, there is ample evidence that psychological factors also make a critical contribution to motor learning to support this growing interest relative to return to sport.5–8
Psychological components are often considered to be difficult to quantify due to their “subjective” nature. However, there are multiple objective measures that have been developed and used across the professions concerned with patient readiness for returning to sport, including the Injury-Psychological Readiness to Return to Sport Scale,9 and ACL-Return to Sport after Injury (ACL-RSI) scale.10 While these measures have clinical utility and are certainly appropriate to use based on their psychometric properties, they are limited either by encompassing all sports generically (I-PRRS) or targeting diagnostic-specific conditions based on the anatomical site of injury (ACL-RSI). Without reference to a specific sport and the particular movements required to be ready to return to that sport, these instruments are helpful but ultimately insufficient to support clinical decision-making. The present study reports on the development of a suite of self-report modules for selected sports to address these limitations and enhance the process of making clinical judgments about adolescent athletes.
The parent instrument from which these modules were developed is the Adolescent Measure of Confidence and Musculoskeletal Performance (AMCaMP). The AMCaMP is 22-item self-report instrument, which was developed as a “core” measure for individuals ages 11 to 18 years of age, characterizes an adolescent’s confidence in the ability to perform general movements (sitting, standing, walking, running, etc.) that are essential to daily life irrespective of sport.11 Similar to the assessment for adults on which it was based,12 the AMCaMP is rooted in the self-efficacy theory developed by Bandura.13 This theory proposes that the situation-specific beliefs which a person holds about the capability to perform specific tasks help to determine what tasks the individual will choose to do, the energy and attention that will be devoted to doing it, and the perseverance that will be displayed in order to execute a specific level of performance when confronted by barriers to success. This theory can be summarized by the familiar saying, “If you think you can, then you can, and if you think you can’t, you’re right.” The psychometric validation of the reliability and validity of the AMCaMP indicated it demonstrated acceptable internal consistency and established a minimal detectable change threshold for documenting clinical progress and outcomes in adolescents.11 The purpose of this study was to test the psychometric properties of sports-specific modules to supplement a general measure of movement self-efficacy, the Adolescent Measure of Confidence and Movement Performance (AMCaMP). These modules were developed to assist the overall clinical determination of readiness to return to play.
METHODS
Instrument
The sports-specific AMCaMP is designed to be an adjunct to the core AMCaMP questionnaire. It is a patient-centric measure to assess confidence in sport-specific movement patterns. The questionnaire presents items on a Likert-type array of five response levels progressively ranging from “no confidence” (1 point) to “fully confident” (5 points). The array of items was selected to explore self-confidence in performing movement patterns particular to each of 13 sports: soccer, baseball, softball, basketball, football, gymnastics, cheerleading, lacrosse, swimming, volleyball, tennis, cross country, and dance (See Appendices 1-6). For example, full court backpedaling is a particular movement that might regarded as essential to returning to basketball. The sports were chosen based on high rates of participation and on the volumes of participants at the survey test sites. A “not applicable” option was available for any item that would be unnecessary for an athlete to have confidence as it is not a requirement of their performance. For example, a baseball pitcher would not need to assess his confidence in the movements essential playing the catching position.
Study Sites and Participants
Data were gathered from patients who were referred to physical therapy for sports-related injuries from 14 outpatient clinics in the Children’s Healthcare of Atlanta system. The institutional review board at Children’s Healthcare of Atlanta deemed the study exempt. Traditionally the onset of puberty is accepted at the beginning of adolescence, which the authors operationalized as 11 years of age. All therapists at each site were instructed on who was eligible to participate in the study and how to collect the data. Any new patient was eligible to participate in the study if the patient was: 1) 11 to 18 years of age; 2) spoke and read English; and 3) had the cognitive ability to complete the questionnaire independently. Demographic data were also collected on the first visit.
Item Selection and Development
An expert group of eight physical therapists with an average of 11 years of experience (range 5-20 years) and at least two years of sports therapy experience participated in providing items. Of these, four were board certified in sports physical therapy and all 11 played competitively in high school, club, or collegiate sports. A board certified pediatric orthopedic surgeon and a board-certified primary care sports medicine physician with a combined 50 years of experience were consulted as well. Suggestions on critical movements for each sport and in some cases for specific positions within the sport were solicited. Consensus was reached when 6 out of the 8 panel members deemed a movement critical. Additional consideration was given to panel members with expertise in a specific sport (e.g., if a panel member played college football). The final item count was as follows: 17 for soccer, 16 for baseball/softball, 16 for basketball, 16 for football, 15 for ballet, 14 for gymnastics/cheerleading, 10 for swimming, 10 for volleyball, 9 for lacrosse, 9 for tennis and 7 for cross country.
STATISTICAL METHODS
Data Preparation
A total of 14,744 patients were potentially available for study aggregating the data from all sports. Each sport was separated into its own individual cohort and each of these datasets were evaluated independently for the analysis reported below. Observations were not dependent because there was only one questionnaire per patient.
Outlier Analysis
Outliers were assessed using the Mahalanobis Distance. This distance corresponds to the distance between an observation and the centroid of all observations in the space of questionnaire items. If this distance exceeded the threshold outlined by the α=.001 significance level, we excluded the observation.14 Because quality of response is also an issue of concern when analyzing response data, the Intra-Individual Response Variability (IRV) was used to exclude patients. Patients with an IRV value in the 99th percentile or above were judged as having haphazard/random response patterns and were subsequently dropped from the study.
Additionally, responders with zero variance in their responses (ex: responded all 5’s for all items) were dropped.
Collinearity Diagnostics
Because collinearity/multicollinearity violates key assumptions of factor analysis, collinearity diagnostics were conducted to determine if there was potential redundancy amongst survey items primarily using condition index and the determinant of the response matrix. A condition index above 30 was used as an indicator of collinearity, but an index above 20 was still scrutinized. In addition to the condition index, this function provides variance decomposition proportion associated with each condition index. Items that both had a proportion of over .5 for the same index were considered candidates for deletion. Once collinearity was no longer deemed a problem, patients who did not have at least two responses in the remaining items were dropped.
Data Analysis
An exploratory factor analysis (EFA) was conducted to identify a hypothetical latent variable structure among questionnaire items and confirmatory factor analysis (CFA) to verify this structure. Not all sports had a sufficient sample size for factor analysis. A minimum of 400 participants15 and 10 participants per item16 were required for each sport (200 for EFA and 200 for CFA).
Factor analysis suitability was assessed using Bartlett’s Test of Sphericity17 and the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO).18 Bartlett’s Test examines the correlation matrix of all questionnaire items under the assumption that there is no relationship between the items. Rejecting this assumption suggests the data is suitable for factor analysis. The KMO Measure represents the proportion of variance among items that may be common variance with values above 0.7 suggesting factor analysis would be appropriate (where 1 is the best possible score).
A set of plausible number of factors for each sport was then determined by using the Kaiser Criterion (KC),19 the Scree Test,19 Parallel Analysis (PA)19 and the Hull Method.20 Finally, sed unweighted least squares and a promax rotation when applicable for the EFA were used. The metrics of success for a hypothetical model were strong loadings on an items primary factor (> 0.6), weak cross-loadings for an item’s secondary factors (< 0.3), and a high proportion of the total variance explained by the model (> 60%).
Participant data that were used in the EFA were not used in used in CFA. Model validity was assessed using the fit indices Root Mean Square Error of Approximation (RMSEA), the Standardized Root Mean Square Residual (SRMR), the Comparative Fit Index (CFI), and the Tucker-Lewis Index (TLI).21 The RMSEA is the difference between the observed covariance matrix per degree of freedom and the hypothesized covariance matrix with an acceptable value being below 0.08. The SRMR is the average of the standardized residuals between the observed and hypothesized covariance matrices with acceptable values being below 0.06. TLI and CFI are not especially sensitive to sample size with high values > 0.97 indicating the hypothesized model is better compared to the independence model.21 Although the chi-square test can be used to evaluate model fit but is not included because when fitting ordinal factor analysis models, this test can have inflated and/or unreliable Type I error rates.22
Lastly, the reliability of the factor scales using Cronbach’s alpha, Standard Error of Measurement (SEM), and Minimum Detectable Change (MDC) was assessed. Cronbach’s alpha was used as a proxy for internal consistency and is considered to be a scale of reliability. Values above 0.8 were set to be considered acceptable. The SEM examines the spread of measured test scores around the estimated true score. The MDC measures the minimal amount of change in score that rules out measurement error.
RESULTS
Based on the tests for sample size, seven sports had a sufficient sample size to proceed: baseball/softball, basketball, gymnastics/cheerleading, football, swimming, ballet, soccer (Table 1). All of these sports passed Bartlett’s Test, and the minimum KMO among the sports was 0.798 (Table 2).
The analysis of the candidate number of factors using all four methods ranged from 1 to 4 depending upon the sport (Table 3). Each sport was evaluated under the same conditions for all plausible numbers of factors.
Based upon the criteria that had been set to determine a successful model, and the pragmatic considerations of clinical utility (i.e., information that most assists clinical decision making and carries a low response burden), a factor structure for each sports module was selected based on strong primary loading, weak cross loading, and percent of variance explained. Results indicated that two factor structures were optimal except for baseball (one factor) and soccer (3 factors). The proportion of variance, interfactor correlation, and breakdown of the items contained within a factor for each sport module are found in Table 4.
Although the football, ballet, and soccer modules failed at least one of the criteria we had set a priori for the EFA, these deficiencies were not sufficient violations to exclude these item groups from CFA. Out of all sports modules tested, only swimming did not have a model that met all a priori requirements. While all fit indices were satisfactory for the two-factor swimming model, the RMSEA was slightly higher for this sport module than the recommended 0.06 cutoff value (Table 5).
The summary of reliability testing (Cronbach’s alpha, SEM, and MDC) are displayed in Table 6.
DISCUSSION
Confirmatory factor analysis was used to determine valid scales for items to be grouped. The reliability metrics suggests that most factors provide adequate measures and could be used to detect improvement over time. While these findings provide a latent variable structure for each sport, it does not provide information on what these latent variables actually are. Although all the items grouped in a factor for each sport demonstrated acceptable psychometric properties, a potential trade-off between statistical performance and clinical utility must be acknowledged. While a different set of items might have performed statistically better, such items might be most informative to clinicians making decisions. The methods by which we included items generated by clinicians and set a priori statistical requirements that were met in all but one scale suggests that this trade-off was successfully negotiated.
In the process of identifying the latent variable structure of these sport specific questionnaires, our approach filtered out several potential problems. Eliminating collinearity helped to ensure that each item is measuring a unique component of its respective construct. Eliminating items that load weakly helps ensure all items are relevant to the constructs of interest. The remaining items are then able to give a much clearer reference point of the construct they are measuring. Additionally, the high Cronbach’s α suggests the psychometric property of reliability for each sport module is strong.
An athlete’s body must be prepared to handle all the stressors of the sports to which they will return, physical and psychological. Given the growing awareness of the impact of self-efficacy and other psychological constructs on rehabilitation and recovery, the need for objectives measures of what was traditionally regarded as unmeasurably “subjective” has also grown. However, a common limitation of existing instruments was the inability to capture the mental readiness of the athlete in the critical context of the specific movement requirements of an athlete’s particular sport, which is essential to rendering sound clinical judgments tailored to the individual’s goals. The Sport-specific AMCaMP modules are patient-centric tools that capture the patient’s point of view (See Appendices 1-6). Furthermore, because self-efficacy is highly predictive of what an athlete actually will do once leaving clinical care, the modules yield highly relevant data on the specific requirements which will be the criteria for determining the success of rehabilitation to achieve to sport. Together, the Sports- specific AMCaMP and its parent the AMCaMP can provide a broader array of data to support clinical decisions.
Limitations
Sport specialization is a limitation with this analysis that was not present in the original AMCAMP publication. Because all items were related to basic movements/functions in the original AMCAMP questionnaire, they were likely to be relevant to all patients. However, in sports, team sports in particular, certain players may specialize in a certain kind of role that is only applicable to players of their position/specialty. Future instrument construction should consider developing separate position-specific modules for each sport to address this limitation.
CONCLUSION
The psychometrically validated modules of the Sport-specific AMCaMP offer distinct advantages in evaluating an adolescent’s confidence in readiness to return to a specific sport. Combining a more complete understanding of the psychological context of each athlete’s confidence in performing specific movement requirements with more traditional physical data will better enable clinicians to assist adolescent athletes in successfully transitioning back to their sports.