Reliability of the Cutting Alignment Scoring Tool (CAST) to Assess Trunk and Limb Alignment During a 45-Degree Side-Step Cut

INTRODUCTION

Anterior cruciate ligament (ACL) tears are one of the most common injuries reported in the United States with 250,000 cases reported annually.¹ Over the last two decades there has been a rise in ACL injuries in the pediatric and adolescent population, many occurring during sports participation.² The mechanism of an ACL injury is commonly classified as contact or non-contact in nature, with the majority of injuries in skeletally immature athletes occurring via a non-contact mechanism.^1–4 ACL injury risk in young athletes appears to increase sharply during the growth spurt (12-14 years of age for girls and 14-16 years of age for boys) and peaks shortly after in adolescence.⁵ A side-step cutting (change of direction) maneuver performed during sport has been associated with a large proportion of non-contact ACL injuries.^4,6–9 Cutting maneuvers have been analyzed to better understand the mechanism of an ACL tear. Increased knee valgus loads, elevated knee abduction moments, abnormal trunk lateral flexion angles, decreased knee flexion angles, and increased lateral foot plant have been identified as predictors of ACL injury during a cutting task.^9–12 In a cross-sectional study of Division 1 Norwegian female handball players, increased cut width, abnormal knee valgus, and toe landing pattern were identified as the strongest predictors of knee abduction moments.¹¹ In addition, larger medial to lateral distances between the center of mass and center of pressure has been shown to be associated with higher internal knee adduction moments.¹³ The aforementioned risk factors were identified with three-dimensional (3-D) motion capture, which is currently considered the gold standard for evaluating human movements.^9,14,15 However, equipment cost, operational complexity, and time-consuming data processing requirements make its clinical use impractical. Given the limited availability of 3-D motion capture, two-dimensional (2-D) video analysis has been identified as a clinically relevant alternative. Frontal plane knee valgus angles, measured during a side-step cut and side jump maneuver, were found to have a moderate positive correlation (R²= 0.60) with hip internal rotation angles derived from 3-D analysis.¹⁶ Additionally, numerous qualitative scoring systems, based on 2-D videos, have been developed and reported as reliable and valid methods to assess trunk and lower extremity biomechanics during squat and landing tasks.^14–18 The Qualitative Analysis of Single Leg Loading (QASLS), was found to have excellent reliability and validity when compared to 3-D motion captured kinematics during single-leg squatting and landing.¹⁴ Similarly, good to excellent reliability was reported for the tuck jump assessment, a repetitive plyometric jump maneuver.¹⁷ The Landing Error Scoring System (LESS) has also been shown to be a reliable and valid tool for identifying high-risk movement patterns during a jump-landing manuever.¹⁸

Despite the high percentage of non-contact ACL injuries that occur during cutting tasks, qualitative evaluation tools of cutting movement with 2-D video are substantially limited. Recently, the Cutting Movement Assessment Score (CMAS), a qualitative scoring system, to evaluate a 90-degree cutting maneuver, was introduced.¹⁹ The CMAS was found to be a reliable and valid tool to assess risky movement patterns during a cutting task in college-aged athletes.¹⁹ Additionally, Weir et al assessed the reliability and validity of a 2D video- based screening tool to predict peak knee moments during an unplanned 45-degree side-step cut in a group of junior (age = 15.1 ± 1.2 years) and senior (age = 22.1 ± 2.3 years) elite female field hockey players.¹⁵ In contrast to the CMAS, the screening tool presented by Weir et al involved 2D kinematic measurement of frontal and sagittal plane variables using video analysis software and reported poor to excellent intra-rater and inter-rater reliability.¹⁵ For both levels of athlete, peak trunk lateral flexion, peak hip abduction, knee flexion angle, and trunk flexion ROM were significant predictors of peak knee valgus moments.¹⁵ However, junior and senior athletes demonstrated different movement mechanics.¹⁵ Juniors demonstrated higher dynamic knee valgus and wider foot plant, while seniors showed higher peak trunk lateral flexion, peak hip abduction and knee flexion ROM.¹⁵ These results suggest that pre-adolescent and adolescent athletes may present with different risk profiles during cutting maneuvers than older athletes and thus may benefit from age specific screening tools. To the authors’ knowledge, there have been no studies that assessed a cutting maneuver performed by pre-adolescent and adolescent athletes using a qualitative scoring system. Given the sharp increase in ACL injury risk in children ages 12-15, there is a need for available screening tools to identify risky movement patterns in this age group.⁵ Additionally, no qualitative assessment tool for evaluating a 45-degree cutting movement was found. Biomechanical demands during cutting and change of direction maneuvers have previously been described as being angle dependent, with differences in knee joint loading and technique found between 90 degree and 45 degree cut directions.^20,21 Furthermore, when implementing cutting technique modification training, clinicians and coaches need to be cognizant of the potential impact on performance. Sagittal plane mechanics have been found to predict performance of the 45-degree cut task, while frontal plane mechanics predicted performance of the 90-degree cut task.²¹ Given that frontal plane variables are most predictive of high knee abduction moments, technique training during a 90-degree cutting task that aims to address frontal plane variables may have the potential to negatively impact cutting performance.²² This may have implications for injury risk screening and targeted movement pattern correction as coaching staff may be less likely to adopt screening and training interventions that may negatively influence performance. Having identified a gap in qualitative scoring analysis for pre-adolescent and adolescent athletes, the authors of this study developed a qualitative scoring system, the Cutting Alignment Scoring Tool (CAST), to evaluate LE alignment and trunk movement during a 45-degree side-step cut. The central purpose of this study was to examine the reliability of the CAST among various sports medicine providers including medical doctors, physical therapists, and athletic trainers. This study consisted of two aims: 1) to assess the inter-rater and intra-rater reliability of the CAST among sports medicine providers, and 2) to evaluate rater agreement of each component of the CAST. The hypotheses were: 1) there would be good–to-excellent inter-rater and intra-rater reliability among sports medicine providers, 2) there would be good to almost perfect agreement for cut width and trunk lean variables and moderate to good agreement for valgus variables of the CAST.

METHODS

Study Design

A repeated measures study design was used. To achieve the primary aim of the study, inter-rater and intra-rater reliability were calculated based on the 1^st and 2^nd reliability sessions. The study protocol was developed based on the Declaration of Helsinki and ethical standards in sport and exercise science research.²³ Institutional Review Board approval was obtained prior to commencement of the study.

Participants

A total of 8 adolescent athletes (5 males, 3 females, age=14.7 $\pm$ 1.2 years, height=165.6 $\pm$ 8.4cm, mass = 62.6 $\pm$ 3.3kg) were recruited from local high school and club sport teams. Inclusion criteria were: 1) age between 12 and 17 years and 2) active participation in sports requiring cutting and pivoting in the last 12 months. The following exclusion criteria were used: 1) LE injury within 6 months, 2) past history of LE surgery, 3) a positive response on the Physical Activity Readiness Questionnaire for Everyone (PAR-Q+), and 4) history of scoliosis. The PAR-Q+ was used to determine the participant’s readiness and safety for physical activity. A positive response of the PAR-Q indicates the need to seek further advice from a physician prior to engaging in physical activity.²⁴ All participants provided written informed ascent, and their parents or legal guardian provided signed consent. Data collection was performed in a Sports Medicine laboratory at a local University.

Data Collection

Prior to performing the 45-degree side-step cut task, a 5-minute warm up on a cycle ergometer (Star Trac Inc, Irvine, CA) was performed. Participants practiced the side-step cutting maneuver 3 times in each direction or until they felt comfortable with the procedure. They were instructed to sprint at 80% of their maximum speed in a forward direction toward the “opponent cone”, to plant, and perform the side-step cut maneuver. (Figure 1).

Figure 1:45-degree side step cut task

This procedure was modeled by a testing protocol described by McLean et al.¹⁶ Specifically, participants decelerated, planted on the right foot, and performed a side-step cut, running in the left direction between cones placed along a 45-degree line of progression. The procedure was repeated planting on the left foot and running to the right direction (Figure 1). Participants then completed three trials planting on the right LE and three trials planting on the left LE, with a trial considered “good” if the subject’s foot landed within the stance/pivot area necessary for successful completion of the task. The testing order was standardized for all participants for ease of set up. Video data were collected with an iPad (iOS 12.4.1) with a frame rate of 30 frames per second. The iPad was mounted on a tripod and placed at a distance of 3.7 m in front of the force plates at a height of 0.86 – 0.96m. Participants performed a total of 6 cutting maneuvers. All videos were processed and slowed by 50% for visual analysis. All participants’ faces were blurred using Adobe Premiere Pro.

A clinically established checklist, CAST, was developed to examine the quality of trunk and LE movement during the cutting maneuver based on 2-D video. The checklist was devised based on previously reported movement screening systems.^14,25,26 It involves a dichotomous rating system, with scoring defined as “1” when a movement fault was present and “0” when optimal movement patterns were observed. Variables evaluated in the CAST include; trunk lean to the opposite direction of the cut, increased cut width, knee valgus at initial load acceptance (Static Evaluation), and knee valgus throughout the cutting task (Dynamic Evaluation). The CAST checklist is shown in Table 1.

Raters

The primary goal in choosing raters was to evaluate inter- and intra-rater reliability among sports medicine providers. Six raters consisting of two medical doctors (pediatric orthopedic surgeons), two physical therapists, and two athletic trainers were recruited for participation in this study. Raters were chosen because of their primary role in providing clinical care to young athletes in a pediatric healthcare system. The raters independently viewed a total of 10 videos. All raters provided their consent to participate in the current study.

Procedures

The 10 videos were randomly selected from the six trials performed by the eight participants. A review of the available studies that had a similar study objective led to the current sample size selection.^27–29 Video trials from three participants performing the side-step cut planting on the right limb were used, trials of five participants performing the side-step cut on the left limb were used and, and for 1 participant both limbs were used. The videos were slowed to 50% speed and were provided to each rater with brief instructions and a preliminary version of the CAST. The raters were instructed to view the videos independently without communicating with one another. They were allowed to review the videos as many times as necessary and could pause the video as needed. Raters were not permitted to make any marks on the screen, including joint angle or distance-based measurements.

The raters were given one week to view and score the videos. After the initial scoring, a peer-to-peer discussion was held to discuss the CAST rating tool and to develop detailed definitions to improve the interpretation of the scoring criteria. The discussion was performed via a two-hour long video conference. During the video conference, each of the 10 videos were displayed, and each rater had an opportunity to share their clinical diagnosis of at-risk movements. After each rater provided their interpretation of at-risk movements and discussed subsequent clinical decision-making, the group provided feedback. After the peer-to-peer discussion, a reference sheet (Figure 2) was developed, and the CAST scoring criteria was finalized (Table 1).

Figure 2:CAST reference sheet

The first reliability session was performed one-week following the interactive peer-to-peer discussion. The same reviewing instructions that were used during the development of the CAST scoring criteria were given to each rater and the importance of the independent reviewing process was emphasized. The first reliability session consisted of the same 10 videos, with sequences altered in random order to minimize memory bias. After the first reliability session, a two-week break was given as a wash-out period. Next, the second reliability session was performed, using the same method outlined for the first reliability session. The sequence of videos was further randomized in the second reliability session.

Statistical Analysis

Reliability was determined by calculating intraclass correlation coefficients (ICC) for the CAST total scores, with 2-way mixed-effects model and their 95% confidence intervals (95% CIs) for inter-rater and intra-rater reliability. For the first aim, the individual and cumulative inter-rater reliability of six raters were calculated within the 1^st and 2^nd reliability sessions. The individual and cumulative intra-rater reliability of six raters were calculated between the 1^st and 2^nd reliability sessions. ICC values less than 0.50, between 0.50 and 0.75, between 0.75 and 0.90, and greater than 0.90 were defined as poor, moderate, good and excellent reliability, respectively.³⁰ To achieve study aim 2, a kappa coefficient was calculated for each of the checklist variables using the formula; k= Pr(a) – Pr(e)/1 – Pr(e), where Pr(a) is equal to the relative observed agreement between raters and Pr(e) is equal to the hypothetic probability of chance agreement. The kappa coefficient was interpreted based on the scale of Landis and Koch³¹ with 0.01-0.2 being slight, 0.21-0.4; fair, 0.41-0.6; moderate, 0.61-0.8; good and 0.81-1.0 almost perfect. All statistical analyses were conducted using SPSS Statistics 22 (IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp).

RESULTS

Inter-rater reliability for the 1^st reliability session was moderate (ICC: 0.632, 95% CI 0.100-0.895) and inter-rater reliability for the 2^nd reliability session was excellent (ICC: 0.901, 95% CI 0.766-0.971). The cumulative inter-rater reliability, a combination of 1^st and 2^nd inter-rater reliability, was good (ICC: 0.808, 95% CI 0.644-0.913). Intra-rater reliability of each rater is depicted in Table 2. The cumulative intra-rater reliability of the six raters was good (ICC: 0.753, 95% CI 0.588-0.852). Kappa-coefficients for each variable are presented in Table 3. Almost perfect kappa coefficients were recorded for cut width (k=0.949). Moderate kappa coefficients were found for trunk lean (k= 0.632), and fair kappa coefficients were noted for dynamic valgus and static valgus (k=0.462 and k= 0.533 respectively).

DISCUSSION

The primary purpose of this study was to assess the within session inter-rater reliability and the between session intra-rater reliability of the CAST. The CAST demonstrated good inter-rater reliability (cumulative ICC: 0.808, 95% CI 0.644-0.913) and good intra-rater reliability (cumulative ICC: 0.753, 95% CI 0.588-0.852). These findings supported our hypotheses that the CAST would demonstrate good-excellent inter-rater and intra-rater reliability among sports medicine providers. The findings also suggest that the CAST may allow clinicians to standardize their assessment of trunk and lower extremity alignment using 2-D video during a cutting task. It needs to be noted, however, that when comparing the first and second inter-rater reliability sessions, the first session showed lower reliability and larger confidence intervals relative to the second session (ICC: 0.632, 95% CI 0.100-0.895). The improvement in reliability from the first to the second session may have potentially been the result of a learning effect as this was the third time the raters viewed the same 10 videos. Interestingly, the medical doctors demonstrated the highest intra-rater reliability (Table 2). This contrasted with the authors’ expectations. Initially, it was speculated that the physical therapists and athletic trainers would show the highest intra-rater reliability due to their frequent use of movement analysis in their daily practice settings. One explanation for this outcome may be certain character traits that are unique to the medical doctor profession. Anecdotally, medical doctors are trained to adhere to medical guidelines to make precise and informed decisions in their clinical settings, especially during surgery. Both medical doctors in this study were orthopedic surgeons, which might have contributed to the higher intra-rater reliability. The findings of the current study are generally in agreement with the work of Dos’ Santos et al¹⁸ who found moderate inter-rater reliability (ICC =0.690) and excellent intra-rater reliability (ICC =0.946) when utilizing a similar qualitative scoring system to evaluate a 90 degree cutting maneuver. There are several differences in study design. Dos’ Santos et al¹⁹ determined inter-rater reliability of the CMAS using three raters rather than six. Despite a larger number of raters, inter-rater reliability of our measurement tool, the CAST, was higher than that of the CMAS. The underlying reason for higher inter-rater reliability found in the current study is likely multifactorial. One potential explanation for the observed higher inter-rater reliability in the study is the interactive peer-to-peer discussion. During the discussion session, all raters openly shared their clinical knowledge, expertise, and experiences. The reference sheet and CAST were developed as on output of the discussion. This process provided an opportunity for sports medicine providers from multiple disciplines to critically review the scoring tool and contribute to the final development. Hence, discussion among peers may be an important step to develop a clinically useful evaluation tool. This lesson also has an important clinical implication, particularly in a multi-center study setting. Reviewing a protocol, images, or variables of interests prior to study commencement is likely helpful to enhance reliability among peers. Another notable difference between this study and that of the CMAS is that the intra-rater reliability of the CMAS was reported for only one rater.¹⁹ The current study reported the average of six raters which may have contributed to the lower intra-rater reliability. Lastly, Dos’ Santos et al¹⁹ only used a one-week wash out period compared to a two-week washout period used in the current study. There are also several differences in the variables evaluated with the CMAS compared to those evaluated with the CAST. The variables evaluated with the CMAS are observed through three different camera views in the frontal, sagittal and 45-degree planes.¹⁹ The variables assessed in the current study were only observed through one camera view which is in the frontal plane. Frontal plane variables were chosen based on the work of Imwalle et al who found that increased knee abduction moments during cutting predominantly stem from the frontal plane.²² Although the three-camera views used in the CMAS likely allowed for improved identification of the variables of interest, obtaining multiple views of a cutting maneuver on the field may pose challenges. A single view assessment tool may result in improved adoptability by clinicians and coaching staff. Additionally, eight variables are evaluated with the CMAS while only four variables are evaluated with the CAST. The lower number of variables assessed with the CAST may have improved the tools efficiency, however, future studies are necessary to determine its validity against 3D motion capture, and in identifying athletes displaying high risk cutting mechanics.

The second aim of this study was to evaluate rater agreement of each component of the CAST. Almost perfect kappa coefficients were found for cut width (k=0.949) while moderate kappa coefficients were observed for trunk lean (k= 0.632) and fair kappa coefficients were noted for dynamic valgus and static valgus (k=0.462 and k= 0.533 respectively). The results suggested that assessments of less complex movements during a cutting maneuver were more reliable than multiplanar movements. This supports the hypothesis that there will be good to almost perfect agreement for cut width, and moderate to good agreement for trunk lean. The hypothesis that valgus variables would demonstrate moderate to good agreement was not supported. Only fair agreement was found for dynamic and static valgus. Again, these findings were generally in agreement with Dos Santos et al¹⁹ who found fair agreement (k=0.551) for frontal plane trunk position and moderate agreement (k=0.605) for excessive knee valgus motion. Several factors may have contributed to the decreased reliability of valgus identification in the current study. For example, changes in knee flexion angle and deviation of the plane of the body out of the frontal plane of the video camera may make it difficult to accurately identify frontal plane movement faults. Additionally, videos in this study were collected on an iPad at a rate of 30 frames per second, future studies should determine if video collection at a higher frame rate would increase the reliability of the CAST. Future studies are warranted to consider those aspects in conjunction with cutting maneuvers, which should help in identifying at-risk movements with 2-D video analysis.

The authors recommend that future studies should also aim to determine the predictive validity of qualitative and 2-D screening tools for ACL injury risk. In a prospective study by Hewett et al,⁹ knee abduction moments predicted ACL injury with 73% specificity and 78% sensitivity and dynamic valgus measures showed a high predictive value (r² =0.88). Screening tools that are able to accurately identify high knee abduction moments and dynamic valgus values may be the most effective in identifying athletes at increased risk for an ACL injury.⁹ A statistically significant strong relationship (r = 0.796, 95% CI 0.647 to 0.887) was found between the CMAS and peak knee abduction moments in a sample of 41 college aged athletes (28 males/13 females, mean age 21.3 + 4.0 years) from multiple sports.¹⁹ These finding suggest that a qualitative assessment of a cutting task may help identify collegiate athletes with high knee abduction moments during a cutting maneuver. Similarly, moderate correlations were found between 2-D measurement of frontal plane valgus angles and 3-D analysis (r² =0.58), suggesting its feasibility for ACL injury risk screening.³² Future studies should continue to explore this area, specifically in the pre-adolescent and adolescent populations, which have been understudied. It is currently unknown if qualitative screening tools evaluating cutting technique in young athletes are predictive of ACL injuries. With this concept, the current study focused on pre-adolescent and adolescent population. The dynamic and static cumulative inter-rater reliabilities were not as high as was anticipated (ICC=0.462, 0.533, Table 3). Specific bony landmarks and/or more stringent definitions of knee valgus in cutting maneuvers may need to be addressed by sports medicine providers. The development of screening tools that are able to accurately identify high-risk cutting movements may provide coaches and practitioners with an efficient and effective strategy to screen young athletes for ACL injury risk. Furthermore, providing clinicians with a practical tool to evaluate cutting tasks may enhance injury prevention interventions. Given the high percentage of ACL injuries that occur during a cutting or pivoting mechanism, future research is warranted to find an association between clinically useful 2-D video analysis and ACL injury in young active populations.

Limitations

Several limitations must be stated. First, the CAST only evaluates frontal plane movements. Cutting maneuvers are multiplanar and incorporating additional views and planes may improve the capacity to identify faulty movement biomechanics. Additionally, the operational definitions for each variable were written with varying criteria. For example, an approximate degree reference was provided for trunk lean, a body reference was provided for cut width and a qualitative description was provided for the valgus variables. This variability may have contributed to rater confusion when using the tool. Another limitation is that only the reilabilty of sports medicine providers was evaluated. Sport specific coaches spend the most time working with athletes; thus, determining the reliability of the CAST amongst coaches may greatly increase its utility. In this study a peer-to-peer discussion was held with all six raters, one week prior to the first reliability session. During this discussion the same 10 videos that were used in the 1^st and 2^nd reliability sessions were viewed and discussed as a group. Despite the use of a two-week wash out period between rounds and the randomization of videos in each round, this discussion may have increased the inter and intra-rater reliability of the tool. Future studies should evaluate the reliability of the CAST without the added discussion. It should also be acknowledged that this study used a planned cutting task. Different outcomes may be expected with the use of an unplanned cutting task which has been shown to result in greater knee joint loads when compared to planned cutting maneuvers.³³ Lastly, it is unknown if the CAST is a valid tool for predicting ACL injury risk during a cutting maneuver. It is important to investigate whether or not 3-D kinematic variables are correlated with visually identified movements. Future studies should aim to determine its predictive validity and its criterion validity with 3-D motion capture.

CONCLUSION

Despite limitations, this study demonstrated that the CAST, a qualitative evaluation tool to identify at-risk movements for ACL tear during a side-step cutting task, demonstrated good inter-rater and intra-rater reliability among sports medicine providers. It also showed almost perfect between rater agreement for cut width, moderate agreement for trunk lean, and fair agreement for knee valgus variables. The two-hour peer-to-peer discussion might have contributed to the relatively high inter-rater reliability. These findings suggest that the CAST can be used as a reliable tool to evaluate trunk and lower extremity alignment during a cutting task by sports medicine providers. Identification of risky movement patterns may serve as a starting point for sports medicine providers to provide targeted technique training to reduce ACL injury risk during cutting tasks. Future work is recommended to determine the predictive validity of the CAST in identifying individuals at risk for ACL injury.

Funding Source

No funding

Approval

This study was approved by the University of Miami and Western International Review Board for human subjects’ research.

Financial Disclosure and Conflict of Interest

The authors affirm that they have no financial affiliation (including research funding) or involvement with any commercial organization that has a direct financial interest in any matter included in this manuscript, except as disclosed in an attachment and cited in the manuscript. Any other conflict of interest (i.e., personal associations or involvement as a director, officer, or expert witness) is also disclosed in an attachment.

Acknowledgements

We would like to acknowledge the University of Miami, Nicklaus Children’s Hospital and the PRiSM Injury Prevention Research Interest Group for their support of this project.