INTRODUCTION

Musculoskeletal (MSK) health problems are one of the greatest healthcare expenses in the United States.¹ Poor MSK health can cause significant loss of time from work and recreational activities, and it is strongly correlated with the development of chronic pain syndrome and opioid use.² According to data from the World Health Organization, MSK health problems are second only to mental health disorders as the primary contributor to years of life with disability and 19^th in years of life lost across the globe.³ Over a 15-year period, the total number of years living with disability due to MSK conditions has increased from 77 million to 103 million and years of life loss have risen by more than 40 percent.³ Consequently, the economic and functional burden of MSK disorders will continue to increase as the general population becomes older due to longer life expectancies.⁴

The risk of sustaining an MSK injury is increased by the presence of modifiable risk factors that can be assessed by a trained healthcare professional.^5–8 The Functional Movement Screen (FMS™) and Y Balance Test-Lower Quarter (YBT-LQ™) are reliable, movement-based tests that have a relationship to heightened injury risk when deficits are identified.^9–12 When multiple risk factors are considered using an evidence-based injury risk prediction algorithm, individuals identified as high-risk were 3.4 times more likely to sustain a non-contact MSK injury.¹³ Additionally, the more risk factors an individual possesses the more likely an MSK injury that limits participation in physical activity is to occur.⁷ Early identification of the development of risk factors or declining physical function could allow patients to take a more active role in preventive strategies,¹⁴ which researchers have shown can decrease the financial burden on both the individual and the healthcare system.^15,16

Patient-driven screening is widespread in its application among other healthcare specialties to create awareness of potential risk factors or conditions. Multiple body systems such as the cardiovascular, integumentary, endocrine, and lymphatic systems have patient-driven screens for early detection and subsequent intervention recommendations. The health of the cardiovascular system can be screened with a sphygmomanometer for blood pressure measurements, which is reliable in identifying possible cardiovascular disease.¹⁷ The use of educational materials and skin self-examinations allows for early detection and proper treatment of melanoma^18,19 and breast cancer which has significantly increased survival rates.^20,21 Glucometers have been useful in helping those with a diagnosis of diabetes to monitor the impact of medication and lifestyle factors on blood sugar levels.^22,23 However, self-screening options for the MSK system are limited despite the growing need to combat rising disability and attenuating quality of life associated with poor MSK health. The increasing prevalence of MSK disorders warrants a user-friendly self-screening tool that can be administrated by the general public to aid in creating awareness of physical risk factors.

To date, there are no studies that have explored a patient-driven self-screening tool to reliably identify MSK risk factors. The Symmio Self-Screen (Symmio) is a downloadable application developed to be a user-friendly and cost-effective tool for the early identification of physical risk factors that may lead to MSK injuries. The primary purpose of this study was to establish the inter-rater reliability of the Symmio application in untrained individuals. It was hypothesized that Symmio will demonstrate moderate inter-rater reliability with a Cohen’s kappa value >0.40 when administered by untrained individuals. The secondary purpose of this study was to investigate the accuracy of Symmio to detect MSK risk factors such as pain with movement, movement dysfunction, and decreased dynamic balance. It was hypothesized that poor performance on Symmio would increase the odds of having pain with movement, dysfunctional movement quality on the FMS™, and deficient dynamic balance on the YBT-LQ™.

MATERIALS AND METHODS

Study Design

A prospective cross-sectional design was used to establish the inter-rater reliability and discriminant validity of Symmio to identify the presence of painful or dysfunctional movement and dynamic balance deficits among a cohort of active individuals. The Standards for Reporting Diagnostic Accuracy Studies (STARD) statement for a diagnostic accuracy study design was followed for standardized reporting.²⁴ Approval was granted from the institutional review board at the University of Evansville and informed consent forms were obtained before data collection.

Participants

A minimum sample size of 50 participants was needed to achieve a Cohen’s kappa value of 0.40 with an alpha of 0.05 and 80% power for a two-tailed test. To detect a sensitivity of 0.90, at least 58 participants would be required considering a prevalence of 0.60 and a 0.10 confidence interval width allowance.²⁵ Anticipating 10% of the participants enrolled may have incomplete data, the final planned minimum target sample size was 64 participants.

A convenience sample of 80 individuals between the ages of 18 and 70 of either sex was enrolled in the study. Participants were excluded from enrollment in the study for lower-extremity amputation, vestibular disorder, lack of medical clearance for participation, current treatment for the inner ear, sinus or upper respiratory infection or head cold, cerebral concussion within the past three months, or inability to read or comprehend English.

Procedures

Physical testing occurred during a single session with no follow-up required. After informed consent was obtained, the participants completed a demographic and medical history questionnaire and then completed Symmio, FMS™, and YBT-LQ™ testing. The order of physical testing was randomized to control for fatigue and changes in movement quality. All testers were blinded to the results of the tests.

The Symmio Self-Screen is a movement-based screening tool intended to identify MSK disorders using an instructional application on a smart phone or tablet. Symmio consists of five tests with two levels of difficulty for each movement including, 1) tandem toe touch, 2) shoulder mobility, 3) rotation, 4) deep squat, and 5) balance and reach (Figure 1). Participants were allowed three attempts to perform each movement and each test was scored as pass or fail per the criteria below on both right and left sides when applicable. Subjective report of pain with movement was recorded for each movement.

1. Tandem Toe Touch: While standing with feet in tandem stance, the participants were instructed to reach down and attempt to touch the toes of their forward foot while maintaining knee extension (Level 1). The test was repeated on the opposite side. The participants then attempted a more challenging test by repeating the movement but now attempting to touch the toes of their back foot (Level 2). Inability to touch the toes of the front foot on both sides at the Level 1 standard was considered failure.

2. Shoulder Mobility: The participants stood with feet together holding a horizontally folded standard piece of paper (8.5 x 11 in) in one hand. In one smooth motion, the participants simultaneously reached one hand behind their head (in a flexed and externally rotated position), and the other hand behind and up their back (in an extended and internally rotated position) attempting to pass the paper from one hand to the other (Level 1). The test was repeated on the opposite side. The participants then performed the test without the folded paper, attempting to touch their fingertips together (Level 2). Inability to pass the folded paper between hands on both sides at the Level 1 standard was considered failure.

3. Deep Squat: The participants stood with their feet together, shoulders flexed to 90°, and fingers extended. Maintaining the heels in contact with the ground, the participants descended into a deep squat and attempted to touch their fingertips to the ground within their footprint (Level 1). The participants then attempted a more challenging test by repeating the movement but with their fists closed (Level 2). Inability to touch the fingertips to the ground at the Level 1 standard was considered failure.

4. Rotation: The participants stood with feet together and shoulders flexed to 90° with fingers interlocked. Maintaining tall posture, the participants attempted to rotate their trunk and hips greater than 90° using their arms as a gauge (Level 1). The test was repeated on the opposite side. The participants then attempted a more challenging test by positioning their feet in tandem stance and repeating the rotation toward the forward leg (Level 2). Inability to rotate greater than 90° with feet together at the Level 1 standard on both sides was considered failure.

5. Balance and Reach: The participants stood two shoe lengths away from a wall, and while maintaining single leg balance on one foot with the heel down, the participants reached with the opposite foot to attempt to touch the point on the wall just above the ground five consecutive times without the foot touching down or losing balance (Level 1). The test was repeated on the opposite side. The participants then stepped back 2.5 shoe lengths from the wall and repeated the movement (Level 2). Inability to touch the wall five consecutive times without loss of balance at two shoe lengths from the wall on both sides was considered failure.

Figure 1.Symmio Self-Screen Tests

1a=Tandem toe touch (front toes), 1b=Tandem toe touch (back toes), 2a=Shoulder mobility (paper), 2b=Shoulder mobility (fingertips), 3a=Feet together rotation, 3b=Tandem rotation, 4a=Deep squat (fingertips), 4b=Deep squat (fists), 5a=Balance and reach (2 foot lengths), 5b=Balance and reach (2.5 foot lengths)

Inter-Rater Reliability

Each participant self-screened themselves using the Symmio application. The participants followed the standard video guidance and instructions from the application using a tablet (IPadOS, 15.6.1, 9^th generation) while a healthcare professional simultaneously scored each movement in real time. The healthcare professional (TAN) was a physical therapist (PT) trained in evaluating the Symmio movements. The participants nonverbally recorded their scores in the Symmio application and on a data collection form. The PT rater stood far enough away from the participants to remain blinded to the participants’ scores. The participant’s self-screen scores and the PT rater’s observed scores for each Symmio test were compared to establish inter-rater reliability.

Discriminant Validity

To determine the validity of Symmio, scores on individual components of the self-screen were compared to established movement screening and motor control testing procedures. Two PTs who were certified in the FMS™ and YBT-LQ™ performed all functional testing and were blinded to the Symmio results. To improve reliability, the same PT performed the FMS™ (RJR) to appraise movement dysfunction while the other PT tested dynamic balance deficits with the YBT-LQ™ (EV).

1. Pain with Movement: Any subjective report of pain, which was defined as discomfort beyond normal stretch or soreness, with any of the physical testing was considered painful.⁸

2. Functional Movement Screen: This screening tool is used to identify limitations or asymmetries in seven movement patterns that are key to functional movement quality. The FMS™ consists of the overhead deep squat, hurdle step, inline lunge, shoulder mobility, active straight leg raise, trunk stability push-up, and rotary stability tests which are each scored on a four-point ordinal scale. Procedures and scoring for the FMS™ were consistent with recommendations from Cook et al.²⁶ Failure was defined as a score of a 1 (movement limitation) or asymmetry on any individual FMS™ movement patterns.²⁷

3. Y Balance Test-Lower Quarter: The YBT-LQ™ tests tri-planar dynamic balance near the limits of an individual’s stability. Testing procedures and scoring were consistent with the protocol and testing kit developed by Plisky et al.²⁸ Failure was defined as an anterior reach asymmetry >3.5 cm, posteromedial and posterolateral asymmetries >5.5 cm, or ankle dorsiflexion ROM limitation (<40°) or asymmetry (≥5°).^7,29

Statistical Methods

Descriptive statistics including means and standard deviations (SD) were calculated. Inter-rater reliability for the categorical scores of each component of Symmio was compared between both raters using Cohen’s kappa coefficient with 95% confidence intervals (CI 95%) and percent absolute agreement. The Cohen’s kappa coefficient quantifies the strength of agreement and was interpreted as: 0.01-0.20 = none to slight, 0.21-0.39 = fair, 0.40-0.60 = moderate, 0.61-0.80 = substantial, ≥0.81 = near perfect.³⁰

The accuracy of Symmio to discriminate between participants with pain, movement dysfunction, and dynamic balance deficits was determined using cross-tabulations. The presence of pain, movement dysfunction, and dynamic balance limitations were dichotomized as described previously and entered into separate 2x2 tables. Univariate analyses were performed using the chi-square test for categorical variables to evaluate significant differences between Symmio performance and the presence of pain, movement dysfunction, or dynamic balance deficits. A Fisher’s Exact test was used to measure the association if one or more cells had an expected count of less than five. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratios (LR), and odds ratios (OR) were calculated with 95% confidence intervals to describe the accuracy of the Symmio self-screen to detect MSK risk factors. All data analyses were performed with R for Mac OS 4.1.2 statistical software (RStudio for Mac, Version 1.4). An alpha level of p < 0.05 was considered statistically significant for all tests.

RESULTS

Demographic information for all participants is provided in Table 1. Eighty participants completed all testing procedures, and the data were used for analysis. The mean age ± SD of the participants in this sample was 26.5 ± 9.4 and ranged from 18 to 68 years old. All participants reported engaging in some physical activity with 80% (n=64/80) reporting that they were moderately or highly active. A total of 25% (n=20/80) of participants reported current pain with activities of daily living before testing.

Inter-Rater Reliability

Results for the inter-rater reliability (kappa, 95% CI, and % agreement) for both raters are presented in Table 2. Cohen kappa values ranged from fair to near perfect agreement between both raters. The tandem toe touch tests had the highest inter-rater reliability with Cohen kappa values that ranged from 0.85 (95% CI, 0.68-1.0) to 0.90 (95% CI, 0.77-1.0) and represented near perfect agreement. The balance and reach test at 2.5-foot lengths from the wall demonstrated the lowest inter-rater reliability with Cohen kappa scores ranging from fair to moderate agreement on the left (k=0.34, 95% CI, 0.20-0.48) and right (k=0.43, 95% CI, 0.26-0.60) leg, respectively.

Pain with Movement

The accuracy of Symmio to discriminate between those who had pain with movement on the FMS™ is reported in Table 3a. The chi-square test showed a significant association for the 2x2 contingency table created (X²=8.73, p=0.003) for participants’ reporting pain with movement on the FMS™ and Symmio. The Symmio Self-Screen was 74 percent (95% CI, 0.63-0.83) accurate in identifying pain with movement on the FMS™. The sensitivity (0.71, 95% CI, 0.49-0.87) and specificity (0.75, 95% CI, 0.62-0.86) for detecting the presence of pain with functional movement were only marginally different. Participants with pain on the FMS™ were 2.8 times (+LR=2.83, 95% CI, 1.68-4.77) more likely also to have pain on Symmio. Overall, participants who had pain on the FMS™ were 7.29 (95% CI, 2.51-21.2) greater odds of reporting pain on Symmio compared to those without pain on the FMS™.

Movement Dysfunction

The 2x2 table created to examine the discriminant validity of Symmio to identify movement dysfunction compared to the FMS™ is presented in Table 3b. There was a significant association between the presence of poor movement quality on the Symmio Self-Screen and failure of the FMS™ tests which were more mobility focused (X²=15.67, p=0.001). The diagnostic accuracy of Symmio to capture mobility dysfunction on the FMS™ was 73 percent (95% CI, 0.62-0.82). There was no significant association between the presence of movement dysfunction on the FMS™ and Symmio when failure on the FMS™ was defined as the presence of 1’s or asymmetries (p=0.31), or when failure was defined as a composite score of less than 14 (p=0.33).

Dynamic Balance Deficits

The 2x2 table created to examine the discriminant validity of Symmio to identify poor dynamic balance compared to the YBT-LQ™ is presented in Table 3c. There was a significant association between individuals who failed the Symmio Self-Screen and poor performance on the YBT-LQ™ (X²=9.13, p=0.003). Poor performance on Symmio had a 69 percent (95% CI, 0.57-0.79) accuracy in detecting individuals who also failed the YBT-LQ™. The Symmio Self-Screen demonstrated similar sensitivity (0.69, 95% CI, 0.55-0.82) and specificity (0.68, 95% CI, 0.49-0.83) for detecting the presence of dynamic balance deficits. Participants who scored poorly on Symmio were more than two-fold more likely (+LR=2.15, 95% CI, 1.25-3.70) to have dynamic balance deficits on the YBT-LQ™ with an odds ratio of 4.76 (95% CI, 1.81-12.5).

DISCUSSION

The goal of the Symmio Self-Screen is to create MSK health awareness through a user-friendly electronic application that the general public can administer without dependence on a trained healthcare professional. The findings from this study support the primary hypothesis that untrained individuals can reliably administer the Symmio application to self-screen for MSK risk factors that may contribute to physical limitations and determine the need for further evaluation by a healthcare professional. All subtests of Symmio, scored as pass or fail, exhibited fair to near perfect inter-rater reliability between both raters regardless of their movement screening experience.

Movement screening performed by trained healthcare providers or fitness coaches has demonstrated moderate to excellent levels of agreement.^10,31 Additionally, untrained individuals with limited experience screening movement have demonstrated the ability to reliably screen movement.³² Leeder et al.³³ reported that untrained practitioners with limited clinical experience were able to correctly identify dysfunction in the FMS™. Likewise, high school baseball coaches with no screening experience and minimal training were able to screen movement quality similarly to a clinician with 10 years of expertise in movement screening.³⁴ However, the investigation of self-appraisal of functional movement in untrained, non-healthcare individuals is a novel concept.

To the authors’ knowledge, this is the first study to examine the ability of untrained individuals to screen their own movement patterns without the aid of an external observer. Participants without previous knowledge of the Symmio scoring criteria could follow the application’s instructional video and score similarly to a trained healthcare provider. Each Symmio subtest was dichotomized as pass or fail with clear biomarkers for success which likely reduced the scoring complexity and minimized errors among the participants. Other movement screening tools and assessments such as the FMS™, Selective Functional Movement Assessment (SFMA™), and Landing Error Scoring System (LESS) have more complex scoring criteria or more category scoring options which is more suitable for trained professionals.^35,36

It is particularly important for untrained individuals who are administering a self-screen to have a simplified scoring system to maximize reproducibility. Interestingly, the balance and reach subtest, which is like the anterior reach of the star excursion balance test (SEBT), demonstrated only fair to moderate agreement and had the lowest agreement of all the Symmio subtests. In a systematic review, Powden et al.¹¹ reported excellent inter-rater reliability (0.88, 0.83-0.96) and intra-rater reliability (0.88, 0.84-0.93) on the anterior reach of SEBT. In the current study, the participants performing the balance and reach test may have had difficulty perceiving when their stance leg heel lifted off the ground which may have been more apparent to a trained observer.

The findings from this study partially support the secondary hypothesis that the Symmio tests can accurately discriminate between individuals with or without MSK risk factors. The Symmio subtests were comprehensive enough to reproduce painful movement similar to the FMS™. Painful movement on the FMS™ has been shown to be a meaningful risk factor associated with seven-fold increased injury risk.⁸ However, an association between movement dysfunction and asymmetries or overall composite score on the FMS™ and Symmio was lacking. Though a stronger association between Symmio performance and FMS™ results was expected, it is interesting to note that Symmio demonstrated a higher specificity (0.75, 95% CI, 0.62-0.86) for detecting the presence of movement dysfunction on the FMS™ mobility tests compared to sensitivity (0.71, 95% CI, 0.49-0.87). Participants with movement dysfunction on the FMS™ mobility tests were nearly three times (+LR=2.86, 95% CI, 1.35-4.88) more likely to also have dysfunctional movement on Symmio. Overall, participants who demonstrated dysfunctional mobility tests on the FMS™ were more than seven (7.40, 95% CI, 2.60-21.0) times greater odds of performing poorly on Symmio compared to those who passed. Symmio was intentionally designed to have a mobility bias to reflect current research findings linking mobility deficits to future injury risk.^37,38 Saddler et al.³⁹ reported that individuals with limited hamstring flexibility were more likely to develop low back pain, whereas, ankle dorsiflexion restrictions can increase the odds of an injury and medical discharge in Army recruits.^40,41 The creation of a patient-driven, self-screening tool for the reliable detection of MSK risk factors can alert individuals of impending MSK needs. The Symmio self-screen can allow individuals to seek evaluation and treatment prior to MSK injury, which may lead to decreased healthcare dollars spent on preventable issues. Future research should explore the prospective longitudinal correlation between poor Symmio performance and the development of MSK conditions throughout the lifespan.

Limitations

This study is not without limitations. First, the prevalence of MSK risk factors was high in this sample limiting confidence in the interpretation of PPV and accuracy due to the oversaturation of risk factors. This is especially true when screening movement dysfunction and dynamic balance as the accuracy does not exceed the upper bound 95% CI for the prevalence. Second, the mean age of participants was relatively young at 26.5 ± 9.4. This decreases generalizability, as the sample may not reflect the general population. Although the sample included individuals in their 50s and 60s, most participants were college-aged limiting generalization throughout the lifespan. Thirdly, only 15% of participants reported current pain prior to testing, but many more experienced pain during the movement and balance testing. Though the goal of Symmio is to detect these issues, the mismatch in awareness may have impacted results. Finally, the reference standards used to establish the criterion validity of Symmio did not include additional movement-based screens beyond the FMS™ and YBT-LQ™. Although comparison utilizing other movement-based screens could be meaningful, the authors determined that no other movement-based screen or dynamic balance test has been as robustly examined in relation to reliability, predictive validity, and modifiability as the FMS™ and YBT-LQ™.

CONCLUSION

The Symmio Self-Screen application is a reliable, user-friendly, and feasible screening tool that can be used to identify MSK risk factors including pain with movement, movement dysfunction, and dynamic balance deficits in untrained, non-healthcare individuals. This application can be used by the general public to create awareness of the user’s current movement health and encourage preventative action before the development of disability.

ACKNOWLEDGMENTS

The authors of this study would like to thank the University of Evansville Doctor of Physical Students Mattie Franklin, Skylar Walden, Rachel Howey, and Kris Bale for their assistance with study organization and data collection.