INTRODUCTION

Appropriately developed patient reported outcome measures (PROMs) with good psychometric properties can play an important role in assessing the effect of treatment in patient care. A large number of PROMs have been developed for specific lower extremity orthopaedic pathologies. However, a consensus as to which PROMs are recommended for use in evaluating treatment outcomes for patients with hip, knee, ankle and/or foot pathology based on the strength of their psychometric properties is lacking.1–4

Many PROMs have been used in research and clinical practice for those with lower extremity orthopaedic pathology or those who have undergone surgery. Haywood et al.5 identified that 28 different PROMs were developed for describing hip fracture outcomes. Similarly, Ramkumar et al.6 found 47 different PROMs for knee arthroplasty and 21 different PROMs for Achilles tendon rupture.7 Other systematic reviews also found that not only were a high number of PROMs used in published research, but many studies incorporated PROMs that lacked appropriate psychometric properties.1,8 This inconsistency in PROM use has led to difficulties in both clinical practice and research reporting.

When a number of different PROMs are used to report outcomes, it may be difficult to compare treatment outcomes in the same or similar populations.3 Additionally, clinicians and researchers may choose to use PROMs that do not have the most appropriate psychometric properties. This may negatively influence the interpretation of the obtained outcomes scores. Often multiple PROMs are used simultaneously to overcome the fact that there is little consensus regarding which instrument is most appropriate. The use of multiple PROMs causes problems with an increase in the time commitment for patients, clinicians, and researchers.

A review is needed to summarize which PROMs are recommended based upon their psychometric evidence for specific orthopaedic hip, knee, foot and ankle pathologies and surgeries. In addition, the actual utilization of recommended PROMs in published research in not known. The purpose of this study was to identify PROMs that are recommended in systematic reviews for those with orthopaedic hip, knee, foot, and ankle pathologies or surgeries and identify if these PROMs are used in the literature.

METHOD

An umbrella review was performed to identify published systematic reviews that made a recommendation regarding the use of PROMs for lower extremity pathologies or surgeries. Keywords in the search strategy were hip, knee, ankle, foot, index, measure, instrument, scale, questionnaire, reliability, validity, responsiveness, and psychometric properties. PubMed, Embase, Medline, Cochrane, CINAHL, SPORTDisucs and Scopus data bases were searched from the data base inception through March 2022. In order to be included in this review, articles needed to be a systematic review with an assessment of psychometric properties, including reliability, validity, and/or responsiveness of more than one lower extremity PROM. The systematic review also needed to include a recommendation for PROM use. Articles or PROMs not available in the English were excluded. Two authors independently evaluated the systematic reviews for inclusion, extracted data, and graded the included systematic reviews using the Joanna Briggs Institute checklist.9 Any conflicts were resolved by a third author.

A second search was done to determine the frequency that PROMs were used in published research articles in seven representative orthopedic journals from January 2011 through March 2022. The seven orthopedic journals included the following: 1) Journal of Orthopedic and Sports Physical Therapy, 2) International Journal of Sports Physical Therapy, 3) Foot & Ankle International, 4) American Journal of Bone and Joint Surgery, 5) American Journal of Sports Medicine, 6) Clinical Orthopedics and Related Research, and 7) Journal of Orthopedic Trauma. These seven journals were chosen because of their wide readership, impact factor, and frequent publications on orthopaedic clinical outcomes for the hip, knee, ankle and foot using PROMs. These methods were similar to those used by Hunt et al.8 and Hijji et al.1 The number of clinical research articles that utilized PROMs in each journal were manually counted by two separate authors. Case reports, review articles, and basic science articles were excluded in this second search. The specific PROM, name of the journal, publication date, and pathology or surgery were extracted independently by two authors.

RESULTS

The initial search identified 1584 articles to produce 19 systematic reviews that recommended a PROM for individuals with a hip, knee, ankle and/or foot pathology or surgery. (Figure 1) The 19 systematic reviews recommended 20 PROMs for 15 lower extremity pathologies or surgeries.7,10–27 (Table 1) The results of the Joanna Briggs Institute checklist appraisal for the included systematic reviews are shown in Table 2.

Figure 1
Figure 1.PRISMA Search Diagram
Table 1.Recommended and Commonly used Patient Reported Outcome Measures
Pathologies Systematic review
recommended		 Most common instruments in seven journals
Most common Second common Third common
Total Hip Arthroplasty WOMAC14 WOMAC (28)
UCLA (28)		 - HOOS (26)
OHS14
HOOS11
Hip Arthroscopy HOS combined with NAHS17 mHHS (27) HOS (19) NAHS (14)
HOS11,18
iHOT-3320,24
iHOT-1218,24
HAGOS18,24
Non-arthritic hip joint pain, femoroacetabular impingement, hip labral pathology HOS12,18 mHHS (96) HOS (69) iHOT-12 and 33 (33)
iHOT-1218,24
iHOT-3324
HAGOS18,24
Groin pain HAGOS18,24 HAGOS (8) HOOS (6) IHOT-33 (5)
Hip osteoarthritis undergoing non-surgical treatment HOOS11,16 mHHS (2) HOOS (1)
WOMAC(1)		 -
Anterior cruciate ligament injury ACL-RSI25 KOOS (110) LKS (107) TAS (94)
Total Knee Arthroplasty KOOS19 KSS (45) WOMAC (33) OKS (23)
WORQ26
OKS14
WOMAC14
Knee osteoarthritis undergoing non-surgical treatment WOMAC16 WOMAC (17) KOOS (11) TAS (3)
Patellofemoral pain syndrome ADLS13 KOOS (8) LEFS (3) N/A
AKPS13
Recovery after Ankle Fractures A-FORM22 OMAS (21) FAOS (18) SMFA (12)
Achilles’ tendon rupture ATRS7 VISA-A (23) ATRS (9) FFI (9)
Chronic Ankle instability FAAM15 KS (20) FAAM (12) FAOS (10)
FADI15
Rheumatoid arthritis in the foot and ankle SEFAS21 FAAM (2) N/A N/A
Hallux valgus surgery MOXFQ10 FAOS (9) MOXFQ (7) PROMIS (5)
SEFAS10
Foot or ankle diseases MOXFQ23 N/A N/A N/A

Patient reported outcome measures recommended by systematic reviews are shown with citations on superscript. Patient reported outcome measures commonly used in seven representative orthopedic journals are shown in most-first, second and third. The number of clinical research articles using patient reported outcome measures is shown parentheses.
ACL-RSI: ACL-Return to Sport after Injury Scale; ADLS: Activities of Daily Living Scale; A-FORM: Ankle-fracture outcome of rehabilitation measure; AKPS: Anterior Knee Pain Scale; ATRS: Achilles tendon Total Rupture Score; CAIT: Cumberland Ankle Instability Tool; FAAM: Functional Ankle Ability Measure; FADI: Foot and Ankle Disability Index; FAOS: Foot and Ankle Outcome Score; HAGOS: The Copenhagen Hip and Groin Outcome Score; HOS: Hip Outcome Score; HOOS: Hip dysfunction and Osteoarthritis Outcome Score; IHOT: International Hip Outcome Tool; KOOS: Knee Injury and Osteoarthritis Outcome Score; KSS: Knee Society scores; LEFS: Lower Extremity Functional Scale; LKS: Lysholm knee scale; mHHS: Modified Harris Hip Score; MOXFQ: The Manchester-Oxford Foot Questionnaire; NAHS: Non-Arthritic Hip Score; OHS: Oxford hip score; OKS: Oxford Knee Score; OMAS: Olerud-Molander Ankle Score; SEFAS: Self-Reported Foot and Ankle Score; UCLA: University of California, Los Angeles (UCLA) activity scores; VISA-A: Victorian Institute of Sports Assessment-Achilles (VISA-A) score; WOMAC: The Western Ontario and McMaster Universities Osteoarthritis Index; WORQ: The Work Osteoarthritis or joint replacement questionnaire

Table 2.Joanna Briggs Institute Checklist for quality assessment
Studies 1 2 3 4 5 6 7 8 9 10 11
Veenhof 200616 Y Y N Y Y Y Y Y N Y N
Eechaute 200715 Y Y Y Y Y Y Y Y N Y Y
Thorborg 201011 Y Y N Y Y Y Y Y N Y Y
Lodhia 201112 Y Y N Y Y Y Y Y N Y Y
Tijssen 201117 Y N N Y Y Y Y Y N Y Y
Kearney 20127 Y Y Y Y N Y U Y N Y Y
Esculier201313 Y Y Y Y Y Y Y Y N Y Y
Thorborg 201518 Y Y N Y Y Y Y Y N Y Y
Schrier 201510 Y Y Y Y Y Y N Y N Y Y
Ramisetty 201520 Y N Y Y Y Y Y Y N Y Y
Harris 201614 Y Y Y Y Y Y Y Y N Y Y
Gagnier 201726 Y Y Y Y Y Y Y Y N Y Y
Jia 201823 Y Y Y Y Y Y Y Y N Y Y
Gagnier 201825 Y Y Y Y Y Y Y Y N Y Y
Gagnier 201827 Y Y Y Y Y Y Y Y N Y Y
Ng 201822 Y Y Y Y Y N Y Y N N Y
Ortega-Avila 201921 Y Y Y Y Y Y Y Y N Y Y
Reynaud 202019 Y Y Y Y Y Y Y Y Y Y Y
Impellizzeri 202024 Y Y Y Y Y Y Y Y N Y Y

Questions:
1: Is the review question clearly and explicitly stated?
2: Were the inclusion criteria appropriate for the review question?
3: Was the search strategy appropriate?
4: Were the sources and resources used to search for studies adequate?
5: Were the criteria for appraising studies appropriate?
6: Was critical appraisal conducted by two or more reviewers independently?
7: Were there methods to minimize errors in data extraction?
8: Were the methods used to combine studies appropriate?
9: Was the likelihood of publication bias assessed?
10: Were recommendations for policy and/or practice supported by the reported data?
11: Were the specific directives for new research appropriate?
Y: Yes; N: No; U: Unknown

From the seven representative journals, 4879 studies were reviewed. The three most commonly used PROMs were matched to the 15 specific pathologies or surgeries and also reported in Table 1. These results identified consistency between recommended PROMs and utilization in clinical research for only two of the 15 lower extremity pathologies or surgeries. This included the use of the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) and the Copenhagen Hip and Groin Outcome Score to assess outcomes (HAGOS) for those with knee osteoarthritis and groin pain, respectively.

DISCUSSION

The main finding of this umbrella review was that a total of 20 PROMs have been recommended for assessing the clinical outcomes for those with 15 specific hip, knee, ankle, and/or foot pathologies or surgeries. In addition, there was a discrepancy between PROMs that were recommended by systematic reviews and those used to assess clinical outcomes for the 15 specific lower extremity orthopedic pathologies or surgeries. To the authors’ knowledge, this is the first umbrella review that summarizes which PROMs are recommended by systematic reviews and compares these recommendations to what instruments have been used in clinical research articles in seven representative journals. The results of this study may help to improve uniformity of using PROMs and promote the use of PROMs that have the most appropriate psychometric properties when reporting treatment outcomes for those with lower extremity pathologies or surgeries.

There was an identified discrepancy between the recommended PROM and those utilized in published clinical research for 13 of the 15 pathologies. The most notable discrepancy occurred for those with non-arthritic intra-articular hip joint pain and anterior cruciate ligament injury. The modified Harris Hip Score (mHHS) and Knee Injury and Osteoarthritis Outcome Score (KOOS) were not recommended but were the most commonly used for those conditions, respectively. Considering the KOOS was recommended for use in total knee arthroplasty, clinicians should be cautious when using the KOOS in other conditions, such as non-arthritic intra-articular hip joint pain and anterior cruciate ligament injury. It should be noted that some of the recommendations of earlier systematic reviews of PROMs had been updated in later systematic reviews. This occurred for total hip arthroplasty, hip arthroscopy, non-arthritic joint pain, and total knee arthroplasty.11,12,14,17–20,24,26 The most updated recommendation included the use of following: the WOMAC for total hip arthroplasty, International Hip Outcome Tool-12 (iHOT-12), International Hip Outcome Tool -33(iHOT-33), and HAGOS for hip arthroscopy and non-arthritic hip joint pain, and the KOOS and Work Osteoarthritis or Joint Replacement Questionnaire (WORQ) for total hip arthroplasty (WORQ).14,19,24,26

The scores obtained from a PROM cannot be accurately interpreted if the PROM does not have established psychometric properties including evidence for validity, reliability, and responsiveness. Many of the most commonly used PROMs may be classified as legacy instruments and were not developed with contemporary methods. For example, the mHHS was the most common instrument used for those with non-arthritic intra-articular hip joint pain and contains items from the 1969 original Harris Hip Score.28 Alternatively, the Hip Outcome Score (HOS) was developed in 2005 using Item Response Theory (IRT) for item selection29 while the iHOT-12 and iHOT-33 were developed in 2012 using advanced item development from patient input.30,31 Clearly the standards behind developing, appraising, and guiding recommendations for PROM use has changed over the past 20 years. This includes the use of the Consensus-based Standard for the Selection of Health Measurement Instruments (COSMIN) checklist.32 PROMs that were developed before 2000 include the Knee Society scores (KSS),33 Victorian Institute of Sports Assessment-Achilles (VISA-A),34 WOMAC,35 KOOS,36 and Oxford Knee Score (OKS).37 Of these, only the WOMAC and KOOS were recommend for use in systematic reviews. Researchers continue to use legacy instruments so that recent clinical outcomes could be compared to previously published research. However, the scores obtained from PROMs that were not recommend because of lacking psychometric evidence must be cautiously interpreted.

It is obvious that a large number of PROMs have been developed. The reason for so many PROMs is unclear. PROM use in clinical assessment became common in 1990s with many new and different PROMs rapidly becoming available. The science behind PROM development did not advance as quickly. This may be due to the fact that researchers had little objective means to evaluate and compare these new PROMs. Also, researchers and clinicians often had their own ideas of critical items that should be included and, therefore, would often create their own PROMs. Implementation of the COSMIN and similar tools allowed the methodological quality of PROMs properties to be reported. In this umbrella review, 11 of the 19 systematic reviews used the COSMIN and six of the remaining eight systematic reviews used a similar evaluation checklist to assess the PROM quality. These evaluation checklists use similar criteria to assess the psychometric properties of PROMs. PROMs which had poor psychometric properties were not recommended by systematic reviews. As the means to develop, evaluate, and compare PROMs becomes better understood, it is hopeful that clinicians and researcher will use PROMs that are recommended based on their psychometric properties.

Limitations

This umbrella review has some limitations that need to be acknowledged. This review assessed the most commonly used PROMs by looking at publications within the past 10 years from only seven journals. However, the seven journals were chosen to be representative because of their wide readership, impact factor, and frequent publications on orthopedic clinical outcomes for the hip, knee, ankle, and foot. These methods were similar to those utilized in previously published studies.1,8 Second, this study only covered instruments that were recommended in systematic reviews. There may be other PROMs that have not been included in these systematic reviews with adequate psychometric properties.

Conclusion

The results of this umbrella review indicate that a total of 20 PROMs have been recommended for assessing clinical outcomes in those with 15 specific lower extremity orthopedic pathologies or surgeries. There is a discrepancy between PROMs that were recommended by systematic reviews and those used to assess clinical outcomes for the 15 specific lower extremity orthopedic pathologies. The results of this study may help to produce more uniformity in the use of PROMs that have the most appropriate psychometric properties when reporting treatment outcomes for those with lower extremity pathologies or surgeries. Further research is needed to identify the available language versions these recommend PROMs which may allow international researchers and clinicians to more consistently implement PROMs with the most appropriate psychometric evidence.

Conflicts of Interest

The authors certify that they have no affiliations with or financial involvement in any organization or entity with a direct financial interest in the subject matter or materials discussed in the article.

Funding

There is no financial support for the research.