Test-Retest Reliability of Cardiorespiratory Parameters Assessed by Metamax3B During Walking in Persons with Stroke
Article Information
Compagnat M1,2,*, Daviet Jc1,2, Batcho Cs3,4, Perrochon A1, Salle Jy1,2, David R2, Mandigout S1
1HAVAE EA6310 (Handicap, Aging, Autonomy, Environment), IFRH, University of Limoges, 87042 Limoges, France
2Department of Physical Medicine and Rehabilitation in the University Hospital Center of Limoges, 87042 Limoges, France
3Center for interdisciplinary research in rehabilitation and social integration (CIRRIS), Centre intégré universitaire de santé et de services sociaux de la Capitale Nationale (CIUSSS-CN), Quebec (QC), Canada
4Department of Rehabilitation, Faculty of Medicine, Université Laval, Quebec (QC), Canada.
*Corresponding Author: Maxence Compagnat M.D, Department of Physical Medicine and Rehabiliation in the University Hospital Center of Limoges in France, 2 Avenue Martin Luther King, 87000 Limoges
Received: 10 April 2020; Accepted: 18 April 2020; Published: 09 April 2021
Citation: Compagnat M, Daviet Jc, Batcho Cs, Perrochon A, Salle Jy, David R, Mandigout S. Test-Retest Reliability of Cardiorespiratory Parameters Assessed by Metamax3B During Walking in Persons with Stroke. Cardiology and Cardiovascular Medicine 5 (2021): 259-271.
View / Download Pdf Share at FacebookAbstract
Introduction: No studies have explored the reliability of the test re-test of the portable respiratory gas exchange analyzer (Metamax3B) in stroke patients despite its increasing use in clinical research. The objective was to evaluate the test-retest reliability of the cardiorespiratory parameters measured by Metamax3B during walking at self-selected speed in individuals with stroke.
Material and Methods: This study included stroke individuals able to walk without human assistance. The participants were equipped with the Metamax3B, calibrated and installed according to the manufacturer's recommendations. The parameters, i.e. oxygen uptake (VO2), oxygen uptake by weight (VO2.kg-1), carbon dioxide production (VCO2), respiratory exchange ratio (RER), expiratory flow rate (VE), tidal volume (VT) and respiratory frequency (BF) were measured twice during 6 minutes of walking at self-selected speed in tests performed 2 days apart. The test-retest reliability of these parameters was evaluated using Bland-Altman analysis.
Results: Sixteen individuals with stroke sequelae were included (mean age 64.5±14.3 years). No significant difference was observed between the test retest measurements for all parameters except for RER (mean difference=0.02; p=0.01). The Bland Altman analyses showed a mean bias of less than 5% between the test retest measurements.
Discussion and Conclusions: The reliability of the cardiorespiratory parameters measured by Metamax3B during walking at self-selected speed in individuals with stroke was high, which encourages the use of this device in this population.
Keywords
Reliability; Stroke; Metamax3B; Metabolic cart; Physical activity; Walking
Reliability articles; Stroke articles; Metamax3B articles; Metabolic cart articles; Physical activity articles; Walking articles
Article Details
1. Introduction
International organizations recommend the use of oxygen uptake ( O2) to individualize the exercise intensities recommended for individuals with stroke [1]. Controlling the intensity of exercises has been proven to improve the benefits on the cardiorespiratory and functional abilities of stroke patients [2].
Over the past decade, a wide range of portable devices have been developed such as Oxycon Mobile (Carefusion, San Diego, CL, USA) Metamax3B (Cortex Medical, Leipzig, Germany), or Cosmed K4b [2] (Cosmed, Chicago IL, USA). The main interest of this type of monitor is to allow measurements of O2 outside of laboratory conditions to determine the metabolic solicitation of individuals during ecological activities. These devices therefore facilitate the measurement of respiratory gas exchanges during common physical activities such as walking on flat or irregular ground, walking up or down stairs, manual activities and sports activities. The main appeal of these devices is their reduced weight and size of the systems thanks to miniature sensors for measuring O2 and CO2 concentrations, as well as the development of smaller flowmeters [3]. In addition, the recording of variables requires an instant wireless connection with a central computer. Such drastic modifications in spiroergometry and data exchange systems may alter the measurement values compared to larger devices [4].
Reliability describes the ability of a device to accurately repeat a measurement of a given parameter on repeated trials. This is one of the parameters of the validity of a measuring device, along with accuracy. Reliability is fundamental because it is essential to know if a difference observed between two tests represents a real change or simply a measurement error [5].
The Metamax3B (Cortex, Leipzig, Germany) is a portable respiratory gas exchange analyzer consisting of a measuring module, a battery module and a wireless connection system. It is one of the 3 main respiratory gas exchange analyzers used in clinical practice and research [6], the two others being Oxycon Mobile (Carefusion, San Diego, CL, USA) and Cosmed K4b[2] (Cosmed, Chicago IL, USA). Although the Metamax3B is frequently described as less precise than Oxycon Mobile and Cosmed K4b [2, 6], it is widely used in healthy subjects and in individuals with chronic diseases such as strokes [7-11]. The reliability of the Metamax3B has been evaluated by numerous studies in healthy subjects [6, 12-14]. Perkins et al. (2004) evaluated the reliability of the Metamax3B in healthy subjects on 2 separate days. The authors reported a high test-retest reliability (intraclass correlation coefficient = 0.97, standard error measure = 0.03 – 0.08 L.min-1) [15]. Macfarlane et al. (2012) have also evaluated its validity using the Gas Exchange System Validator test (MedGraphics, MA, USA). This test has been carried out twice and resulted in a high test-retest reliability (ICC=1) with a technical measurement error (TEM) of 1.5% for O2 [14].
Although this device has shown high accuracy and reliability in healthy subjects, our experience shows that several factors can compromise its reliability in stroke patients. First, each individual present variable levels of impairment such as spasticity over time [16]. Some of these variations are not related to their recovery but may be related to attention level, stress, medication or intercurrent medical issues [17]. In addition, individuals affected with a stroke feel the effects of fatigue more than healthy subjects, which may set boundaries in their involvement [18]. Moreover, some complications of stroke, such as facial paralysis, can also complicate the use of breathing masks. All these factors can impact the reliability of the exploration of cardiorespiratory parameters carried out by portable respiratory gas exchange analyzers in this population.
To our knowledge, no work has evaluated the reliability of the measurement of O2 by Metamax3B during walking at self-selected speed in stroke individuals despite the publication of numerous studies on related topics [8-11]. Measuring cardiorespiratory parameters of walking is essential in stroke individuals because they have a prognostic value in terms of functional ability and social participation [19, 20]. Moreover, recent study reported that the required walking activity necessary to reach physical activity level recommended by the World Health Organization could be adjusted with the oxygen consumption during walking [8].
The objective of this work was to evaluate the reliability of the measurement of cardiorespiratory parameters measured by Metamax3B in individuals with stroke during walking at self-selected speed on flat ground.
2. Materials and Methods
2.1 Study design
This work is a cross-sectional study evaluating the test re-test reliability of the cardiorespiratory parameters reported by Metamax3B during a 6-minute walking activity in individuals with stroke sequelae.
2.2 Population
Participants were recruited in the Physical Medicine and Rehabilitation Department. The inclusion criteria were the following: a single stroke in any area of the brain confirmed by brain imaging, and the ability to walk without human assistance. The non-inclusion criteria were the following: acute cardiac or respiratory pathologies or decompensated chronic pathologies. Cardiac disorders were searched using the complementary examinations usually carried out in post-stroke assessment, i.e. electrocardiogram and cardiac ultrasound.
Motor function was evaluated using the Demeurisse motricity index [21], which quickly assesses a patient’s motor function using three tests per deficient limb and which is validated in stroke patients. The patient’s score is calculated from five stages of voluntary motor control from a scale to 0-100, in which a score of 100 is considered healthy [21, 22]. Spasticity was evaluated using the Modified Ashworth Scale (MAS) [23]. Autonomy related to activities of daily living was evaluated using the Barthel Index [24]. Each patient’s walking speed was estimated during the walking task by dividing the distance walked by the duration of the task, which was established at 6 minutes. All of these evaluations were performed by the same examiner for all participants.
The health professional responsible for the data collection informed the participants himself of the details of the protocol before registering their consent. This consent was transcribed in the database. The research protocol and this method of consent were accepted by an ethics committee (RCB: 2012-A01456-37).
2.3 Metabolic cart
The Metamax3B (Cortex Medical, Leipzig, Germany) is a device which measures gas volume using a bidirectional digital turbine [25]. The O2 and CO2 concentrations are measured using an electrochemical cell and an infrared analyzer. O2 and CO2 were calculated using standard metabolic algorithms based on the Haldane transformation [26]. Respiratory volume data and respiratory gas concentrations were transmitted live by telemetry to a computer. The system was paired to the Metasoft 3 software, version 3.7.0 SR2. The device was switched on at least 20 min before each use. It was calibrated according to the manufacturer’s recommendations with the following procedure: calibrating the gas analyzers every 14 days by using a reference gas (14.97% O2, 4.96% CO2, balance N2: ±0.02% absolute, Hong Kong Specialty Gases) and checking the calibration exposed to ambient air. Additionally, a volume calibration was performed every day using a standardized 3L syringe (5530 series, HansRudolph, Inc., MO, USA) [25].
2.4 Protocol
We tested the reliability of the measurement of cardiorespiratory parameters measured by Metamax3B in stroke individuals during a walking activity at self-selected speed, which is considered as a moderate-intensity physical activity (6). We have selected this activity as it is both the most common activity performed by stroke individuals in our department because of its impact on functional capacities and social participation. In addition, the metabolic intensity it requires [27] makes it a main activity for physical activity programs offered to our patients. We have selected a duration of the walk of 6 minutes in order to achieve metabolic stability in each subject [28]. Each walk was preceded by 5-10 minutes of rest in a sitting position on a chair to ensure a stable metabolic state. The test was carried out a few hours after meals and in the absence of exciting products that could cause a change in respiratory gas exchanges. The testing protocol was then carried out a second time, ideally 2 days after the first, with equivalent schedules and medical context (nursing, medical care, physical therapy) to avoid confounding factors that could modify the cardiorespiratory parameters measured during the test.
2.5 Cardiorespiratory parameters
The following physiological parameters were used as they are the most frequently reported in scientific work in the field [28, 29]: O2 (Lmin-1); O2 uptake by weight (mLkg-1min-1); CO2 (Lmin-1); respiratory quotient (RER); expiratory flow rate (VE) (Lmin-1); tidal volume (VT) (L); breath frequency (BF) (bmin-1).
2.6 Statistical analysis
We calculated the number of subjects required from the reliability results of the research by Vogler et al 2010 [13]. The mean difference in the test retest was 2.3% IC95%(1.9-2.6%) for O2. Taking into account this mean difference and dispersion we calculated a number of subjects of 12 individuals to obtain a statistically significant difference between the 2 test/re-test measures. Considering the risk of dropping individuals due to intolerance to wearing Metamax3B or medical problems in the interval between the test/re-test measures, we decided to include 15 individuals with stroke.
We calculated the difference between the 2 tests and its significance by means of a t-test in the case of a normal distribution and a Wilcoxon test for matched samples in the case of a non-normal distribution. Pearson correlation coefficients were calculated. The rule of thumb for interpreting the size of a correlation coefficient was: .90 to 1.00, very high correlation; .70 to .90, high correlation; .50 to .70, moderate correlation; .30 to .50, low correlation; .00 to .30, negligible correlation [30]. A Bland-Altman analysis was also carried out to assess the test retest reliability as recommended by Berchtold 2016 [31]. The significance rate was 0.05 for all statistical analyses. All the analyses were carried out by “blandr” and “tidyverse” packages in Rstudio (Version 1.2.5033) © 2009-2019 RStudio, Inc.
3. Results
16 patients with stroke sequelae were included in the test protocol. All participants were in the chronic phase of their stroke with moderate impairments (mean Demeurisse score in lower limb = 81 (43 – 100) ; MAS = 1 (0 – 3)). Their walking speed was 0.83±0.31ms-1 for the 1st test and 0.84±0.32ms-1 for the 2nd test. The 2nd test took place on average 1.5 0.8 days after the first. We did not find any statistical difference between the 2 evaluations on the different impairments and autonomy scores collected (Table1).
No specific adverse events were recorded and there was no discontinuation of the evaluation due to poor tolerance of Metamax3B.
A statistically significant difference was found between the 2 tests for the RER (mean difference=0.02; p=0.01). No statistical differences were found for all other comparisons. (Table 2).
Mean (SD) / Mediane (Q1-Q3) |
p-value |
||
D0 |
D1 |
D0 vs D1 |
|
Age (years) |
64.6 (56.8-73.5) |
N/A |
|
BMI (kgm-2) |
25 (23-29) |
N/A |
|
Stroke side (L/R) |
9/7 |
N/A |
|
Time since stroke (day) |
37 (30-285) |
38 (31-286) |
0.74 |
Index of Motricity of Demeurisse Lower Limb (/100) |
78 (59-100) |
81 (59-100) |
0.98 |
Index of Motricity of Demeurisse Upper Limb (/100) |
80 (69-100) |
80 (69-100) |
0.96 |
Spasticity (Ashworth scale modified) (/4) |
1 (0-2) |
1 (0-2) |
0.78 |
Barthel Index (/100) |
88 (69-100) |
88 (69-100) |
0.88 |
FACm (/8) |
6 (4-7) |
6 (4-7) |
0.96 |
Speed (ms-1) |
0.83 (0.31) |
0.84 (0.32) |
0.43 |
Table 1: Characteristics of the population. SD: standard deviation; BMI: Body Mass Index; FACm: Functional Ambulation Classification modified; Speed (walking at self-selected speed) D0: first exploration; D1:second exploration; N/A: Not adapted.
Table 2: Test-retest reliability of cardiorespiratory parameters measured by Metamax3B in individuals with stroke. ( O2: oxygen consumption; CO2: carbon dioxide production; O2.kg-1: oxygen consumption per kilogram; RER: respiratory exchange ratio; VE: expiratory flow; VT: Tidal volume; BF: breath frequency; r: Pearson correlation coefficient; IC: confidence interval; MB : Mean Bias of Bland-Altman analysis ; MB% : Mean Bias expressed in percentage of the mean of the variable assessed ; Upper LoA : Upper limits of agreement of Bland-Altman analysis ; Lower LoA :Lower limits of agreement of Bland-Altman analysis ; RMSE: root mean square error; % RMSE: RMSE expressed as a percentage of the measured value. D0: First exploration; D1: second exploration).
The Bland-Altman analysis showed mean bias below 5% for all measured parameters. (Table 2 ; Figure 1)
4. Discussion
This work showed high test retest reliability of O2(L.min-1), O2.kg-1 (mL.min-1.kg-1), VT, VE and BF measured by Metamax3B in individuals with stroke during walking tests at self-selected speed. The reliability parameters were high with mean bias of Bland Altman analysis below 5% between the 2 evaluations performed 2 days apart.
These results are in line with previous work on the validity of Metamax3B for this type of exploration. Indeed, Macfarlane and Wong reported excellent reliability values (ICC=1) in a study comparing the values of O2, CO2 and VE between Metamax3B and a gas calibrator [14]. Other similar studies have reported equivalent reliability parameters [32-34]. This reinforces the evidence that the Metamax3B is a reliable device for measuring respiratory gas exchange parameters.
Our main concern before this study was that the stroke sequelae would lead to a lack of reliability in the measurement of cardiorespiratory parameters due to numerous reports that stroke individuals have variable impairments [17], ways of walking [35], and show increased fatigue during exercise [36]. Several studies reported similar results regarding portable respiratory gas exchanges analyzers from other manufacturers. Eng et al. reported an ICC of 0.96 for O2 values measured by the KB1-C portable respiratory gas exchange analyzer (KB1-C, Aerosport) during a six-minute walk test (6MWT) in 12 stroke subjects [37]. Stookey et al. also reported an ICC of 0.90 for O2 values measured by the K4b2 portable respiratory gas exchange analyzer (K4b2; COSMED USA, Chicago IL) during a 6MWT in 23 stroke subjects [38]. Our results confirm the reliability of this type of exploration in stroke individuals; furthermore, for the first time, it provides reliability results for Metamax3B in stroke individuals, which encourages its use in this population.
We found a significant difference between the 2 trials for the RER (MD=0.02; p=0.01). However, the RER is considered by many works as the least reliable parameter. Indeed, Tang et al. reported an ICC of 0.58 for the RER peak during stress tests in 20 stroke subjects [39]. Stookey et al. also reported an ICC of 0.64 (0.33-0.82) for RER in a 6MWT test in 23 stroke subjects [38]. Nonetheless, the ICC of RER was better in our study than in the aforementioned studies. We believe that this parameter has a lower reliability because it is calculated from O2 and CO2, and thus multiplies the reliability errors of both O2 and CO2.
This study boasts several research and clinical practice interests. Indeed, in recent years, many studies have been published based on the validity of Metamax3B on healthy or non-stroke pathological populations [8-10, 27, 40, 41]. This work is the first to explore the reliability of Metamax3B and thus confirms the validity of the previously studies which used the Metamax3B in their explorations in stroke individuals. In addition, on a clinical point of view, this device could be used routinely as it is well-tolerated by patients. Of the 32 explorations we performed, there was no complaint about wearing Metamax3B. It is also easy to use and can be installed in minutes. Its use in common clinical practice would enable medical practitioners to evaluate the O2 during walking at the beginning and end of rehabilitation programs, for example. It would thus make it possible to assess the improvements in the oxygen cost of walking, a parameter calculated by dividing O2 by the self-selected walking speed at the metabolic plateau which reflects a patient’s recovery of walking efficiency [20]. The oxygen cost of walking has a high prognostic value in terms of walking ability [42] and social participation in stroke patients [43, 44].
5. Limitations
There were several limitations to this work. First, the small number of included subjects (16 patients) limits its generalization to the entire stroke patient population. However, this number of participants was determined on the basis of previous work [13] to obtain sufficient power to show statistically significant differences. This is what was found for the RER parameter despite a very small difference between the 2 measures (mean difference=0.02; p=0.01). Another limitation is the type of physical activity we have chosen. On the one hand, walking at self-selected speed has the advantage of being an ecological activity that is well tolerated by patients and crucial in terms of autonomy and social participation [45, 46]. On the other hand, it could be interesting to include various physical activities, e.g. walking up and down stairs or manual activities to assess the reliability of the Metamax3B during activities with variable stress intensities. The incorporation of various activities represents an interesting lead in future work investigating the reliability of portable respiratory gas exchange analyzers. Finally, regarding the choice of parameters, we could also have integrated the heart rate and thus the O2 pulse. However, many studies have shown the extreme variability of heart rate [47] in stroke patients due to cardiac co-morbidities and cardiotropic treatments [48], which explains why this parameter is rarely used in clinical practice.
6. Conclusions
This work showed that the measurement of cardiorespiratory parameters such as O2, O2.kg-1, CO2, VE, BF and VT by Metamax3B during a walk at self-selected speed in individuals with stroke shows satisfying reliability. These results encourage the use of portable respiratory gas exchange analyzers in stroke population to assess the cardiorespiratory parameters.
Disclosure
The authors have no competing interest to declare.
Acknowledgments
We thank all our patients for volunteering in this research. We thank our partners Autonom’Lab and European Network of Living Labs (ENoLL) and the region of Nouvelle-Aquitaine for their support throughout our work.
Financial support
This work was supported by the region of Nouvelle-Aquitaine and the laboratory of clinical research HAVAE (Handicap, Aging, Autonomy, Environment).
References
- Billinger SA, Arena R, Bernhardt J, et al. Physical Activity and Exercise Recommendations for Stroke Survivors: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke 45 (2014): 2532-2553.
- Saunders DH, Sanderson M, Hayes S, Kilrane M, Greig CA, Brazzelli M, et al. Physical fitness training for stroke patients. In: Cochrane Database of Systematic Reviews [Internet]. John Wiley and Sons, Ltd; 2016 [cited 2016 Jul 7].
- Meyer T, Davison RCR, Kindermann W. Ambulatory Gas Exchange Measurements - Current Status and Future Options. Int J Sports Med 26 (2005): S19-S27.
- Meyer T, Georg T, Becker C, Kindermann W. Reliability of Gas Exchange Measurements from Two Different Spiroergometry Systems. Int J Sports Med 22 (2001): 593-597.
- Polit DF. Assessing measurement in health: Beyond reliability and validity. Int J Nurs Stud 52 (2015): 1746-1753.
- Overstreet Bs, Jr Drb, Crouter Se, Rider Bc, Parr Bb. Portable open-circuit spirometry systems. J Sports Med Phys Fitness (2017).
- Brandes M, VAN Hees VT, Hannöver V, Brage S. Estimating energy expenditure from raw accelerometry in three types of locomotion. Med Sci Sports Exerc 44 (2012): 2235-2242.
- Compagnat M, Mandigout S, Chaparro D, Salle JY, Daviet JC. Predicting the oxygen cost of walking in hemiparetic stroke patients. Ann Phys Rehabil Med 61 (2018): 309-314.
- Compagnat M, Mandigout S, Chaparro D, Daviet JC, Salle JY. Validity of the Actigraph GT3x and influence of the sensor positioning for the assessment of active energy expenditure during four activities of daily living in stroke subjects. Clin Rehabil (2018): 9.
- Compagnat M, Salle JY, Mandigout S, Lacroix J, Vuillerme N, Daviet JC. Rating of perceived exertion with Borg scale in stroke over two common activities of the daily living. Top Stroke Rehabil. November (2017): 1-6.
- Faria GS, Polese JC, Ribeiro-Samora GA, Scianni AA, Faria CDCM, Teixeira-Salmela LF. Validity of the accelerometer and smartphone application in estimating energy expenditure in individuals with chronic stroke. Braz J Phys Ther (2018).
- Meyer T, Georg T, Becker C, Kindermann W. Reliability of gas exchange measurements from two different spiroergometry systems. Int J Sports Med 22 (2001): 593-597.
- Vogler AJ, Rice AJ, Gore CJ. Validity and reliability of the Cortex MetaMax3B portable metabolic system. J Sports Sci 28 (2010): 733-742.
- Macfarlane DJ, Wong P. Validity, reliability and stability of the portable Cortex Metamax 3B gas analysis system. Eur J Appl Physiol 112 (2012): 2539-2547.
- Perkins CD, Pivarnik JM, Green MR. Reliability and Validity of the VmaxST Portable Metabolic Analyzer. J Phys Act Health 1 (2004): 413-422.
- Rowland LP. Stroke, spasticity, and botulinum toxin. N Engl J Med 347 (2002): 382-383.
- Li S. Spasticity, Motor Recovery, and Neural Plasticity after Stroke. Front Neurol (2017).
- Marzolini S, Oh P, McIlroy W, Brooks D. The Feasibility of Cardiopulmonary Exercise Testing for Prescribing Exercise to People After Stroke. Stroke 43 (2012): 1075-1081.
- Lord SE, Rochester L. Measurement of Community Ambulation After Stroke: Current Status and Future Developments. Stroke 36 (2005): 1457-1461.
- Kramer S, Johnson L, Bernhardt J, Cumming T. Energy Expenditure and Cost During Walking After Stroke: A Systematic Review. Arch Phys Med Rehabil 97 (2016): 619-632.
- Demeurisse G, Demol O, Robaye E. Motor evaluation in vascular hemiplegia. Eur Neurol 19 (1980): 382-389.
- Collin C, Wade D. Assessing motor impairment after stroke: a pilot reliability study. J Neurol Neurosurg Psychiatry 53 (1990): 576-579.
- Bohannon RW, Smith MB. Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys Ther 67 (1987): 206-207.
- Mahoney FI, Barthel DW. Functional Evaluation: The Barthel Index. Md State Med J 14 (1965): 61-65.
- Cortex Medical, Metamax3B [Internet]. Accessed on 12.04.2018.
- Haugen HA, Chan L-N, Li F. Indirect Calorimetry: A Practical Guide for Clinicians. Nutr Clin Pract 22 (2007): 377-388.
- Compagnat M, Mandigout S, David R, Lacroix J, Daviet JC, Salle JY. Compendium of physical activities strongly underestimates the oxygen cost during activities of daily living in stroke patients. Am J Phys Med Rehabil October (2018).
- McArdle WD, Katch FI, Katch VL. Exercise Physiology: Energy, Nutrition and Human Performance. 4th Revised edition edition. Baltimore: Lippincott Williams and Wilkins (1996).
- Port I, Kwakkel G, Wittink H. Systematic review of cardiopulmonary exercise testing post stroke: Are we adhering to practice recommendations? J Rehabil Med 47 (2015): 881-900.
- Mukaka M. A guide to appropriate use of Correlation coefficient in medical research. Malawi Med J J Med Assoc Malawi 24 (2012): 69-71.
- Berchtold A. Test–retest: Agreement or reliability? Methodol Innov 9 (2016): 2059799116672875.
- Brehm M-A, Harlaar J, Groepenhof H. Validation of the portable VmaxST system for oxygen-uptake measurement. Gait Posture 20 (2004): 67-73.
- Laurent CM, Meyers MC, Robinson CA, Strong LR, Chase C, Goodwin B. Validity of the VmaxST portable metabolic measurement system. J Sports Sci 26 (2008): 709-716.
- Prieur F, Castells J, Denis C. A methodology to assess the accuracy of a portable metabolic system (VmaxST). Med Sci Sports Exerc 35 (2003): 879-885.
- Balasubramanian CK, Neptune RR, Kautz SA. Variability In Spatiotemporal Step Characteristics And Its Relationship To Walking Performance Post-Stroke. Gait Posture 29 (2009): 408-414.
- Duncan F, Kutlubaev MA, Dennis MS, Greig C, Mead GE. Fatigue after stroke: a systematic review of associations with impaired physical fitness. Int J Stroke 7 (2012): 157-162.
- Eng JJ, Dawson AS, Chu K. Submaximal exercise in individuals with stroke: Test-retest reliability and concurrent validity with VO2max. Arch Phys Med Rehabil 85 (2004): 113-118.
- Stookey AD, McCusker MG, Sorkin JD, et al. Test-Retest Reliability Of Portable Metabolic Monitoring After Disabling Stroke. Neurorehabil Neural Repair 27 (2013): 872-877.
- Tang A, Sibley KM, Thomas SG, McIlroy WE, Brooks D. Maximal Exercise Test Results in Subacute Stroke. Arch Phys Med Rehabil 87 (2006): 1100-1105.
- Mandigout S, Lacroix J, Ferry B, Vuillerme N, Compagnat M, Daviet J-C. Can energy expenditure be accurately assessed using accelerometry-based wearable motion detectors for physical activity monitoring in post-stroke patients in the subacute phase? Eur J Prev Cardiol 24 (2017): 2009-2016.
- Serra MC, Treuth MS, Hafer-Macko CE, Ryan AS. Increased Energy Cost of Mobility in Chronic Stroke. J Gerontol Geriatr Res 5 (2016).
- Polese JC, Ada L, Teixeira-Salmela LF. Relationship between oxygen cost of walking and level of walking disability after stroke: An experimental study. Physiother Res Int July (2017): e1688.
- Jarvis Hannah L, Brown Steven J, Price Michelle, et al. Return to Employment After Stroke in Young Adults. Stroke 50 (2019): 3198-3204.
- Franceschini M, Rampello A, Agosti M, Massucci M, Bovolenta F, Sale P. Walking Performance: Correlation between Energy Cost of Walking and Walking Participation. New Statistical Approach Concerning Outcome Measurement. Plos One 8 (2013): e56669.
- Oh DW. Community Ambulation: Clinical Criteria for Therapists? Reasoning and Decision-making in Stroke Rehabilitation. Int J Phys Med Rehabil (2013).
- Lord SE, McPherson K, McNaughton HK, Rochester L, Weatherall M. Community ambulation after stroke: how important and obtainable is it and what measures appear predictive? Arch Phys Med Rehabil 85 (2004): 234-239.
- Lees T, Shad-Kaneez F, Simpson AM, Nassif NT, Lin Y, Lal S. Heart Rate Variability as a Biomarker for Predicting Stroke, Post-stroke Complications and Functionality. Biomark Insights 13 (2018).
- Ruthirago D, Julayanont P, Tantrachoti P, Kim J, Nugent K. Cardiac Arrhythmias and Abnormal Electrocardiograms After Acute Stroke. Am J Med Sci 351 (2016): 112-118.