segunda-feira, 4 de maio de 2015

Effects of Augmented Exercise Therapy Time After Stroke

A Meta-Analysis

  1. Peter Langhorne, PhD MD

  1. From the Department of Physical Therapy and Research Institute for Fundamental and Clinical Human Movement Sciences (G.K., R.v.P.), VU University Medical Center, Amsterdam, the Netherlands; the Center of Excellence for Rehabilitation Medicine “De Hoogstraat” (G.K.), Rudolf Magnus Institute of Neuroscience, University Medical Centre Utrecht, the Netherlands; the Department of Rehabilitation Sciences and Center for Neurorehabilitation (R.C.W.), Sargent College of Health and Rehabilitation Sciences, Boston University, Boston, Mass; the Department of Epidemiology and Biostatistics (S.W.D.), School of Physical and Occupational Therapy, McGill University, Montreal, Quebec; the Department of Rehabilitation and Centre for Interdisciplinary Research in Rehabilitation and Social Integration (C.R.), Laval University, Quebec City, Quebec; the School of Health Professions and Rehabilitation Sciences (A.A.), University of Southampton, UK; the School for Physiotherapy (K.M.), University of Melbourne, Australia; the School of Psychology (N.L.), University of Nottingham, UK; the Centre for Health Services Studies (C.P.), University of Kent at Canterbury, UK; and the Academic Section of Geriatric Medicine (I.W., P.L.), Royal Infirmary, Glasgow, Scotland.
  1. Correspondence to Dr G. Kwakkel, Department Physical Therapy, University Hospital Vrije Universiteit, PO Box 7057, 1007 MB Amsterdam, the Netherlands. E-mail g.kwakkel@vumc.nl

Abstract

Background and Purpose— To present a systematic review of studies that addresses the effects of intensity of augmented exercise therapy time (AETT) on activities of daily living (ADL), walking, and dexterity in patients with stroke.
Summary of Review— A database of articles published from 1966 to November 2003 was compiled from MEDLINE, CINAHL, Cochrane Central Register of Controlled Trials, PEDro, DARE, and PiCarta using combinations of the following key words: stroke, cerebrovascular disorders, physical therapy, physiotherapy, occupational therapy, exercise therapy, rehabilitation, intensity, dose–response relationship, effectiveness, and randomized controlled trial. References presented in relevant publications were examined as well as abstracts in proceedings. Studies that satisfied the following selection criteria were included: (1) patients had a diagnosis of stroke; (2) effects of intensity of exercise training were investigated; and (3) design of the study was a randomized controlled trial (RCT). For each outcome measure, the estimated effect size (ES) and the summary effect size (SES) expressed in standard deviation units (SDU) were calculated for ADL, walking speed, and dexterity using fixed and random effect models. Correlation coefficients were calculated between observed individual effect sizes on ADL of each study, additional time spent on exercise training, and methodological quality. Cumulative meta-analyses (random effects model) adjusted for the difference in treatment intensity in each study was used for the trials evaluating the effects of AETT provided. Twenty of the 31 candidate studies, involving 2686 stroke patients, were included in the synthesis. The methodological quality ranged from 2 to 10 out of the maximum score of 14 points. The meta-analysis resulted in a small but statistically significant SES with regard to ADL measured at the end of the intervention phase. Further analysis showed a significant homogeneous SES for 17 studies that investigated effects of increased exercise intensity within the first 6 months after stroke. No significant SES was observed for the 3 studies conducted in the chronic phase. Cumulative meta-analysis strongly suggests that at least a 16-hour difference in treatment time between experimental and control groups provided in the first 6 months after stroke is needed to obtain significant differences in ADL. A significant SES supporting a higher intensity was also observed for instrumental ADL and walking speed, whereas no significant SES was found for dexterity.
Conclusion— The results of the present research synthesis support the hypothesis that augmented exercise therapy has a small but favorable effect on ADL, particularly if therapy input is augmented at least 16 hours within the first 6 months after stroke. This meta-analysis also suggests that clinically relevant treatment effects may be achieved on instrumental ADL and gait speed.
Key Words:
Stroke is a leading cause of disability among adults in developed countries. Any treatment that improves functional outcome can significantly reduce suffering and the financial burden of this illness on the individual, the family, and society. Rehabilitation is recognized as a corner stone of multidisciplinary stroke care.1 Two systematic reviews2,3 suggested that early implementation of intensive stroke rehabilitation is associated with enhanced and faster improvement of the performance of activities after stroke. Both groups found small but statistically significant summary effect sizes (SES) in favor of the group that spent more treatment time focused on activities of daily living (ADL). In a sensitivity analysis, larger overall effect sizes were found in studies that weighted individual effect sizes for the differences in amount of rehabilitation between experimental and control groups.3 In these 2 systematic reviews, however, methodological limitations of the primary studies, differences in organizational settings, and marked heterogeneity of patient characteristics proved to be major confounding factors.
In the past few years, new trials that provided augmented exercise training by physical and occupational therapists have been conducted. The results of these trials show results that range from no measurable benefits4–7 to significant effects on ADL.8–11 This discrepancy can be related to differences in: (1) methodological quality of the trials; (2) patient selection; (3) amount of contrast between the intensity of treatment in experimental and control groups; (4) differences in type, focus, and timing of intervention after stroke; (5) differences in outcome measures; and (6) statistical power to show true effects. In particular, there is much debate about the amount of therapy that is needed and whether there is a minimum threshold below which there is no benefit.
The purpose of the present research synthesis was to examine the effects of treatment time by reviewing studies evaluating the effects of intensity of exercise therapy in patients with stroke on ADL, gait, and dexterity. The hypothesis was that the extra treatment time provided to the augmented therapy group would result in clinically relevant improvements in ADL, walking speed, and dexterity. A secondary goal was to determine whether there is a minimum threshold of additional time provided to the experimental group below which no clinically relevant benefit might be expected.

Materials and Methods

Definitions

Stroke has been defined by the World Health Organization (WHO) as “a clinical syndrome typified by rapidly developing signs of focal or global disturbance of cerebral functions, lasting more than 24 hours or leading to death, with no apparent causes other than of vascular origin.”12
Exercise therapy was defined as “physical activity that is usually regular and done with the intention of improving or maintaining physical fitness or health” (PubMed [MEDLINE]), MeSH database, 2003) and included both physical therapy and occupational therapy. ADL was defined as performance of the basic activities of self-care, such as dressing, ambulation, and eating, and instrumental ADL (IADL), such as shopping, preparing meals, washing clothes, and pursuing hobbies. Both outcomes were included in the present analysis. Finally, treatment contrast was defined as the amount of time spent on exercise training for the experimental group minus that for the control group.3 Studies that investigated the effectiveness of constraint induced movement therapy or application of special equipment to augment exercise therapy, such as balance platforms, treadmills, biofeedback equipment, or robotics, were excluded from the present study. Two independent reviewers (G.K., R.v.P.) selected articles based on the title and abstract.

Study Identification

Potentially relevant literature was identified through computerized and manual searches. Two independent literature searches (R.v.P., H.K.) were conducted in the following databases: MEDLINE (1966 through November 2003), CINAHL (1982 through November 2003), Cochrane Central Register of Controlled Trials, EMBASE (1988 through November 2003), PEDro, DARE, SCISEARCH (1974 through November 2003), and Picarta. The search was performed for the period 1966 to November 2003 using the following keywords (MesH): cerebrovascular disorders, stroke, physical therapy, physiotherapy, occupational therapy, exercise therapy, rehabilitation, intensity, dose–response relationship, effectiveness, and randomized controlled trial. Bibliographies of review articles, empirical articles, and abstracts published in proceedings of conferences were also evaluated. Only articles written in English, German, or Dutch were included. Studies were included when: (1) patients had a diagnosis of stroke; (2) effects of the intensity of physical therapy and/or occupational therapy were presented in the article; (3) the outcome was measured in terms of ADL (including walking ability, dexterity, or IADL such as leisure therapy); and (4) the study was a randomized controlled trial (RCT).

Methodological Quality

The methodological quality of each study was assessed by 2 independent reviewers (R.v.P., G.K.) by applying an adapted methodological scoring list derived from a list developed by Kwakkel et al3 and Cambach.13 The original list contained 16 items, with each scored on a binary scale. Because the present review was focused only on RCTs, items 2 and 3 were deleted so that 14 questions remained to be addressed (Table I, available online at http://www.strokeaha.org). The revised tool showed good inter-rater reliability according to Cohen kappa (κ=0.85). When disagreement between the 2 reviewers persisted, a third reviewer made the final decision. Reviewers were not blind to author(s), institution(s), or journal. One reviewer (R.v.P.) extracted all relevant data. The following items were evaluated: (1) randomization and blinding procedures; (2) descriptions of dropouts and intention-to-treat analysis; (3) reliability and validity of assessment instruments; (4) control for cointervention(s); (5) comparability of baseline patient characteristics; and (6) presentation of amount of therapy provided.
View this table:
TABLE I. Operationalization of Validity Criteria (Modification of List Kwakkel et al3)

Quantitative Analysis

The abstracted data (mean age, type of stroke, numbers of patients in experimental and control group, days of treatment, average length of daily treatment in minutes, mean difference in change scores in ADL, and standard deviation [SD] of ADL scores in experimental and control groups at baseline) were entered into Excel for Windows. The formal statistical methods used to test the results of different trials have been described elsewhere.3 The effect size gi (Hedges’ g) for individual studies was established by calculating the difference between means of the experimental and control groups divided by the average population SDi.14 If necessary, means and SDi were requested from the respective authors. Otherwise, point estimates were obtained from the graphs of included articles by recording the bitmap coordinates after scanning the graphs into Microsoft Paint.
To estimate SDi for gi, baseline SDs of control and experimental groups were pooled (eg, Hedges, 1985). Because the gi tend to overestimate the population effect size in studies with a small number of patients, a correction was implemented to obtain an unbiased estimation gu. The impact of sample size was addressed by estimating a weighting factor wi for each study and applying more weight to effect sizes from studies with larger samples that resulted in smaller variances. Subsequently, gu of individual studies were averaged to obtain a weighted SES (Ť). Finally, the wi of each study were combined to estimate the variance of the SES.15 When information about point estimates and standard errors was lacking, the original authors were consulted. The effect size gu for individual studies was computed for degree of disability in day-to-day activities, walking speed, and dexterity. In addition, SESs expressed as number of standard deviation units (SDUs) were calculated for studies comparing effects of different intensities in rehabilitation in the chronic stage of stroke (>6 months after onset), and those initiated within 6 months of stroke. The fixed effects model was used to decide whether a SES was statistically significant. The homogeneity (or heterogeneity) test statistic (Q-statistic) of each set of effect sizes was examined to determine whether studies shared a common effect size from which the variance could be explained by sampling error alone.14,16 Although the Q-statistic underestimates the existing heterogeneity in meta-analysis, the percentage of total variation across studies was used by calculating I2, which reveals a better measure of the consistency between trials.17 When significant heterogeneity was found on the Q-statistic (or I2 values >50%),17 a random effects model was applied.18,19 Pearson correlation coefficients were calculated between individual effect sizes, and additional time exercising in the experimental groups, as well as the methodological quality of the studies. To investigate the effects of differences in treatment contrast in selected studies, a cumulative meta-analysis adjusted for treatment contrast was applied for the ADL outcome, using a random effects model. Finally, a sensitivity analysis was performed for studies with high and low treatment contrasts. Effect sizes were calculated in Excel for Windows XP, whereas SPSS 11.5 for Windows was used for statistical analysis. For all outcome variables, the critical value for rejecting H0 (ie, there is no evidence for augmented therapy time) was set at 0.05.

Results

The search strategy resulted in a list of 7483 citations. After selection based on title and abstract, 507 full articles were obtained. Thirty-two studies were identified as being relevant. Five studies used a pretest–posttest assessment design;20–24 3 studies included a control condition but no randomization;25–27 and 25 studies were RCTs.4–11,28–44 Despite being an RCT, the article by Peacock et al29 was also excluded because of lack of information about treatment contrast and missing point measures and estimates of variability. Four studies referred to the same patients, who had been reported in 3 RCTs, which had already been included in this meta-analysis.35,38,39,40 One study also included patients with traumatic brain injury;41 only the patients with stroke were included in the present analysis.
Table 1 shows the main characteristics of the 20 eligible studies included in the meta-analysis. Fourteen of these reported statistically significant effects for functional outcomes in favor of the group with augmented therapy time. In 6 studies, additional exercise therapy did not result in a significant difference in efficacy. In total, 2686 patients with stroke were involved. The start of therapy ranged from within the first week after stroke9,31,42 to >1 year after stroke onset.11,33,36 Seventeen of the 20 studies investigated the effects of intensity within the first 6 months after stroke, whereas in 3 studies the research protocol was initiated >6 months after stroke.11,33,36
View this table:
TABLE 1. Study Characteristics of Trials on Intensity of Stroke Rehabilitation
On average, the experimental group received twice as much physical therapy (44.5 minutes; SD, 30.8) and occupational therapy (13.9 minutes; SD, 23.6) daily as the control group (21.1 minute; SD, 18.0 and 7.0 minutes; SD, 16.8, respectively; Table 1). The additional time that exercise therapy was provided to the experimental group ranged from 132 minutes11 to 6816 minutes,30 with a weighted average of 959 minutes or ≈16 hours of additional therapy time per patient.

Methodological Quality

The results of the methodological quality scoring of the 20 RCTs are presented inTable II (available online at http://www.strokeaha.org). Initially there was disagreement between the 2 independent reviewers on 21 of the 266 criteria scored. Cohen kappa for agreement was 0.84. The methodological quality score ranged from 228,31 to 11 points.43
View this table:
TABLE II. Application of Methodological Criteria to Studies Investigating the Effects of Intensity of Exercise Therapy
Ten studies used a randomization procedure with concealed allocation.4,5,7,9–11,31,37,40,43 In 16 studies, the observers were blinded to treatment allocation.4–11,32–34,36,37,41,43,44 However, none of the RCTs reported blinded statistical analysis or adjunct (medical) interventions for each group, separately. With the exception of Stern et al,28 all studies described dropouts for the experimental and control groups, separately. In 14 studies, the patients in the 2 groups had had a first stroke and were comparable for age, ADL index, and type of stroke.4–7,9–11,32–34,36,41,43,44 In 12 studies, the originally scheduled exercise therapy time was reported,4–10,28,34,36,41,43 and in 11 trials the actual exercise therapy time was provided.5–7,9,11,30,32,34,41,43,44

Meta-Analysis: ADL

Pooling the effects was only possible for those studies assessing ADL as an outcome. Fifteen studies evaluated outcome with the Barthel Index, 1 study used the Functional Independence Measure motor,35 and 4 studies used other measures that assess ADL.4,28,30,31
After intervention, a small but statistically significant heterogeneous SES (χ2=37.65, P<0.05) was found in favor of augmented exercise therapy (SES [random] 0.13 SDU; CI, 0.03 to 0.23; Z=2.49, P<0.007) (Figure 1 and Table 2). The SES obtained was almost the same for the 9 studies (N=1570) that also measured IADL (0.23 SDU [fixed]; CI, 0.13 to 0.33; Z=4.40, P<0.001) (Figure 2).5–7,9,10,33,37,43,44
Figure 1. Meta-analysis of augmented exercise therapy trials on measuring ADL.
View this table:
TABLE 2. Summary Effect Sizes for ADL at the End of the Intervention (N=2686)
Figure 2. Meta-analysis of augmented exercise therapy trials on measuring instrumental ADLs.
A homogeneous nonsignificant SES was found (χ2=0.72, P=0.70) for studies in which therapeutic intervention was initiated after 6 months after stroke (0.07 SDU [fixed]; CI, −0.17 to 0.28; Z=0.49), whereas the homogeneous (χ2=28.61, P=0.10; I2=33.6%) SES was significant (0.15 SDU [fixed]; CI, 0.06 to 0.23; Z=3.24, P<0.001) when therapy was applied within the first 6 months.
A positive association was found between the additional number of minutes of therapy provided in the experimental group as compared with the control group with an unbiased effect size (r=0.393; P<0.058). Methodological quality, however, was negatively associated with effect size (rs=−0.438; P<0.053).
A cumulative meta-analysis, adjusted for differences in treatment contrast across all 20 studies, showed a gradual shift from no significant effect in studies with a low treatment contrast (0 SDU; CI, −0.30 to 0.30; Z=0.0; P=1.0) to statistically significant overall effect sizes when studies with a high treatment contrast were added (0.13 SDU; 0.03 to 0.23; Z=2.49; P=0.007). (Figure 3). Sensitivity analysis revealed significant homogeneous (χ2=7.07, P=0.60) SES for those studies that applied 15 hours or more (0.22 SDU; 0.07 to 0.37; Z=2.95; P=0.005) when compared with studies with a lower treatment contrast (0.08 SDU; CI 0.06 to 0.22; Z=1.10; P=0.136).
Figure 3. Cumulative meta-analysis (in minutes) of augmented exercise therapy trials on measuring ADL adjusted for treatment contrast.

Comfortable Walking Speed and Dexterity

Six (N=524) of 20 studies measured the effects of augmented therapy on walking speed.4,9,11,33,34,43 Pooling individual effect sizes revealed a significant homogeneous SES of 0.19 (SDU) (CI, 0.01 to 0.36; Z=2.12; P=0.017).
Another 5 studies (N=420) measured the effects of augmented therapy for upper extremity function using the Action Research Arm test.5,7–9,43 Pooling these studies using this outcome showed no significant SES in favor of augmented exercise therapy (0.03 SDU; CI, −0.13 to 0.19; Z=0.352; P=0.637) (Tables III and IV, available online at http://www.strokeaha.org) for comfortable walking speed and dexterity, respectively (related figures for comfortable walking speed and dexterity may be requested by e-mailing the first author).
View this table:
TABLE III. Summary Effect Sizes for Comfortable Walking Speed at the End of the Intervention (N=524)
View this table:
TABLE IV. Summary Effect Sizes for Dexterity at the End of the Intervention (n=627)

Discussion

From the present systematic review, it may be concluded that augmented exercise therapy time spent in exercise training in the first 6 months after stroke results in a small improvements in ADL. Pooling reported differences in ADL by applying a fixed and random effects model showed small but significant SES. The effects were mainly restricted to therapies focused on the lower limb and ADL in general, as well as to those studies conducted within the first 6 months of stroke.
The SES in the cumulative meta-analysis denotes an overall change of ≈4% to 5% in favor of more therapy time when a minimum of at least ≈16 hours of additional exercise therapy time is provided. We must acknowledge, however, that the findings do not allow us to be precise about optimal treatment contrast. Another important point of discussion is the clinical significance of such a small finding reflecting only a 1-point change (5%) in outcome for the Barthel Index in favor of augmented therapy. Acknowledging that >80% receive exercise therapy, one may argue from an epidemiological perspective that even this small change in ADL score is likely to have a disproportionately large impact for health care polices.1
The benefits of augmented therapy time were mainly related to studies that focused the extra time on the lower limb or general ADL and not to the 5 RCTs that provided additional therapy time to the upper limb. It is important to note, however, that an ADL outcome, such as the Barthel Index, is more sensitive to lower limb improvement than to that in the upper limb.9 Moreover, improvements in mobility are more easily obtained than improvements in dexterity. We also know that functional outcome of the upper limb at 6 months after stroke is closely related to the level of recovery achieved in the first month, at least in patients with a primary middle cerebral artery stroke.45 In addition, there are indications that gains in the upper limb may require more intense repetitive practice and may be limited to those with less severe upper extremity deficits.32,45,46 This finding also suggests that for patients who are expected to achieve at least some dexterity, every opportunity should be given to regain function in the affected upper limb. In contrast, those patients for whom a poor motor recovery without return of dexterity is anticipated should have treatment focused on achieving and maintaining a comfortable mobile arm and hand. Compensation strategies with the nonparetic arm should be fostered.47
Lastly, the findings of our meta-analysis showed that augmented therapy also may lead to improvements of ≈5% in IADL such as household and leisure activities. It should be noted, however, that the number of such studies (n=9) is limited.
Although, in the present study, the intensive rehabilitation groups received ≈16 hours more exercise therapy than the control group, considerable differences in the total amount of additional therapy provided, as well as in the timing and the focus of interventions, were observed. The augmented time of exercise therapy ranged from a minimum of 13211 to a maximum of 6816 minutes.30 Cumulative meta-analysis of studies showed a positive trend in favor of those studies that applied a larger treatment contrast between experimental and control therapies. This suggests that the treatment contrast should exceed 16 hours to promote significant differences in ADL and that this more intensive therapy should be provided in the first 6 months after stroke. Interestingly, no ceiling effect for therapeutic intensity, beyond which no further response is observed, was found in the present study. This finding is consistent with a recent RCT on the effects of additional rehabilitation intensity after brain injury.48 Increasing the number of hours of therapy per week given to adults recovering from brain injury accelerated the rate of recovery of personal independence. In agreement with this finding, Chen et al24 found in a retrospective analysis of 554 records of patients with stroke that gains on Functional Independence Measure were weakly, but significantly, related to therapy intensity and rehabilitation duration after controlling for other variables. Future studies should focus on the effects of larger treatment contrasts in stroke, either by increasing the intensity of exercise time in the experimental group and/or by restricting the therapy in the control group. However, this latter suggestion may cause ethical concerns about depriving control subjects of the usual and expected amount of treatment.9
The present meta-analysis has several limitations. First, we defined intensity and treatment contrast on the basis of differences in time that therapy was provided to the experimental and control groups. This is, of course, a crude estimate of the actual effort and energy that is spent in performing exercises.2,3 Other aspects, such as patients’ motivation, attention paid by the therapist, and time spent on home exercises, may have confounded the reported outcomes. Second, although all included studies investigated the effects of additional exercise therapy, the content of therapy differed between studies with regard to goals set and the type of reference treatment (or condition) applied. Finally, we may have missed relevant studies not published in scientific journals or published in languages other than English, German, or Dutch.
It should also be noted that a number of other factors may have influenced the present findings, including different intervention goals, treatment content in the experimental group and control groups, patient selection criteria, and outcome measures. For example, most studies investigated the additional effects of a particular method of treatment such as the neurodevelopmental approach,4–6facilitation exercise techniques,28 or task-specific exercise programs.9,34 In all but 1 study, the control group received some form of therapeutic intervention. In the RCT by Kwakkel et al,9 the affected limbs were immobilized applying an inflatable pressure splint. The increased treatment contrast between the intervention and control groups obtained by preventing these patients from an active motor learning process may have contributed to a relatively larger effect size for leg training compared with other studies. Finally, when considering the impact of intensity of rehabilitation on stroke outcome, it should be realized that the intensity of rehabilitation programs is often limited.49,50 For example, in stroke units the usual direct contact time may be as little as 4% of the total waking time. Ten hours of therapy per week (2 hours daily) represents only 9% of the waking time.51 It should also be acknowledged that >2 hours of therapy each day is not feasible for every patient or clinical setting because of inability to tolerate the extra therapy sessions or to limited personnel. Several studies have shown, however, that augmentation of functional oriented therapy may be achieved by applying “constraints” to the less affected arm, forcing the affected limb to be used during ADL for 6 hours per day during 2 weeks (ie, augmentation of 60 hours).52 From these studies, it may be hypothesized that a high dose of task-specific exercise training should be applied over a shorter period of time. Recent studies have also shown that the efficiency of limited therapeutic resources can be increased by using circuit training programs in which a group of patients is allowed to practice at different workstations simultaneously under the supervision of a therapist.46,53
Future studies should focus on the most cost-effective intensity of therapy in stroke rehabilitation and the identification of patients who should benefit most from early and intensive exercise therapy. Differences were found between studies comparing the effects of intensity in the chronic stage of stroke to those comparing effects within 6 months of stroke onset. It should be noted, however, that the absence of significant SESs for the 3 studies that started augmented exercise therapy time in the chronic phase may be caused by insufficient statistical power. Future studies should indicate if the effects of enhanced exercise programs are transient, suggesting an increased speed of functional recovery or sustained in the chronic phase if therapy is continued.39,51 Thus far, findings from trials using constraint-induced movement therapy in patients with an incomplete upper limb deficit suggest that intensive, task-specific exercising in the chronic stage after stroke may result in improved dexterity.52 Therefore, research on subjects likely to benefit from higher intensities of stroke rehabilitation should be part of the future research agenda. This review has demonstrated the lack of data to define the treatment contrast needed to optimize effects of rehabilitation. At least it suggests that treatment contrasts between control and experimental groups should be extended in future high-quality RCTs.

Acknowledgments

This part is a research project supported by a grant from the Royal Dutch Society of Physical Therapy (KNGF; grant reference number IKPZ/KNGF/05/2000). Chest, Heart & Stroke Scotland provided funding for Ian Wellwood and the PINTAS study. In addition, it is supported by a grant of ZonMw in the Netherlands (1435.0017). We thank Hans Ket from the VU Medical Library for his independent literature search.
  • Received April 19, 2004.
  • Revision received July 23, 2004.
  • Accepted August 3, 2004.

References

  1.  
     
  2.  
     
  3.  
     
  4.  
     
  5.  
     
  6.  
  7.  
     
  8.  
     
  9.  
     
  10.  
     
  11.  
     
  12.  
     
  13.  
     
  14.  
  15.  
  16.  
  17.  
     
  18.  
     
  19.  
  20.  
  21.  
     
  22.  
  23.  
  24.  
     
  25.  
     
  26.  
     
  27.  
     
  28.  
     
  29.  
  30.  
     
  31.  
     
  32.  
  33.  
     
  34.  
     
  35.  
  36.  
     
  37.  
     
  38.  
     
  39.  
  40.  
     
  41.  
  42.  
     
  43.  
  44.  
     
  45.  
  46.  
     
  47.  
     
  48.  
     
  49.  
     
  50.  
  51.  
     
  52.  
  53.  
     

Nenhum comentário:

Postar um comentário