•  

Keeping Current #03-6 Alternative And Complementary Therapies For Children And Youth With Acquired Brain Injury Part 2: Finding and Evaluating the Evidence This Keeping Current is one of a series of reports that discuss the effectiveness of rehabilitation interventions for children and youth with brain injury. When reading the Keeping Currents in this series, you will notice that they include research studies that involve children and youth with a variety of conditions. These studies were included because of the limited amount of research on rehabilitation interventions for children and youth with brain injury. We encourage you to be critical when deciding what information relates to your situation. The most relevant information will come from studies that address the specific condition in question. However, be aware that research findings from studies involving children with different conditions may be relevant when the causes of the conditions are similar (for example, brain injury and cerebral palsy are both caused by injury to the brain) or when the children's functional difficulties are the same. ©2003, Rosenbaum, P. & Stewart, D., CanChild Centre for Childhood Disability Research Support for the research to conduct this critical review comes from a grant from The Economical Insurance Group

Introduction

When considering different rehabilitation interventions for children and youth with brain injury, it is important to think about the evidence available to support their effectiveness. As identified in Part 1 of this Keeping Current, there is limited conventional evidence available to support the use of alternative and complementary therapies for children and youth with brain injury. This can make it difficult for families and service providers to make informed decisions about whether or not to use alternative and complementary therapies.

The purpose of this report is to support informed decision-making by providing guidelines for evaluating conventional and Internet evidence. This report focuses specifically on evidence about alternative and complementary therapies for children and youth with acquired brain injury, however, the guidelines can be applied to other types of information.

Conventional Evidence

In health care, conventional evidence typically comes from journals, textbooks, databases and agencies that specialize in evidence-based practice. Each of these sources is very likely to have been "peer-reviewed" by experts in the field. This means that one can have a reasonable degree of confidence that the material has some degree of credibility.

A review of conventional literature involves a literature search, which is usually done by someone who has access to multiple sources of medical and health-related literature. The findings of this type of literature search can be "critically appraised", which means that "rules of evidence" that have been developed in the field of epidemiology and evidence-based medicine are applied, to determine the worth or quality of the findings.

The rules of evidence focus on:

• the design of the study (with a randomized controlled trial considered to be the highest quality when assessing rehabilitation interventions)
• the population (type of people) being studied
• the measures used to assess the treatment
• the methods used to collect data and analyze the results.

Limitations or biases of a research study are examined closely to see if, and in what ways, they might have influenced the results.

Searching for conventional evidence:

Our search of conventional sources of information included the following steps:
• We looked for general information about alternative and complementary therapies, particularly in relation to children with acquired brain injury.
• We narrowed our search to those therapies that were used in the "rehabilitation" phase of recovery, meaning, after the first 6 months of "acute" treatment, and were applied to children under the age of 19 years.
• We focused on information written in the last 12 years (1990 - 2001).
• We reviewed current books and information about alternative or complementary therapies, along with the results of a survey of families and service providers to define what we meant by alternative and complementary therapies.
• We used the following terms as "keywords" in this search:
• General terms: alternative therapy or treatment or medicine, complementary therapy or treatment or medicine, acquired brain injury, traumatic brain injury, and children.
• Specific terms: acupuncture, AND, chiropractic, Chinese medicine, constraint induced therapy, craniosacral therapy, herbal medicine, homeopathy, massage, naturopathy, and therapeutic touch.

Evaluating conventional evidence:

The results of our conventional search are summarized in Table 1 of the appendix. Overall, the results can be summarized as "inconclusive", because there is insufficient evidence in the form of high quality research to draw conclusions about the effectiveness of alternative and complementary therapies. In other words, there have not been enough research studies published in the medical/health literature about any alternative therapies and children with ABI to conclude that they are either beneficial or harmful.

Where else can you look for information?

With the global explosion of information available in the public domain, people are using the Internet and other accessible sources to get and share information. The Internet offers a vast array of information from many different sources. Information can be posted on the Internet by anyone and it does not have to be reviewed. Therefore, one of the main challenges with Internet information relates to "critical appraisal" - how do we evaluate the information we find to know if it is good evidence? The following provides some guidelines for searching for and evaluating Internet information.

Internet Evidence

Searching for evidence on the Internet: To begin your search, try using a search engine (see the list at the end of this Keeping Current for suggested web sites). You can enter the topic in which you are interested (for example, acupuncture) in the search box. If you do not get very many "hits" (that is, web sites that are related to your topic), try using terms that are broader. For example instead of searching for a specific treatment, search for a general category such as "alternative therapy".

If you get too many "hits", try to focus your search on a more specific topic. There are a number of ways to focus your search:

• Add words that make your search more specific (for example, search for: acupuncture, brain injury, and children).
•  Check if the search engine has advanced search features and follow the instructions provided.
• Use the search engine's topic directory (a list of different categories of information available on the search engine). Click on the category you are interested in and you will be directed to related topics or web sites. For example, if you are looking for information about alternative therapy or brain injury, you could try clicking on the "health" section of a directory.

It is important to remember that Internet information can change daily. Also, every search is different and can provide different results. This is both a benefit and a drawback to finding information on the Internet.

Evaluating Internet information:

The most important skill for searching information is being able to examine the information carefully to know what is worthy of your attention. This will assist you in making informed decisions about the therapies available for children with acquired brain injury. There are some excellent books and web resources about evaluating information on the Internet. The bibliography at the end of this Keeping Current provides some examples of these resources. The following is offered as one strategy for evaluating Internet information.

A "who, what, why, how, when, where" approach to evaluating web based information:

• Who is the author of the information? Is the information signed? Are the authors from a reputable agency/association or government, or from a commercial source, such as the owner of a clinic or sponsor? Who is in charge of the web site? Has the information been "peer-reviewed" by experts in the field?
• Who is the treatment intended for? Is it being used with children with traumatic/acquired brain injury who are similar to the child you are concerned about, to whom the treatment would be applied? Remember that a treatment may work well with other types of conditions, or for people of different ages, but may be ineffective or even harmful to children with brain injury if it has not been carefully tested first with similar children.
• What is the information about? Is the treatment fully explained? Is the information reliable and accurate? What research or evidence is available to back it up?
• Why was the treatment developed? Is there a rationale behind the treatment (how and why it was developed) that makes sense? Is it proven in any scientifically credible way?
• How is the information presented? Is it presented in an unbiased way (one that avoids opinions and advertising)?
• When was this information written? Is it current? What's the date/last update?
• Where is the treatment available? Is it accessible and affordable enough for you to consider using it?

 

One example of this evaluation strategy is outlined in Table 2 at the end of this report. We evaluated one web site about "craniosacral therapy". A summary of our evaluation is provided here:

• The information is mainly descriptive and provides a very brief overview of the topic. The information is current.
• We don't know if the sources/authors are reliable or unbiased, but the information is presented objectively. We don't know if the information has been peer-reviewed by experts in the field.
• The information lists traumatic (acquired) brain injury as an appropriate condition for treatment, but nothing is described specifically about children with acquired brain injury.
• Some resources and further readings are listed.
• Further searching is needed to learn more about this therapy, and to find the evidence that is listed.

This appraisal is an example of information on the Internet that is descriptive and possibly biased, which makes it difficult to know if the treatment is useful or harmful. Furthermore, this information about one alternative and complementary treatment focuses mainly on adults, and is often used with a variety of conditions, so we don't know for sure if the treatment is effective with children with acquired brain injury.

What else can you do to evaluate information?

After your own appraisal, share your findings with other people whose opinion you trust - friends, relatives, your family doctor, other service providers and organizations - and ask them to comment on the information you found.

Weigh this "evidence" with your other evidence, which includes your own experiences, other people's information, and conventional information from sources such as published articles or books.

Conclusions

When searching for information, it is important to be able to find, access and evaluate the evidence that supports the use of a specific approach or raises questions about it. We believe that this is especially true when considering the use of alternative and complementary therapies.

The risks of unproven therapies include the possibility of unexpected side-effects; losses of time, money and opportunities to pursue other avenues; and the likelihood that a therapy that has not yet been studied is one that does not "work". Selecting therapies based on "good" evidence can decrease these risks and increase the likelihood that a child will reach his or her best potential. We encourage all consumers of therapies for children with acquired brain injury to use the guidelines presented to evaluate conventional and Internet information critically to help make informed decisions.

* For further discussion of alternative and complementary therapies for children and youth with acquired brain injury, please see Part 1 of this Keeping Current "Alternative and Complementary Therapies for Children with Acquired Brain Injury; Part 1: Controversies" published by CanChild in 2003.

Update written by:

Peter Rosenbaum, MD, FRCP(C), Professor of Pediatrics, McMaster University; Co-Director of CanChild Centre for Childhood Disability Research; and Canada Research Chair in Childhood Disability.

Debra Stewart, MSc. OT Reg. (Ont.), Assistant Clinical Professor, School of Rehabilitation Science and Associate Member, CanChild, McMaster University

Want to know more? Contact:

Peter Rosenbaum
CanChild Centre for Childhood Disability Research
Institute for Applied Health Sciences, Room 408
1400 Main St. W., Hamilton, ON L8S 1C7
Tel: 905-525-9140 x 27850 Fax: 905-522-6095
canchild@mcmaster.ca

Table 1: Summary of Findings of Conventional Search of "Evidence" on Alternative and Complementary Therapies for Children with Acquired Brain Injury

Sources	Search Findings	Summary of Critical Appraisal
Cochrane Collaboration & Cochrane Library An international agency committed to conducting and sharing systematic reviews of the health literature. It is accessed through the Canadian Cochrane Centre: http://hiru.mcmaster.ca/ cochrane/centres/canadian	Search 1: Acquired Brain Injury • 13 Systematic reviews or meta-analyses were found, with 2 reviews described as alternative therapy - neither focused on children with ABI. Search 2: Alternative/complementary therapy/medicine • 25 systematic reviews or meta-analyses were found, with 2 related to brain injury, neither focused on children.	With no reviews found specifically on our topic of interest, we cannot draw any conclusions about the effectiveness of alternative therapies for children with ABI.
Medline A large medical database of over 11 million records of journal articles dating back to 1963. It can be accessed through the National Library of Medicine (NLM) website: www.ncbi.nlm.nih.gov	Search 1: Review articles on ABI and alternative /complementary therapies • Two articles were appropriate for our topic: EEG operant conditioning for individuals with ABI, & craniosacral therapy for all conditions. • No review articles focused on children with ABI.	Search 1: EEG operant conditioning article only reviewed 3 studies of EEG bio-feedback treatment for adults with brain injury. Results indicate some promise, as there were reduced symptoms in all studies. Review of craniosacral therapy found insufficient evidence to support its use. One study reported negative side effects for adults with TBI.
	Search 2: Articles on ABI and a specific alternative or complementary therapy • Two articles were appropriate for our review: 1. Acupuncture and brain trauma; 2. Visual feedback for children with brain injury. Articles on other specific alternative therapies for people with ABI were found, but the study population was not children.	Search 2: The article on acupuncture presented 12 "cases" of people with ABI, which included some children. The quality of the study was poor, there was no control group, and results were mixed. The study of visual biofeedback was a single-case design of a child following a severe traumatic brain injury. The results showed that continuous visual biofeedback techniques were superior to traditional speech therapy.
Library data base search Many large libraries, at the national level (e.g., Librarian's Index to the Internet - see below) or at universities, have web sites with data bases that are accessible to the public.	Search for books on ABI, alternative therapy or complementary therapy • Many books have been published about ABI - some have chapters on alternative/complementary therapy. • Many books on alternative/complementary therapies were also found. They covered general and specific subjects and most focused on many conditions or populations.	Most of the information in books about ABI and alternative therapies is descriptive or anecdotal. Very little research evidence is provided in books, therefore one cannot conclude that the therapies as described are beneficial or harmful.

Table 2: Appraisal of Internet Information on an Alternative Therapy for Children with ABI

Topic	Craniosacral Therapy
Source	Yahoo! Health (we are sorry, this link no longer functions)
Who:	• written for Yahoo! Health by "Medical Economics Company Inc." • no description of authors provided, therefore we are unclear as to the objectivity of the information • the information about craniosacral therapy is written for all ages. Traumatic brain injury is listed as one of the conditions targeted by craniosacral therapy (but not specifically for children with ABI)
What:	• information is provided on what the treatment hopes to accomplish and how treatments are done. • Yahoo! Health is a reliable website, but we do not know if the source of information is reliable or accurate.
Why:	• the underlying belief for craniosacral therapy is briefly described. There is reference to "numerous case studies as proof of its effectiveness" although they are not described. • five resources are offered for further reading.
How:	• information is brief and objectively written, and it is stated that it is intended for general reference purposes only. • don't know about biases, but there are no advertisements.
When:	• copyright is 2001.
Where:	• 2 resources are offered for further information and referrals - both in the U.S.

References

Alexander, J. & Tate, M.A. (1999). Web Wisdom: How to Evaluate and Create Information Quality on the Web. Lawrence Erlbaum Associates.

Beck, Susan (1997). Evaluation Criteria: The Good, the Bad & the Ugly: or Why it's a Good Idea to Evaluate Web Sources. http://lib.nmsu.edu/Instruction/evalcrit.html (sorry, this link no longer functional).

Grassian, E. (2000). Thinking Critically about World Wide Web Resources. UCLA College Library(http://www2.library.ucla.edu/libraries/college/11605_12337.cfm).

United Health Foundation (2001). How to Evaluate Medical Resources on the Web. www.unitedhealthfoundation.org

These are some examples of Internet resources to get you started.

Please note: This list is not a complete list of sites, but ones that we have found to be useful.

General directories and search engines:

• Librarians' Index to the Internet (http://lii.org)
• About.com (http://www.about.com)
• Google (http://www.google.ca)
• Sympatico (http://www1.sympatico.ca)
• Yahoo (http://www.yahoo.com)

Specific health/medical directories and sites:

• HealthWorldOnline (http://healthy.net)
• PubMed (http://ncbi.nlm.nih.gov/PubMed/)
• PedInfo (http://www.pedinfo.org- sorry, this link no longer functional)
• Ask Noah (New York Online Access to Health) (http://www.noah-health.org)
• CanChild Centre for Childhood Disability Research (http://www.canchild.ca)

Specific directions and site for Alternative and Complementary Therapy OR Traumatic Brain Injury:

• National Centre for Complementary and Alternative Medicine (http://nccam.nih.gov)
• Ontario Brain Injury Association (OBIA) (http://www.obia.on.ca)
• Research and Training Centre on Community Integration of Individuals with Traumatic Brain Injury (http://www.mssm.edu/tbinet)