Do the DSM Decision Trees Improve Diagnoatic Ability

Please download to get full document.

View again

of 16
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Documents

Published:

Views: 7 | Pages: 16

Extension: PDF | Download: 0

Share
Related documents
Description
Experiment 1 examined whether the use of the DSM-III-R decision trees increased the accuracy of DSM-III-R diagnoses. Results indicated that the use of the decision trees interacted with the level of DSM-III-R experience to affect diagnostic accuracy. The use of the decision trees resulted in a modest increase in diagnostic accuracy for participants with less DSMIII- R experience; for participants with more DSM-III-R experience, the use of the decision trees had no significant effect on diagnostic accuracy. Experiment 2 examined whether the use of the DSM-III-R decision trees increased the accuracy and confidence and decreased the time of DSMIII- R diagnosis across participants with varying levels of DSM-III-R experience. The primary analyses consisted of a 33232-multivariate analysis of variance (MANOVA) to determine whether the use of the decision trees increased diagnostic accuracy and diagnostic confidence and decreased diagnostic time. Results indicated (1) the experienced participants made more accurate diagnoses than the less-experienced and no-experience participants; (2) the decision trees, combined with practice, increased class diagnostic accuracy and decreased diagnostic time; and (3) participants were more confident in their diagnosis when they used the decision trees than when they did not use the decision trees. Supplementary analyses consisted of two one-way analysis of variance (ANOVA) procedures and indicated that participants’ preference for and knowledge of how to use the decision trees did not significantly affect their diagnostic accuracy.
Transcript
  Do the DSM Decision Trees ImproveDiagnostic Ability?  Robert D. Morgan Oklahoma State University  Kenneth R. Olson, Randy M. Krueger,Richard P. Schellenberg, and Thomas T. Jackson Fort Hays State University Experiment 1 examined whether the use of the DSM-III-R decision treesincreased the accuracy of DSM-III-R diagnoses. Results indicated that theuse of the decision trees interacted with the level of DSM-III-R experienceto affect diagnostic accuracy. The use of the decision trees resulted in amodest increase in diagnostic accuracy for participants with less DSM-III-R experience; for participants with more DSM-III-R experience, the useof the decision trees had no significant effect on diagnostic accuracy.Experiment 2 examined whether the use of the DSM-III-R decision treesincreased the accuracy and confidence and decreased the time of DSM-III-R diagnosis across participants with varying levels of DSM-III-R experi-ence. The primary analyses consisted of a 3  2  2-multivariate analysisof variance (MANOVA) to determine whether the use of the decision treesincreased diagnostic accuracy and diagnostic confidence and decreaseddiagnostic time. Results indicated (1) the experienced participants mademore accurate diagnoses than the less-experienced and no-experienceparticipants; (2) the decision trees, combined with practice, increasedclass diagnostic accuracy and decreased diagnostic time; and (3) partici-pants were more confident in their diagnosis when they used the decisiontrees than when they did not use the decision trees. Supplementary analy-ses consisted of two one-way analysis of variance (ANOVA) proceduresand indicated that participants’ preference for and knowledge of how touse the decision trees did not significantly affect their diagnostic accuracy.© 2000 John Wiley & Sons, Inc J Clin Psychol 56: 73–88, 2000. Robert D. Morgan is now a postdoctoral fellow in forensic psychology in the Department of Psychiatry at theUniversity of Missouri-Kansas City.Correspondence concerning this article should be addressed to Robert D. Morgan, Department of ForensicServices, Western Missouri Mental Health Center, 600 E. 22nd Street, Kansas City, MO 64108; e-mail:RobertDMorgan@juno.com JOURNAL OF CLINICAL PSYCHOLOGY, Vol. 56(1), 73–88 (2000)© 2000 John Wiley  &  Sons, Inc. CCC 0021-9762/00/010073-16  IntroductionClinical diagnosing is salient to the helping process as it is critical to developing a suc-cessful treatment plan (Stout, 1991, p. 141), facilitates meaningful communication amongmental health professionals (Malt, 1986), and is a requirement for insurance reimburse-ments; however, to be useful in clinical and research settings, diagnoses must be validand reliable. Several studies have pointed out the problems of low reliability in clinicaldiagnosing (e.g., Ash, 1949; Mehlman, 1952; Spitzer & Fleise, 1974).Clinicians and theorists alike have attempted to develop a more reliable diagnosticclassification system. The Diagnostic and Statistical Manual of Mental Disorders (DSM)has been under continual revision since the first edition was published in 1952 (AmericanPsychiatricAssociation, 1952). Each edition of the DSM has sought to improve its clinicalusefulness with efforts being directed at overcoming the serious weaknesses in reliabilityand validity encountered with the previous editions (Carson, Butcher, & Coleman, 1988).Ward, Beck, Mendelson, Mock, and Erbaugh (1962) indicated three reasons for diag-nosticdisagreement:patientinconsistencies,diagnosticianinconsistencies,andmoreimpor-tantly, inadequacies in the nosology. Revised editions of the DSM, with their attempts atimproving reliability, take aim at these problems. In an attempt to increase the validity of diagnosing, DSM-III-R revisions have consisted of major changes in the diagnostic dis-orders and their criteria rather than altering the DSM criteria-based format (e.g., Vaglum,Friis, Vaglum, & Larsen, 1989).Astudy by Malt (1986) demonstrated that the DSM-III classification system is supe-rior in reliability to other systems of classification. Further evaluations of the DSM-IIIsuggested that its reliability was enhanced over previous editions (Carson et al., 1988).The extensive field trials conducted by Spitzer, Forman, and Nee (1979) are furtherevidence of the increase in diagnostic reliability of the DSM-III over previous versions,and Adams and Cassidy (1993) concluded that “DSM-III and its successors represent adecided improvement over previous efforts” (p. 23).In addition to reliability, the accuracy of clinical diagnosis must be considered and isespecially important with recent emphasis in developing and identifying specific treat-ments for specific DSM-III-R disorders (e.g., American Psychiatric Association, 1989).Areliable diagnostic system is of little use if it does not result in more accurate diagnoses.While DSM-IV is a continued attempt at developing the most reliable and accurate diag-nostic system, efforts must continue to focus on facilitating the most reliable and accuratediagnostic decision-making process. One such attempt would be to use the manual in themost efficient manner possible, which may include the use of the decision trees.Clinical judgment or intuition is integral to the clinical diagnostician’s repertoire;however, as LeLaurin (1990) argues, judgment-based assessment is only as good as theobjectivity, reliability, and validity of the data collection used to make judgments anddiagnoses. Diagnosing relies heavily on clinical judgment and intuition, but even thisapproach is based on underlying baseline data as gathered by the clinician through expe-rience. As LeLaurin points out, this baseline data is only good if used objectively andreliably. In an analysis of medical decision-making, Elstein, Shulman, and Sprafka (1978)indicate that diagnostic errors occur as the result of mistakes in the analysis of largequantities of complex information, and diagnostic accuracy may be improved by the useof strategies aimed at “systemizing” a clinicians inferences (e.g., flow charts). Decisiontrees offer a systematic method for determining a diagnosis that may eliminate much of the subjective decision-making on the part of the clinician.The DSM decision trees do not follow a fixed methodological approach; rather theyfunction as a guide by encouraging clinicians to be more comprehensive in their consid- 74  Journal of Clinical Psychology, January 2000  erations of history, signs, and symptoms (Reid & Wise, 1989). The DSM-III-R (Ameri-can PsychiatricAssociation, 1987) decision trees for differential diagnosis were developedto assist professionals “in understanding the organization and hierarchic structure of theclassification” (p. 377). It does not seem unreasonable to suggest that increased under-standing in this regard would result in increased diagnostic accuracy.Millon (1983) described what are probably two typical but contrasting views of theusefulness of the decision trees: on the one hand he commented that the decision trees are“likely to be considered an unnecessary encumbrance for routine diagnostic tasks, quiteimpractical for everyday decision making and perhaps most relevantly, abhorrent to cli-nicians accustomed to the diagnostic habit of ‘intuitive’ synthesis” (p. 809). Contrary toMillon’shypothesis,TimmermansandVlek(1992)indicatethat“decisionaidsaredesignedto solve difficult decision problems. The justification for using decision aids lies in theshortcomings of human judgment” (p. 50). Furthermore, complex problems require morecognitive effort and often result in the use of simplified decision strategies and a lesscomplete evaluation of information (e.g., Olshavsky, 1979; Payne, 1976). Millon (1983)also appears to recognize the potential of the decision trees as he observed that “shouldthe method guarantee significantly greater diagnostic accuracy . . . then it might gain asufficient following to override the inertia of traditional practice” (p. 809).The DSM decision trees appear to be a novel experiment by the DSM-III task forcethat has not been taken too seriously as there are no studies that have investigated theiruse in diagnostic decision-making. In fact, no studies have investigated the use of theDSM decision trees in diagnostic decision-making. The purpose of these experimentswas to evaluate the effects of the DSM decision trees on diagnostic accuracy. Experiment1 examined whether the use of the decision trees increased the accuracy of DSM-III-Rdiagnoses, and it was hypothesized that they would. The purpose of Experiment 2 was to:(a) replicate and expand Experiment 1, and (b) examine whether the use of the DSM-III-R decision trees increased the accuracy of diagnoses for those participants with lessDSM experience and also decreased the time of diagnoses. It was hypothesized that theuse of the decision trees would increase diagnostic accuracy across groups with variouslevels of DSM-III-R experience and that the use of the decision trees would decrease thediagnostic time across these groups.Experiment 1  Method Participants . Participants consisted of 15 graduate students in a Masters degree programat a midwestern university. The students had been enrolled in an advanced AbnormalPsychology and / or an Applied Practicum course and had received training (i.e., class-room lecture and practical experience) in the use of the DSM-III-R. Of the 15 partici-pants, 10 were female and 5 were male. Their ages ranged from 23 to 43 (  M   30.1,  SD  7.9). The participants had varying amounts of previous DSM-III-R experience and train-ing (  M     22.0 weeks;  SD    21.9); three of the participants were first-year graduatestudents; 11 were second-year graduate students; one was a third-year graduate student.  Materials . Ten case vignettes were selected randomly from the DSM-III-R casebook (Spitzer, Gibbon, Skodol, Williams, & First, 1989) and used as case studies for the par-ticipants to make their diagnoses. Two case studies were selected for each decision tree.For each category of decision trees (e.g., psychotic symptoms) two disorders were selectedrandomly, and then case studies representing those disorders were selected randomlyfrom the DSM-III-R casebook. Case vignettes were assigned randomly to each condition  Do the DSM Decision Trees Improve Diagnostic Ability?  75  (with and without the decision trees), and the order of presentation of the vignettes alsowas assigned randomly. Thus, each vignette had an equal opportunity to be in eithercondition and appear in any position (e.g., first, second, third, etc.).The participants used the DSM-III-R and photocopies of the five DSM-III-R deci-sion trees when making their diagnoses. An 11-item questionnaire concerning the use of the decision trees also was administered to the participants.The first six questions inquiredabout the participants’previous use, understanding, and attitude about using the decisiontrees, and possible future use of the decision trees. Participants responded to these ques-tions by making ratings on Likert-type scales ranging from one (“not at all”) to five(“very much”). The last five questions pertained to the method employed when using thedecision trees and demographics including sex, age, year in graduate school, and numberof weeks of DSM-III-R training. Procedure . The participants first were informed that they would receive five casevignettes and that they were to make anAxis I DSM-III-R diagnosis.They were instructedto use the method that they usually used when making their diagnoses with the exceptionthat if they usually used the decision trees provided in the manual they were to refrainfrom using them with these vignettes. The participants then were given five of the casevignettes and instructed to make anAxis I diagnosis for each case. They were told that theAxis I diagnosis was to come from one of 11 categories, all of which were represented inthe decision trees (e.g., schizophrenia, mood disorder, and anxiety disorder).Upon completion of the five vignettes, the participants were informed that they wouldbe given five more case vignettes, and that again they were to make anAxis I DSM-III-Rdiagnosis. This time however, they were to use the decision trees when making theirdiagnosis.The experimenter then gave two suggestions on how to implement the decisiontrees when making a diagnosis. Method one consisted of using the decision trees as acheck of their usual method of diagnosing (i.e., method used on the first five vignettes).Method two consisted of locating the appropriate decision tree and then proceeding downthe tree, referring to the diagnostic criteria in the manual as needed, until the participantsarrived at a diagnosis.The participants then were given the other five case vignettes (again representing thefive decision trees) and copies of the five DSM-III-R decision trees. They were instructedto implement the decision trees to make an Axis I DSM-III-R diagnosis from one of the11 categories using either one of the two methods suggested by the experimenter or anyother preferred method. When finished with these case vignettes, the participants wereasked to complete the questionnaire.  Results and Discussion The primary analysis was a one-way repeated-measure analysis of variance (ANOVA)design with (and without) decision trees being the independent variable. The purpose of the analysis was to determine whether the use of the decision trees increased diagnosticaccuracy. Accuracy scores were computed by determining the number of diagnoses onthe vignettes that correctly matched those of the casebook.Aparticipant’s accuracy scoresfor “with” and “without” the trees ranged from 0 to 5. The results indicated that using ornot using the decision trees had no significant affect on the participants’diagnostic accu-racy,  F  (1,13)  2.27,  p  0.156. Supplementary analyses were conducted in order todetermine whether other variables might interact with the use of the decision trees ineffecting diagnostic accuracy. These variables included (a) whether or not participantsliked using the trees, (b) the method participants employed when using the trees (Method 1 76   Journal of Clinical Psychology, January 2000
Recommended
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks