Abstract
Objective
To evaluate the consistency of patterns and pattern identification (PI), and the improvement in the diagnosis of blood stasis after the standard operating procedures (SOPs) were enhanced by using a diagnostic flflowchart. Methods
A total of 358 subjects who were admitted by the Jaseng Hospital of Korean Medicine between June 2013 and December 2014 were included in the study. Each patient was independently diagnosed by two experts from the same department. In 2014, the SOPs included the use of a diagnostic flflowchart for each indicator. Inter-observer consistency was assessed by simple percent agreement and by the kappa and AC1 statistics.
Results
The improvements in inter-observer agreement for the indicators (for all patients) and PI were greater in 2014 than in 2013. In 2013, the j-value measure of agreement between the two experts ranged from "poor" (κ=–0.051) to "good" (κ=0.968). The AC1 measure of agreement between the experts was generally high for the indicators and ranged from–0.010 to 0.978. In most cases, agreement was considerably lower when it was assessed by the j-values compared with the AC1 values. In 2014, the j-value measure of agreement on the indicators (for the subjects) generally ranged from poor to good, with the values ranging from–0.007 to 0.994. Moreover, the AC1 measure of agreement between the experts was generally high for all of the indicators and ranged from "moderate" (AC1=0.408) to "excellent" (AC1=0.996).
Conclusions
In four examinations, there was moderate agreement between the clinicians on the PI indicators of blood stasis. To improve clinician consistency (e.g., in the diagnostic criteria used), it is necessary to analyse the reasons for inconsistency and to improve clinician training. (Trial registration No. KCT0000916)
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Gunter R N, ed. Blood stasis-China’s classical concept in modern medicine. Elsevier; 2007.
WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region. Geneva: WHO; 2007.
Lee JA, Kang BK, Park TY, Lee HY, Kim JI, Park SU, et al. CORE-DITEC-BS (COnvergence REsearch of the DIagnostic TEChnology for Blood Stasis): study protocol. Eur J Integr Med 2015;7:417–422.
Yang DH, Park YJ, Park YB, Lee SC. Development of questionnaires for blood stasis pattern. J Korea Instit Orient Diagn 2006;10:141–152.
Yao K, Chu F, Wang J. A clinical epidemiological study of the qualtitative diagnosis scale of blood stasis syndrome. Chin J Integr Med 2011;17:200–204.
Wu HZ, Fang ZQ, Cheng PJ, eds. Introduction to diagnosis in traditional Chinese medicine. Singapore: World Century Publishing Corporation; 2013.
The Collaboration Group of Encephalopathy emergency of State Administration of Traditional Chinese Medicine. Guideline for diagnosis and therapeutic effect evaluation of stroke. J Beijing Univ Tradit Chin Med (Chin) 1996;19:55–56.
Kang BK, Park TY, Lee JA, Moon TW, Ko MM, Choi J, et al. Reliability and validity of the Korean Standard Pattern Identification for Stroke (K-SPI-Stroke) questionnaire. BMC Complement Alternat Med 2012;12:55.
Ko MM, Lee JA, Kang BK, Park TY, Lee J, Lee MS. Interobserver reliability of tongue diagnosis using traditional Korean medicine for stroke patients. Evid Based Complement Alternat Med 2012;2012:209345.
Ko MM, Park TY, Lee JA, Choi TY, Kang BK, Lee MS. Interobserver reliability of pulse diagnosis using traditional Korean medicine for stroke patients. J Alternat Complement Med 2013;19:29–34.
Grant SJ, Schnyer RN, Chang DH, Fahey P, Bensoussan A. Interrater reliability of chinese medicine diagnosis in people with prediabetes. Evid Based Complement Alternat Med 2013;2013:710892.
Gwet K. Computing inter-rater reliability with the SAS system. Stat Methods Inter-rater Reliab Assess 2003;3:1–16.
Jelles F, Van Bennekom CA, Lankhorst GJ, Sibbel CJ, Bouter LM. Inter- and intra-rater agreement of the rehabilitation activities profile. J Clin Epidemiol 1995;48:407–416.
Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990;43:551–558.
Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990;43:543–549.
Man YN, Liu XH, Wu XZ. Chinese medicine herbal treatment based on syndrome differentiation improves the overall survival of patients with unresectable hepatocellular carcinoma. Chin J Integr Med 2015;21:49–57.
Hua B, Abbas E, Hayes A, Ryan P, Nelson L, O’Brien K. Reliability of Chinese medicine diagnostic variables in the examination of patients with osteoarthritis of the knee. J Altern Complement Med 2012;18:1028–1037.
O’Brien KA, Abbas E, Zhang J, Guo ZX, Luo R, Bensoussan A, et al. Understanding the reliability of diagnostic variables in a Chinese medicine examination. J Altern Complement Med 2009;15:727–734.
O’Brien KA, Birch S. A review of the reliability of traditional East Asian medicine diagnoses. J Altern Complement Med 2009;15:353–366.
Arai YC, Nishihara M, Inoue S, Makino I. Kampo diagnostic procedure, fuku shin, could be a useful diagnostic tool for psychopathological patients suffering from chronic pain. Evid Based Complement Alternat Med 2013;2013:816216.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.
Gwet K, ed. Handbook of inter-rater reliability. Gaithersburg: MD: STATAXIS Publishing Company; 2001.
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the Korea Institute of Oriental Medicine, Republic of Korea (No. K15111)
Rights and permissions
About this article
Cite this article
Lee, J.A., Jung, J., Ko, M.M. et al. Inter-observer reliability of indicators and decision of pattern identification using diagnostic flowchart with traditional Korean medicine. Chin. J. Integr. Med. 23, 338–344 (2017). https://doi.org/10.1007/s11655-016-2580-4
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11655-016-2580-4