The Basic Knowledge Assessment Tool (BKAT) for
Emergency Department (ED-BKAT)©Critical Care Nursing
(Version One, 2006)
Introduction
Basic knowledge in critical care nursing is a body of knowledge beyond that required for licensure as an RN that the critical care nurse uses in order to provide safe nursing care to patients. Since safe practice is regarded as a moral and professional responsibility, basic knowledge is information that is necessary for entry into critical care nursing and represents the foundation for job performance.
A primary aim of in-service education in critical care nursing is to assure that staff nurses demonstrate an understanding of this basic knowledge. Because of on-going research studies (since 1979), publications, and the use of the BKAT over the past 29 years, it has become accepted as one standard for measuring basic knowledge in critical care nursing. To date, over 7,500 critical care nurse educators and nurse managers in the USA have requested and have received a copy of the BKAT to photocopy for use in their practice. Critical care nurses in over 23 different foreign countries have also requested and received a copy of one of the BKATs.
The ED-BKAT is a 100-item paper and pencil test that measures basic knowledge in critical care nursing in Emergency Departments. These items measure content related to the following areas of ED critical care nursing practice: cardiovascular, pulmonary, neurology, endocrine, renal, gastrointestinal/parenteral, OB/GYN, pediatrics, and other. The category ‘other’ includes such areas as drug abuse, trauma, psychiatric situations, rape, and mass casualty.
The BKAT tests take approximately 45 minutes to complete and has been administered on a supervised and unsupervised basis with no statistical difference in scores. The total possible score for the ED-BKAT is 100 points. Items on the test contain multiple choice and fill-in-the-blank questions that measure both the recall of basic information and the application of basic knowledge in practice situations. Psychosocial aspects of critical care nursing practice are integrated into specific questions in the BKAT.
The ED-BKAT is based on the parent test, the adult BKAT. Content for the initial version of the BKAT for adult critical care was identified through a review of the literature and interviews with staff nurses and head nurses working in critical care units, and through the suggestions from two critical care physicians and a nine member panel of experts in critical care nursing practice and education.
Validity for each of the seven versions of the BKAT was established through a panel of experts. Validity has also been supported through replication of research findings related to group differences, learning theory, and variables associated with (and not associated with) scores on the BKAT. Samples of baccalaureate nursing students and new graduate nurses have been used as groups known to be different from practicing critical care nurses. The mean (M) BKAT scores of the students and the new graduates were, as expected, significantly lower than the M scores of the nurses.
Content validity. The ED-BKAT was based on the adult BKAT, Version Six. A 9 member Panel of Experts from Illinois, Maryland, Massachusetts, Texas, and West Virginia, was used to establish content and construct validity for the ED-BKAT. Panel Members included two Educators for Emergency Services, two Clinical Nursing Educators, one Clinical Nurse Specialist for Critical Care, and four clinicians working in the Emergency Department: Two were certified in critical care nursing (CCRNs) and four in emergency critical care nursing (CENs). Four held a masters degree with three in nursing, two were BSNs and one was a PhD candidate. All were RNs.
The Panel was also used to validate that items for the ED-BKAT are basic to safe nursing practice in the Emergency Department. Each item was rated from 1 to 5 points with 1 being the most basic and 5 representing more advanced knowledge. Of items that were retained, only two Panel Members rated an item with a 5. Items were deleted if the majority of Panel Members rated them advanced knowledge or not relevant to the Emergency Department, and/or if they were outdated. A total of 25 items were deleted, 21 new items were written: Two items were used from the PEDS-BKAT5, one item from the BKAT-7, and one from the NICU-BKAT3. Content areas for the new items came from suggestions from the Panel. Rewording of items (stems, answers, and/or distractors) was done on 40 items to improve clarity and update content. These revisions were guided by existing item analyses from the BKAT-5 (110 nurse subjects), BKAT-6 (624 subjects), and BKAT-7 (144 subjects).
Construct validity of the ED-BKAT was supported through the technique of known group differences. The two groups, ‘known’ to be clinically different were new graduate nurses with zero to < 1 year experience working in the ED and RNs with 1 to 39 years experience in ED nursing, also working in the ED at the time the test was taken. Scores of the new graduates (M= 75.1, SD= 8.4) were compared to the experienced RNs (M=83.2, SD=7.5) using a one-tail independent t-test. Results showed the two groups to be significantly different, with the new graduates scoring lower on the ED-BKAT: t(137) = 3.8, p< .0005. Of note, the ED-BKAT measured this difference.
Previous BKATs. Cronbach’s Coefficient Alpha (a) has been used as the measure of internal consistency for all versions of the BKAT. Reliability of the first two versions of the adult BKAT that was computed on two samples of critical care nurses (N=100 & N=92) and on a sample of baccalaureate nursing students (N=38) from the Washington, DC metropolitan area ranged from a = 0.83 to 0.86. Reliability of Version Three of the BKAT that was computed on a national random sample of 84 nurses who were members of the American Association of Critical Care Nurses was a = 0.83. Version Four was tested on a purposive national sample of 106 critical care nurses with an a = 0.88. The reliability of Version Five of the BKAT was a = 0.84, measured on 528 critical care nurses from 24 states throughout the USA and in the District of Columbia, and was a = 0.91 for l54 non-USA critical care nurses from six different countries. Years of experience of critical care nurses who have taken one of the BKATs has ranged from zero to >34 years. The reliability of the BKAT, Version Six was a = 0.80, measured on 101 critical care nurses from seven states in the USA. Scores ranged from 62 to 99 points, with a M=87.1 points and a standard deviation of 6.7 points.
ED BKAT. The reliability of the ED-BKAT, Version One was measured on 125 nurses working in an ED or Trauma Center. Years of experience in ED nursing ranged from 1 to 39 years. Fifteen (16.3%) were certified in ED nursing with the CEN.
Scores ranged from 59 to 98 with a mean score of 83.3 or 83% correct answers, and a SD of 7.5 points. Reliability was measured using Cronbach’s coefficient alpha and ranged from 0.80 to 0.83.
Following data collection, the ED-BKAT was revised to improve clarity and accuracy, and was guided by item analysis and comments from the subjects. The revisions were minor and included changes to four answers, six stems, and 19 distractors.
The BKAT can be used prior to orientation classes in critical care nursing to identify needed content for the classes, and as a pretest and/or a posttest to measure learning in groups of nurses. It can also be used as a dependent variable to test different teaching methods for orientation classes, and as a means to identify content for in-service education programs for currently employed critical care nurses. In addition, it has been successfully used in nursing research and for advance placement of nurses with prior experience in critical care nursing, so that they do not have to attend classes that present content that they already know. The BKAT is only one measure of basic knowledge in critical care nursing and is not to be used in screening, hiring, or firing situations. It is copyrighted Ó and may not be altered, added to, or used in part. Permission must be obtained to use the BKAT. No BKAT may be placed on any computer for any reason.
No one is expected to achieve 100%. Rather, it is expected that following orientation, critical care nurses will achieve an average score of approximately 83-84 points on the ED-BKAT. Whether or not an average score is considered to be a passing grade depends upon which specific questions are missed; for example, being able to recognize the ECG pattern of ventricular fibrillation is critical to know in any unit. Since the BKAT is being used in a wide variety of clinical settings, which specific questions are essential to know for that setting is decided by the nurse administering the BKAT.
Contact Dr. Jean C Toth, RN (toth@cua.edu).
The Panel of Experts for the ED-BKAT included the following critical care nurses:
Ann Czarny, RN, BSN, MSN, Illinois
Robin Ferguson, RN, MS, CCRN, Massachusetts
Shawn Gauvin, RN, BSN, CEN, Michigan
Sandy Jones, RN, Michigan
Joanne Kim, RN, MSN, Maryland
Joanne McCall, RN, MA, CEN, Michigan
Diane Narron, RN, BSN, CEN, Maryland
Mary Beth Reid, RN, CNS, CCRN, CEN, PhD, Texas
Michelle Shekey, RN, BSN, West Virginia
Data collectors for the ED-BKAT include the following:
Linda Alwine, RN, MSN, South Carolina
Major Patricia Bradshaw, RN, MS, CEN, CCRN, CCNS, Texas
Dorothy Brownlie, RN, CCRN, Maryland
Ann Czarny, RN, MSN, APN, CNS, Illinois
Kerstin Duncan, RN, Washington, DC
Edie Fowlkes, RN, BSN, CEN, Washington, DC
Merri Frank, RN, MSN, CNS, CCRN, Ohio
Therese Frank, RN, California
Anne-Marie Guthrie, RN, MSN, CEN, Pennsylvania
Miriam Halimi, RN, MBA, Maryland
Norma Hall, RN, BSN, BC, Indiana
Christine Hoover, RN, BSN, Texas
Penny Huddleston, RN, BSN, CCRN, Texas
Tanya Hurley, RN, BSN, California
Kimmith Jones, RN, APN, Maryland
Ardelle Kleinsasser, RN, BC, MS, South Dakota
Lisa Koser, RN, BSN, CCRN, Ohio
Terry Laidlaw, RN, MS, Maryland
Ann Lunday, RN, JD, Mississippi
Amy Luckowski, RN, MSN, CCRN, Pennsylvania
Deborah Martin, RN, MBA, MSN, CNA, BC, Arizona
Edythe McGoff, RN, MSN, CEN, CAN, FAEN, West Virginia
Mary Beth Reid, RN, CNS, CCRN, CEN, PhD, Texas
Kelly Rumsey, RN, MS, CEN, APRN, BC, Virginia
Juli Shea, RN, MSN, South Carolina
Shelley Whitacre, RN, CEN, Michigan
The initial version of the BKAT (BKAT1) was co-authored by Jean C Toth, RN, CNS, BCC, PhD, The Catholic University of America Washington, DC toth@cua.edu and Kathleen Ritchey, RN, CNS, MSN, formerly of the Veterans Administration Medical Center Washington, DC. All subsequent BKATs and their versions were authored by Dr. Toth.
BKAT-7 for Adult Critical Care (2006)
BKAT-7S for Telemetry/Progressive Care (2007)
PEDS-BKAT5 for Pediatric Intensive Care Unit (PICU)
NICU-BKAT3 for Neonatal ICU (2004)
ED-BKAT for the Emergency Department (2006)
Boyle M, Butcher R, Kenney C. Study to validate the outcome goal, competencies and educational objectives for use in intensive care orientation programs. Aust Crit Care. 1998;11(1): 20-24.
McConnell EA, Fletcher J, Nissen JH. A comparison of Australian & American registered nurses’ use of life-sustaining medical devices in critical care & high-dependency units. Heart Lung. 1993;22(5):421-427.
Oermann MH. Effectiveness of a critical care nursing course: Preparing students for practice in critical care. Heart Lung. 1991;20(3):278-283.
Recker D, O’Brien C. Reliability and validity of critical care orientation tests. J Cont Educ Nurs. 1995;26(1):21-27.
Runton NG, Toth JC. Introducing the Basic Knowledge Assessment Tool for pediatric critical care nursing (PEDS-BKAT). Critical Care Nurse. 1998;18(3):67-72.
Ritmala-Castren, M, Suominen, T, Leino-Kilpi, H, & Toth, JC. Assessing [Finnish] critical care nurses’ basic knowledge. American Journal of Critical Care (submitted for review, 2005).
Santiano N, Daffurn K, Lee A. The Basic Knowledge Assessment Tool: Is it useful? Aust Crit Care. 1994;7(4):18-23.
Smith-Blair N, Neighbors M. Use of the Critical Thinking Disposition Inventory in critical care orientation.J Cont Educ Nurs. 2000;31(6):251-256.
Toth JC. Follow-up Survey 10 years later: Use of the Basic Knowledge Assessment Tools (BKATs) for critical care nursing and effects on staff nurses. Critical Care Nurse. 2006:26(4):49-53.
Toth JC. Development of the Basic Knowledge Assessment Tool (BKAT) for the NICU: The NICU-BKAT3, its uses and effect on staff nurses. Journal of Perinatal Neonatal Nursing. (2007, In Press).
Toth JC. Comparing basic knowledge in critical care nursing between USA and foreign nurses: An international study. American Journal of Critical Care. 2003;12(1):41-46.
Toth JC. Basic Knowledge Assessment Tool for critical care nursing, Version Four (BKAT-4): Validity, reliability, and replication. Critical Care Nurse. 1994;14(3):111-117.
Toth JC, Dennis MM. The Basic Knowledge Assessment Tool (BKAT) for critical care nursing: Its use and effect on orientation programs. Critical Care Nurse. 1993;13(2):111-117.
Toth JC. The Basic Knowledge Assessment Tool (BKAT)—Validity and reliability: A national study of critical care nursing knowledge. W J Nurs Res. 1986;8(2):181-196.
Toth JC. Evaluating the use of the Basic Knowledge Assessment Tool (BKAT) in critical care nursing with baccalaureate nursing students. Image: The Journal of Nursing Scholarship. 1984;16(3):67-71.
Toth JC , Ritchey KA. New from nursing research: The Basic Knowledge Assessment Tool (BKAT) for critical care nursing. Heart Lung. 1984;13(3):271-279.
JT 3/26/06
Last Revised 28-Feb-08 04:04 PM.
