Exam Analysis and Scan Only

Instructional Data Processing (IDP) utilizes a high-speed optical page scanner that collects data from machine-readable student answer sheets.   IDP manipulates the electronic response file to provide customers with a wide range of data analysis reports. 

IDP has several types and sizes of answer sheets available, which can be adapted to a variety of data collection tasks. For users with more unique requirements, Scantron Corporation can assist in the design and development of custom answer sheets.

Surveys may also be conducted using these same answer sheets.  Students must bubble in either a name or PUID but it may be fictitious to preserve anonymity. A raw data file and Exam Analysis report makes data interpretation very easy.

Blogs

Android Intelligence

Nursing Exam Pass Rates Appear to Be Rising. Why?

In January 2023, we reported on the reasons why nursing exam pass rates fell in 2020 and 2021. In this report, we follow up on what has happened since.

During the first 2 years of the COVID-19 pandemic, the share of candidates who passed the national licensure exam to work as a registered nurse fell sharply, from 88.2% to 82.5%, for first-time U.S.-educated candidates, with a smaller decline -- from 72.8% to 68.9% -- for all candidates, including internationally educated and repeat test-takers.

In 2022, pass rates continued to drop, averaging 79.9% (8 percentage points lower than in 2019) for first-time U.S.-educated candidates, and 63.4% for all candidates, the lowest point in the last decade.

In 2023, to most experts' surprise, that spiral appears to be turning around.

Things Are Looking Up

The reason for this reversal depends on whom you ask. Test developers have argued that rates improved due to radical transparency and massive outreach to stakeholders, while some online critics have suggested the test simply got easier. Other nurse educators agreed that the increased transparency and outreach impacted rates, but worry those same measures exacerbated a culture of "teaching to the test."

Understanding trends in exam pass rates also requires context.

The National Council of State Boards of Nursing (NCSBN) produces the exam that aspiring nurses take to gain licensure: the National Council Licensure Exam (NCLEX). This exam uses computerized adaptive testing, which means each test-taker gets virtually a different exam, the difficulty of which changes based on the response given.

Every 3 years, the NCSBN assesses the pass rates for the exam and determines whether the current passing standard is appropriate. If the NCSBN's board of directors decides that the level of clinical judgment required of nurses in practice has increased, it can vote to raise the passing standard.

In December 2022, the NCSBN's board voted to keep the current passing standard on the NCLEX through March 31, 2026. Months later, on April 1, the NCSBN launched the Next Generation NCLEX (NGN), a new version of the NCLEX that aims to more effectively measure test-takers' clinical judgment.

Anytime a new exam is introduced, pass rates tend to dip by a few percentage points in the first two to three quarters, explained Philip Dickison, PhD, RN, CEO of the NCSBN. He said he was naturally a bit concerned about a new exam compounding the "drastic drop in the ability curve" seen during the pandemic, "but it was still the right thing to do."

Keith Rischer, PhD, RN, a nurse educator and owner of KeithRN, a nursing education company, recalled that the last major change to the NCLEX occurred when the passing standard was raised by 0.16 percentage points in 2012. Pass rates fell more than 7 percentage points -- from 90.34% for first-time candidates to 83.04% -- in a single year.

While the NCSBN kept the same passing standard in 2022, the stronger emphasis on clinical judgment and the "unique six-question case studies" in the NGN was predicted to increase the difficulty of the exam in some respects.

"There was an anticipation that this was going to be another precipitous decline in NCLEX pass rates," Rischer said.

Post-Pandemic Rebound

However, that wasn't what happened. Instead, preliminary data showed a jump in pass rates, from 79.9% in 2022 to 88.6% in 2023, for all first-time U.S.-educated candidates. Importantly, the 2023 data exclude the fourth quarter of the calendar year, which typically has the lowest pass rate, experts noted.

Still, Dickison said he was "pretty amazed" at the speed of the recovery. Some viewed the change as a "huge increase," but he stressed that rates were starting from a low baseline following a 3-year period when other variables, namely pandemic-related disruptions, impacted pass rates.

"What I think you're seeing ... is that we have rebounded to pre-pandemic ability levels in our measurements," he said.

Dickison credits the rebound to NCSBN's decision to let educators, regulators, and -- controversially -- preparatory groups "under the hood" of the new exam for several years before the NGN actually launched.

"The idea was to be as transparent as possible to all stakeholders," Dickison said. This meant leveraging opportunities at conferences and during webinars, and sharing what to expect of the new exam -- from case studies and measurement models -- in newsletters. Dickison also credited educators for the big role they played in helping prepare students.

Rayna Letourneau, PhD, RN, executive director of the Florida Center for Nursing, said that while some nurse educators have noted the exam has gotten "too easy," she suggested that perhaps the NGN is simply "a more logical way to measure what nursing students are being taught."

Similarly, she attributes the rise in pass rates to the focus on increased resources and preparation of candidates, including the implementation of "student success" coaching programs.

Teaching to the Test?

Rischer seemed to have a different view, pointing out that the NGN offers partial credit for certain "select-all-that-apply" multiple-choice questions. He said that he believes the real reason for improved pass rates is a shift in nursing education, though he added that his hypothesis is, at this point, "conjecture."

"What we have in the nursing literature for over almost 50 years ... is this widening gap between how nursing is taught in the university and college settings and how it's actually practiced at the bedside," he said. "We're not preparing our graduates for real-world practice realities."

Rischer said he grew even more concerned when he learned from online discussions that some professors were using NGN sample items to teach first-semester students.

"That's called teaching to the test," he argued. Instead of teaching students "alternative multiple-choice items, we need to be teaching our students the open-ended thinking of clinical practice."

For example, for a patient who had an appendectomy, Rischer said the appropriate steps are to take vital signs and conduct a head-to-toe exam.

"There is not a 'select-all-that apply' multiple-choice item on your forehead that I could say, 'You know what? This is the correct answer,'" he added, noting that the NCSBN "is part of the problem." Currently, the primary mechanism that state nursing boards use to measure a program's performance is their first-time pass rate on the NCLEX, which only "reinforc[es] this unhealthy culture of teaching to the test."

The end result of that, as studies have shown, is that "currently less than 10% [of new nurses] have what practice partners would say is entry-level clinical judgment competency," Rischer said, citing a 2021 study. "Complications develop when a nurse doesn't notice or recognize what's most important until it's too late. So ... clinical judgment competency is a big deal. And teaching to the test ... isn't going to solve that."

He said one solution is to move away from first-time test-taker metrics and potentially allow students to take the exam twice and then average their score.

Another, more dramatic, change would be to include a skills test or what Rischer calls "demonstrable competencies" for essential skills involved in clinical decision making, which could be evaluated by an "objective observer."

"It will take more time. It will take more money, but this is something that we can and should be doing," he said.

'Not the Only Measurement'

For his part, Dickison noted that the NGN, like any exam, has limitations.

While it "measures your cognitive and your thinking ability, our exam does not measure the affective domain, doesn't measure character," he said. For example, the exam cannot assess a "less-than-acceptable social interaction" a student has with a professor, nor does the exam know which student is always late to class.

"If they don't show up to class on time every day, that might be a [sign] that they're not going to show up to a patient when they need to," Dickison explained. "That's why you have a regulatory system. That's why we have an education [system] ... these all have to work together. If you don't pass the NCLEX, you don't go to work, but it is not the only measurement that tells us we have a good nurse."

Please enable JavaScript to view the