💾 Archived View for gemini.spam.works › mirrors › textfiles › programming › epstein.txt captured on 2020-10-31 at 14:45:29.
-=-=-=-=-=-=-
Neural Nets Improve Hospital Treatment Quality and Reduce Expenses
June 9, 1990
by Jeannette Lawrence
A new hospital information and patient prediction system has improved
the quality of care, reduced the death rates and saved millions of
dollars in resources at Anderson Memorial Hospital in South Carolina.
The CRTS/QURI system uses neural networks trained with BrainMaker (from
California Scientific Software) to predict the severity of illness and
use of hospital resources. Developed by Steven Epstein, Director of
Systems Development and Data Research, the CRTS/QURI system's goal is to
provide educational information and feedback to physicians and others to
improve resource efficiency and patient care quality.
The first study showed that the program was directly responsible for
saving half a million dollars in the first 15 months even though the
program only included half of the physicians and three diagnoses. Since
then, the number of diagnoses and physicians included in the program
have increased. The quality of care has improved such that there are
fewer deaths, fewer complications, and a lowered readmission rate.
Expenses have been reduced by fewer unnecessary tests and procedures,
lowered length of stays, and procedural changes.
Over the past five months the hospital's accounting department has
reported a savings in the millions of dollars, although it is difficult
to say exactly how much of this is directly due to CRTS/QURI. According
to Epstein, "The hospital may now be experiencing a "spillover effect"
of new physician knowledge and behavior into areas not initially
targeted by the CRTS/QURI system." The reported success has motivated
several other hospitals to join in the program and has provided the
impetus to begin a quality program with the state of South Carolina.
At the core of this new system lies several back-propagation neural
networks which were designed using the BrainMaker program on a PC.
Individually trained neural networks learn how to classify and predict
the severity of illness for particular diagnoses so that quality and
cost issues can be addressed fairly. After attempts to use regression
analysis to predict severity levels for several diagnoses failed,
Epstein turned to the BrainMaker program for a new approach and taught
his neural networks to classify and predict severity with 95% accuracy.
The neural networks are also used to predict the mode of discharge -
routine through death - for particular diagnoses.
Without these neural networks, the system could not automatically
compare cases with the same diagnoses and like severity levels. Most
commercially available severity programs provide mortality as the only
measure of quality. Physicians typically reject systems which only
compare costs or mortality rates as being too crude or unfair.
In order to provide more meaningful feedback and valid comparisons
within homogeneous groups of patients, the CRTS Acuity Indexing Scale
was developed. Patients are rated for the severity of illness on a
scale from -3 to +3. Training information is based upon the length of
stay in the hospital which has a direct relationship to the severity of
the illness (acuity). When making predictions, a -3 patient is expected
to require the least hospitalization and resources, a 0 patient average,
and a +3 patient the most. Cases which are run through the trained
network can then be indexed in a database by their severity level and
retrieved by such for comparisons.
Neural Network Designs
The neural network was trained to make the severity prediction using
variables of seven major types: diagnosis, complications/comorbity, body
systems involved (e.g., cardiac and respiratory), procedure codes and
their relationships (surgical or nonsurgical), general health indicators
(smoking, obesity, anemia, etc.), patient demographics (race, age, sex,
etc.), and admission category.
Using a neural network to learn the effects that these variables have on
the severity of illness takes the CRTS/QURI a step beyond the other
programs available for indexing severity of illness. With a neural
network, it is possible to discover relationships that were previously
unrecognized. There is no need to define how the variables are related
to the output, the neural network learns it on its own. BrainMaker
provides the ability to view how any one of these variables effects the
output of the network. Using BrainMaker's neuron graphics any input can
be varied over some range and the effect that this has can be plotted.
The neural network learns (just as profession literature states) that
elderly patients with the same diagnosis require more care than younger
patients and that patients with family support tend to require shorter
hospital stays. Unlike traditional programs, there is no need to
translate premises like these into program statements or formulas. The
neural network learns these and other associations by viewing case after
case until it picks up the patterns using the back-propagation
algorithm.
Three years of patient data was chosen for training. There were
approximately 80,000 patients to choose from and 473 primary diagnoses.
The cases were pre-selected with multiple regression techniques to
eliminate outliers so that no "bizarre" cases were used for training the
neural network. For a given diagnosis, about 400 to 1000 cases were
used to train on.
Data was collected from automated medical records, the QURI program
(explained in the next section), and Federally required files for
Medicare patients. These were downloaded and read into the CRTS
database which is in an rBase format on a PC. The selected training
data was output from the rBase file as a dBase file because there is a
direct link to BrainMaker this way. The dBase file was read by
BrainMaker's preprocessor called NetMaker which automatically translates
the data into neural network files for BrainMaker. These files define
the network architecture, the screen display, and the training and
testing data in BrainMaker format. After these automatic conversions
were done, the neural network was trained in BrainMaker.
During the initial neural network design phase, decisions had to be made
as to which variables were important so that unnecessary data collection
could be avoided. In order to help make this decision, Epstein used
BrainMaker's Hinton diagrams to see which inputs affected the trained
network the most. These diagrams present a picture of connection
strengths (weights) at the hidden and output layers. A neuron which
connects to all the neurons in the next layer with the same strength
transmits no useful information.
Two neural networks for each diagnosis were trained - one to predict the
use of resources and the other to predict the type of discharge. For a
single diagnosis network, there are 26 input variables and one output
variable. BrainMaker trained in about 4 hours using a NorthGate 386
running at 33 MHz. Epstein began training with a .1 tolerance which
means every prediction for every case must be 90% accurate. Then he
lowered the tolerance to .085 and eventually stopped with the network
trained at .05 training tolerance (95% accurate). Then a test set of
cases was run through. The BrainMaker program has built-in testing
capability. In addition, Epstein and Jones verified the results of the
network before using the network to categorize patients and make
predictions.
The CRTS/QURI System
Once the cases are indexed according to severity level, the CRTS
(Computerized Resource Tracking System) can be used to provide
educational feedback to physicians concerning quality care issues and
the use of hospital resources. CRTS combines computer software which
produces automated graphs, reports and summaries along with a
comprehensive educational format. An artificial intelligence program,
VP Expert, is used to generate individual physician's reports.
The CRTS program is more affordable than other indexing systems in that
no additional chart abstracting is necessary. The majority of
information is retrieved from the hospital's "mainframe" computer.
Information sources include the medical records abstract, UB82 files
(medicare), and QURI data.
The "QURI" portion of the CRTS/QURI system is an integrated data
collection and reporting program. Epstein developed this stand-alone
menu-driven, user-friendly software program which provides computer
integration for departments which provide additional data for quality
analysis.
QURI stands for Quality assurance, Utilization review, Risk management
and Infection control. Quality Assurance monitors various criteria for
specific procedures and diseases. Utilization Review monitors resource
related items such as admission justification and intensity of services.
Risk Management determines things that put a patient or hospital at risk
(such as a fall out of bed), and Infection Control monitors infections
acquired during the hospital stay.
CRTS is used for evaluation at various levels. At the hospital level,
comparisons are made to data published by the profession to identify
areas of major concern for the hospital. Individual diagnoses that show
problems of quality and/or finance are identified. At the physician
specialty group level, treatment for various severity levels of a
diagnosis are discussed. At the individual physician level, one page of
written summary and five pages of comparative graphs are provided. This
confidential report shows how a physician's patients compared to the
hospital wide average for a particular diagnosis. Comparisons are
provided for areas such as the types of procedures used, ancillary
services used, patient demographics, infection rate, complications,
readmissions, and predicted (versus actual) illness severity level,
charges, and mortality rate. A summary of problem areas and a list of
suggestions are also included in the report.
CRTS is implemented in two primary modules 1) collecting and analyzing
the data with neural networks, and 2) conducting the physicians'
education program. Epstein, who originally designed the system for his
dissertation, is in charge of the data-related portions. Dr. Fred
Jones, Executive Vice President of Medical Affairs at Anderson Memorial,
sets up physician education (feedback) programs, trains them in the use
of system, and verifies trained neural networks results.
Physician Response
Getting the physicians to accept the new system was no easy task.
"Initially some of the physicians must have thought the program was a
Communist Plot. One even stood up in an early meeting to declare the
whole thing 'crap'," says Epstein. He explains that Jones has done a
tremendous job in getting the physicians involved. The physicians are
given a say as to what information is important, what should be in the
reports, and how to use the data. Epstein continues, "The problem we're
having now is that we're OVERWHELMED with requests for more diagnosis
studies and more special reports from the physicians."
What Can Be Learned from CRTS/QURI
The QURI system includes the following components: infection
surveillance, drug use evaluation, blood usage review, surgical case
review, monitoring and evaluation, physician demographics, case
review/peer review, incident/occurrence monitoring, claims management
and utilization review. This information, as well as information from
patient's charts is used to provide reports, graphs and charts to
individual physicians or specialty groups.
Hospital-wide comparisons can be trended over time and be compared to
statewide averages or averages reported in the literature when
available. Comparisons between the 473 diagnoses can be evaluated and
compared to literature or statewide averages when available. Using the
severity groups, computer generated graphs and charts can be presented
to physician specialty groups and/or individual physicians displaying
how they compare to their average hospitalwide peer or to their average
within a specialty group for many resource and quality issues (see
Comparative Information sidebar). Cases with similar severity levels
can be found and treatments compared. Trends are also revealed in these
same graphs depicting how a physician compares throughout severity
levels. A physician or group may practice more efficiently and with
improved quality when treating higher severity level patients. More
indepth studies can be provided, such as how a physician's expired
patient with a certain severity level was treated, what procedures and
services were used, etc., versus other similarly ill patients.
During a report review a question may come up such as why did Risk
Management note that there was a problem? One possible scenario might
be that the patient fell out of bed and a closer look at the report
would show if Quality assurance reported that too much sedation was
given. One report from the CRTS brings it all together.
How It Began at Anderson Memorial
The first diagnosis chosen for the program was DRG89 - pleurisy/
pneumonia. Compared to the national norm, the hospital mortality rate
was higher. In addition, the hospital was losing money from cost
overruns since insurance companies and medicare coverage have limits on
what they will pay. It was learned that pneumonia of unknown etiology
had the highest mortality. The reports showed that patients of family
practitioners had longer lengths of stay.
A typical scenario unfolded. Physicians would start their patients on
antibiotics, wait for lab results (which were inconclusive more than 25%
of the time) then ask for an internal medicine consult on the third day.
Then treatment started all over. These patients stayed an average of 4
days longer and the hospital averaged a $2764 loss. Since this
discovery, Family Practitioners now get an earlier consult. Further
research found that the sputum collection procedure was a problem. The
hospital now uses respiratory therapy to collect specimens. Samples are
quickly tested and thrown out if they are spit instead of sputum from
the lungs. The inadequate sputum rate decreased from 26% to 5%. The
improvements caused a drop of $1091 in treating an average case of
pneumonia during the pilot program even though only half of the
physicians were involved in the program.
Other improvements for DRG89 followed. The length of stay and charges
continued to fall through 1989. Complications and mortality rates also
decreased. The use of ancillary services and ICU decreased. When
another diagnosis was added to the program, cerebovascular disease, the
hospital experienced a decrease in total charge of $1043 per patient,
even though CAT scans increased. Average length of stay also decreased.
Anderson Memorial is now using nine diagnoses in its program.
What's Downstream
In response to the national concern for the quality of health care, the
South Carolina Hospital Association voted to approve a statewide Quality
Indicators Project with the state. The tracking method was developed by
the Systems Development and Data Research department at Anderson
Memorial. Epstein is currently working with the state of South Carolina
to develop a software package which will look at quality issues for
statewide comparisons.
Seen as only the beginning, Epstein hopes to use the results of this
statewide quality project as a basis for designing another neural
network for use in the hospital. This neural network would screen all
patient charts and other data to provide a quality index (based upon the
statewide issues such as mortality, infections, etc.). He proposes that
the results could be used to decide which cases should be reviewed,
instead of randomly picking charts to look at. "There are too many
patients and too much data for someone to look at every single one and
decide which ones have quality issues that need looking into with a peer
review," Epstein says. This new neural network would make educating
physicians a much more efficient process as well as improve the quality
of care even further.
Currently, Anderson Memorial is working with three other hospitals as
they start their own program. A recent presentation of the CRTS/QURI
system at an American College of Physician Executives conference
produced an overwhelming response. The hospital has been beseiged with
requests for more information from other interested hospitals.
In the very near future at a meeting with seven hospitals from a
hospital network, Epstein will propose a hospital-group program. In
this program, there would be one neural network trained for each
hospital, plus one for all the hospitals using one diagnosis initially.
An overall advisory group would be formed. Comparisons could be made
between hospitals, between physicians and the seven-hospital average, as
well as all the currently implemented comparisons. If approved, he
plans to start the program in a few months. After a six-month pilot
program, results will be checked and verified. And this is only the
beginning.