Customer
Satisfaction Management System
PURPOSE
Behavioral healthcare organizations understand
that service quality, as measured by customer satisfaction,
is the key to success in healthcare service delivery. MHCA has developed a
series of statistically valid and reliable satisfaction assessment surveys
to be utilized in improving the quality of behavioral healthcare services.
The instruments, applicable across client populations and treatment
modalities, provide "actionable" information to provider groups
for quality improvement. Use of MHCA’s national database allows analysis
of comparable data for benchmarking and identification of best practices.
Top
DEVELOPMENT
As the MHCA Customer
Satisfaction Management System was developed, several questions were
addressed.
-
Who Are The
Customers?
-
What Are The
Customers’ Expectations?
-
Are We Meeting The
Customers’ Expectations?
-
What Improvements
Need To Be Accomplished First?
The first stage of
development focused on “who are the customers”. Customers are
divided into two categories – external and internal. External
customers are clients and referral sources. Internal customers are
employees. For each category two surveys were developed: MHCA
Customer Satisfaction Survey Form C for clients, MHCA Customer
Satisfaction Survey Form R for referral sources, Staff Satisfaction
Survey for employees, and Interdepartmental Customer Survey for
department evaluation.
The challenge of developing
survey tools for the external customer (clients and referral sources) was
addressed first. Several MHCA member organizations across the nation
conducted client focus groups and client interviews to determine the
expectation of the clients. These organizations conducted in-depth
interviews with referral sources to determine their expectations.
The surveys were completed and the first survey data was entered into the
national database in July 1995. By late 2001 there were over 200,000
surveys in the client database for analysis of comparable data and for
identification of best practices.
The development of survey
tools for internal customers was not addressed until late 1996.
Again several MHCA member organizations provided assistance with staff
focus groups for the development of the instruments. After extensive
beta testing the first staff and interdepartmental surveys were entered
into the national database in 1997. By the end of 2001 there were
nearly 9,000 staff surveys for analysis of comparable data and for
identification of best practices.
All four surveys
instruments used the same methodology for development. The instruments
were based on information obtained directly from customer groups through
focus interviews, thus ensuring the "voice of the customer" in
the design of the instruments. The following flowchart demonstrates the
process for development of the instruments.
Top
Client
Satisfaction Survey
Interviews were conducted
with 103 clients at 10 MHCA member organizations. These interviews
yielded 535 critical incidents that were then imploded to create
satisfaction items and subsequently clustered to create 6 satisfaction
dimensions. Factor analysis grouped these categories into 3
statistically significant domains: 1) personal therapy perceptions;
2) physical environment; and 3) client/staff interaction
perceptions. The following treatment modalities were included:
- Outpatient
- Inpatient
- Partial/Day Treatment
- Residential
- Case Management
- In-Home Services
- Vocational Rehabilitation
The instrument was then
pre-tested by 9 provider organizations across the country, yielding 517
surveys. Independent experts analyzed the psychometric properties of the
instrument reporting an overall scale reliability of 98 percent. Content
validity is evidenced by the survey design methodology. Tests for
validity indicated the satisfaction domains positively influenced
responses on the following items and accounted for variance as
indicated:
- 64.8 %
- Overall Quality Of Care
- 67.4 %
- Degree To Which Treatment Helped
- 64.8 %
- Willingness To Return For Treatment
Because of the limited number of surveys
received for In-Home Services in 2002, that treatment modality was
dropped and Emergency Services was substituted. In 2004 the survey
method of distribution (mail, phone, person-to-person) was dropped and
replaced with ability to distinguish between Mental Health and Drug and
Alcohol Client survey responses.
Top
Referral
Satisfaction Survey
Interviews were conducted
with 25 referral sources across the country, yielding 131 critical
incidents which were subsequently clustered into 7 satisfaction
categories:
- Interaction
- Services
- Access
- Communication
- Outcome
- Procedures
- Financial
Factor analysis yielded 4
significant domains: 1) services perceptions; 2) procedure
perceptions; 3) communication perceptions; and 4) referral
agencies/clinic interaction perceptions. The instrument was
pre-tested by 9 geographically diverse provider organizations, yielding
140 surveys. Referral source populations tested included:
- Social Service Agencies
- Physicians
- School System Personnel
- Private Practitioners
- State Agency Personnel
Again, independent
experts conducted the instrument psychometrics reporting an overall
scale reliability of 0.95. Tests for validity indicated the satisfaction
domains positively influenced responses on the following items and
accounted for variance as indicated:
- 49.3 %
- Degree To Which Treatment Helped
- 31.9
% - Counselors' Relationship To Client
- 59.3 %
- Overall Quality Of Care
Because of the large number of responses
under "other", in 2003 the categories for source's occupation were
updated.
Top
Staff
Satisfaction Survey
Employees were interviewed
at several MHCA member organizations. When the responses were
clustered there were 2 distinct types of responses. As a result, 2
surveys were developed. The Staff Satisfaction Survey focuses
on the employees’ perception of their environment and employment
conditions. The Interdepartmental Customer Survey focuses on
the ability of a specific department to provide needed resources and
services for staff to effectively perform their duties.
Lessons learned in the
development of the first 2 surveys indicated that a much smaller number of
participants in the focus groups would produce the validity and
reliability required for a staff survey. For that reason only 5
organizations collected employee responses for the development of the
staff survey. MHCA member organizations in Washington, Arkansas, Texas,
New Jersey, and Florida participated in the employee focus groups and beta
testing of the instruments. Even with the decreased participation
the instruments still rated high with the Cronbach Alpha overall scale of
reliability in the range of 0.75 – 0.85 for both instruments.
Factor analysis did not yield significant domains for the Staff
Satisfaction and Interdepartmental Surveys.
Staff Satisfaction
Survey
Based
on staff responses from across the country, there are 13 satisfaction
categories:
-
Accomplishment
-
Co-workers
Interaction
-
Supervision
-
Management
– Leadership
-
Communication
-
Paperwork
– Resources
-
Growth
– Personal and Job
-
Pressure
– Stress Aspects of Job
-
Compensations
and Benefits
-
Organization
Policies and Procedures
-
Physical
Environment
-
Recognition
Interdepartmental
Customer Survey
Based
on staff responses across the country, there are 5 satisfaction
categories:
Top
IMPLEMENTATION
The MHCA Customer
Satisfaction Management System allows for several data entry options,
including: a scannable form format and key-in
entry software. A support document, entitled "MHCA
Customer Survey Users' Guide", provides information and
instruction on survey administration, sampling, and survey analysis. An
additional guide, entitled "MHCA Customer Satisfaction
Management System", is designed to assist in developing a
system to integrate customer feedback into the organization's quality
improvement efforts.
Top
ANALYSIS
Analysis
of the survey data provides the organization insight as to how well they
are meeting customers’ expectations.
The surveys are designed to provide information pertinent to the
organization's quality improvement efforts. As such, the information can
be analyzed from many different perspectives. Pre-coding of the surveys
allows for analysis by units within an organization and comparison to
similar organizations (budget size, geographical location,
etc.). The demographic section of the MHCA Client Satisfaction Survey
provides many variations in data cross-tabs, including: age, sex,
education, marital status, employment status, payment
source, voluntary/involuntary treatment, perception of
problem improvement, treatment completion, etc. The demographic
section of the MHCA Referral Satisfaction Survey provides
information on: referral source type, frequency of referrals,
problem resolution, etc.
Top
NATIONAL
DATA CENTER
The
National Data Center provides numerous services for user
organizations, such as: technical assistance, optional scanning
and analysis of data, communication and updates via the MHCA web
site, coordination of a national users group, and continual
product development. The Center establishes national norms and
publishes "best practices" information in MHCA Customer
Satisfaction Management System Quarterly Reports.
Data analysis available through the National Data Center includes:
-
Reports
created with the organization’s survey data.
Each report includes the mean, standard deviation, and
frequency of responses for each question.
The data is shown in data table format as well as in graphic
form. The demographic
information of respondents is also included for evaluation of types of
clients being served. This
report also includes the calculation of the gamma correlation as a
statistical approach in determining the importance of each question to
the customer in order to assist the organization in selection of areas
of highest priority for improvement.
For an additional fee a narrative for the client or referral
report, including executive summary and descriptive analysis of each
dimension, is also available.
-
Comparative
reports are also available.
The comparative reports compare an organization’s data to the
national data of like customer surveys or a comparison to the
organization’s past survey results. This report does not include the gamma correlation. For an
additional fee a narrative client or referral data report, including
executive summary and descriptive analysis of each dimension, is also
available.
Top
QUARTERLY
REPORTS
The
Quarterly Reports of the National Data Center for Customer
Satisfaction provide different views of the customers’ satisfaction
with, and perception of, services provided in the behavioral healthcare
industry.
Each quarter, different populations are selected for analysis.
In addition to norms based on client type and survey method are
comparisons of different populations (e.g.. adolescent vs. adult, male vs.
female, MHCA organizations vs. Non-member organizations, etc.).
Norms are presented for client, referral, and staff survey data
collected from the past 12 month period.
The fourth quarter report of each year includes best practices
based on the overall grand mean of all survey data provided by an
organization. The best practices are divided into several categories,
including: client type, number of services provided, staff
surveys, etc.
The fourth quarter report also includes a comparison of current
year data to past year data and trends are highlighted.
Quarterly
Reports are available for download in PDF format in the Member
Areas of our website:
Top
JCAHO
Oryx
Formerly a participant in the Joint
Commission on Accreditation of Healthcare Organizations' (JCAHO)
Oryx Program, MHCA withdrew when it was no longer required
for behavioral health centers to transmit data to JCAHO to meet their ORYX
requirements. MHCA still provides graphic interpretation for the following
indicators as JCAHO continues to require behavioral health centers to
collect data and provide results when surveyed. The following questions on the MHCA Customer Satisfaction (Form
C) are approved as perception of care indicators for
ORYX:
| Measure ID |
Description |
| 6447 |
Extent to which your
individual needs were addressed-Form C |
| 6449 |
Opportunity to
participate in decisions about your treatment-Form C |
| 6450 |
Degree to which
treatment helped you deal with your problem/complaint-Form C |
| 6451 |
Ability of services to
meet your needs-Form C |
| 11267 |
Appropriate therapies
and interventions offered - Form C |
| 11270 |
Attention to privacy -
Form C |
| 11271 |
Degree of
confidentiality - Form C |
|
|
Top |
