HomeMy WebLinkAbout2223-9 Behavioral Health Practices Improvement (Final 9-6-23)Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Health Services –
Behavioral Health Practices Improvement
To request this information in an alternate format, please call (541) 330-4674 or send email to internal.audit@deschutes.org
Deschutes County,
Oregon
The Office of County Internal Audit
Elizabeth Pape, CIA, CFE – County Internal Auditor, current
David Givans, CPA, CIA – County Internal Auditor, former
Aaron Kay – Performance Auditor
Audit committee:
Daryl Parrish, Chair - Public member
Jodi Burch – Public member
Joe Healy - Public member
Summer Sears – Public member
Stan Turel - Public member
Patti Adair, County Commissioner
Charles Fadeley, Justice of the Peace
Lee Randall, Facilities Director Take survey by
clicking HERE
Recommendations
9
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Table of Contents:
1. INTRODUCTION .......................................................................................................................................... 1
1.1. BACKGROUND ON DEPARTMENT ................................................................................................................. 1
Funding .................................................................................................................................................................. 1
Structure ................................................................................................................................................................ 2
Organization .......................................................................................................................................................... 3
Staffing ................................................................................................................................................................... 3
Staffing Challenges ............................................................................................................................................... 4
Culture .................................................................................................................................................................... 6
1.2. BACKGROUND ON PRODUCTIVITY MONITORING SYSTEMS ..................................................................... 6
2. FINDINGS AND OBSERVATIONS ................................................................................................................ 7
2.1. FINDINGS ......................................................................................................................................................... 8
2.2. OBSERVATIONS ............................................................................................................................................ 20
3. MANAGEMENT RESPONSE ....................................................................................................................... 22
Health Services Department ............................................................................................................................ 22
APPENDIX A: OBJECTIVE, SCOPE, AND METHODOLOGY ............................................................................. 29
I. OBJECTIVES AND SCOPE .................................................................................................................................. 29
II. METHODOLOGY .............................................................................................................................................. 30
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
HIGHLIGHTS
Why this audit was
performed:
Behavioral Health
remains concerned about
improving and ensuring
the consistency of
productivity among
clinicians across the
division.
What was
recommended:
Recommendations include:
• going through the
process to develop
appropriate productivity
measures;
• reviewing the accuracy of
the data contributing to
measures;
• creating a library of
smart tools;
• evaluating onboarding
processes;
• clarifying the policy for
intern supervision; and
• optimizing fleet
utilization at downtown
Bend locations.
Behavioral Health – Practices Improvement
The review focused on developing an understanding of practices used by highly productive
Behavioral Health clinicians to leverage best practices to improve overall productivity.
Additionally, it included reviewing the supporting operational systems, client workflows,
scheduling, and division productivity measures.
What was found:
The Behavioral Health division currently utilizes many effective performance measures.
However, there is room within the division’s productivity measures system to improve the
following areas:
• developing appropriate productivity measures for staff;
• clarifying expectations for the staff;
• accurately reflecting the time clinicians spend on client care;
• equitably distributing cases within teams; and
• enhancing the division's ability to identify community needs for resource
allocation.
EPIC, Behavioral Health's electronic health records system, features a collection of
documentation aids collectively known as smart tools. Broadening the promotion,
utilization, and training on these tools could greatly assist with the documentation
requirements faced by all clinicians in the division.
The division could refine its onboarding processes for new employees and provide clearer
guidelines regarding the supervision of interns.
When clinicians in downtown Bend need to provide services outside their workplace and
division fleet vehicles aren't readily available nearby, their productivity is affected as they
spend time finding and using alternative transportation options.
Deschutes County Internal Audit
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 1
1. Introduction
Audit Authority
The Deschutes County Audit Committee authorized the assessment of the Behavioral Health
division within the Health Services Department in the Internal Audit Program Work Plan for 2022-
2023. Audit objectives, scope, and methodology can be found in Appendix A.
1.1. BACKGROUND ON DEPARTMENT
The Deschutes County Health Services Department includes the Deschutes County Behavioral
Health (Behavioral Health) division, which operates as the Community Mental Health Program for
the County. Behavioral Health plays a vital role in promoting mental health, supporting recovery,
and providing compassionate care to individuals coping with behavioral health challenges,
substance use disorders, as well as intellectual and developmental disabilities. Its programs and
services encompass assessment, therapy, crisis intervention, case management, psychiatric
evaluations, medication management, and more. These services support the community’s most
vulnerable and acute populations who are not able to be served by other outpatient providers.
Collaboration with other local agencies, medical providers, social service organizations, and
community partners is common to create a comprehensive support network.
Funding
Behavioral Health receives the greatest share of its revenue from the state of Oregon through grant
programs and contracts, as well as services provided to Oregon Health Plan members (Capitation
and fee-for-service). Additionally, it actively participates in the federal Certified Community
Behavioral Health Clinic (CCBHC) grant program, facilitating care coordination to assist individuals in
navigating behavioral health care, physical health care, social services, and other systems they are
engaged with. General fund transfers tend to fluctuate but typically remain at approximately 10% of
funding resources, and these transfers are primarily allocated to the highest acuity clients, such as
those in crisis and older adults.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 2
Graph I
Trend in revenues
and expenses for
Behavioral Health
FY19-FY23
Table I
Programmatic
areas and their
associated teams
Structure
Behavioral Health operates through six programmatic areas, subdivided into twenty-four specialized
teams dedicated to specific projects or initiatives. Teams in bold were included in the audit.
Teams were chosen through discussions with the division based on areas of interest or needing
improvement.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 3
Diagram I
Organizational
workforce
structure
Graph II
Trend of FTE1
growth by select
program areas
and division total
FY19-FY23
Organization
Behavioral Health’s programs are led by managers and are organized
into a hierarchy. Clinical staff (clinicians) are overseen by a team
supervisor. Clinical staff consists of qualified mental health
professionals (QMHP),also known as Behavioral Health Specialist II
(BHS-II), who play a pivotal role in providing behavioral health services
(therapy) to mitigate the impact of mental and emotional client
disturbances. The State requires BHS-II to hold either a license or
certification. Additionally, Behavioral Health Specialist I (BHS-I) are
State-certified qualified mental health associates (QMHA) who deliver
services under the guidance of QMHPs. Teams also comprise of
administrative and peer support staff; however, they were not included
in the audit.
Organizational Growth
Behavioral Health has seen a
29% increase in staffing in
recent years, growing from
191 FTE in Fiscal Year 2019 to
269 in Fiscal Year 2023. The
growth primarily stems from
bolstering the Crisis &
Stabilization team to support
the 24/7 Mobile Crisis
Assessment Team and the
Deschutes County Stabilization
Center, which opened in 2020.
1 FTE – Full time equivalent
Program
Manager
Team
Supervisor
QMHP
(BHS-II)
QMHA
(BHS-I)
Crisis &
Stabilization has
grown 285%
since FY19.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 4
Graph III
Trend of BHS-I and
BHS-II staffing and
the service hours
for those staff.
Evolution of Clinical Care
The division has shifted from a traditional office-based outpatient care provider to a more
community-oriented approach, delivering a diverse range of services to meet the evolving needs of
the community. While this shift has undoubtedly brought about valuable improvements in outreach
and engagement, it is worth noting that it may not have directly resulted in a substantial increase in
billable services.
Billable Services
Billable client care refers to medical services, treatments, or procedures that can be billed to the
patient's insurance company or directly to the patient. These services are typically reimbursed by a
third-party payer, such as private health insurance, government programs like Medicare or
Medicaid, or the patient themselves if they are paying out-of-pocket. Billable client care could
include individual therapy sessions, crisis support, skills building, or case management.
The division’s
allocation of
resources to
Crisis has not
correlated to
increased
service hours.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 5
Graph IV
Trend of employee
turnover for the
County compared
to Behavioral
Health FY19-FY23
Non-billable client care is considered as part of the overall client care process, and their costs are
most often covered by the division’s operational budget. Non-billable care could include tasks like
travelling to the client, clinicians co-facilitating a group therapy, or providing services to a potential
client who is not currently enrolled.
The clear distinction between billable and non-billable client care is essential for the division to
accurately track and manage their finances, comply with billing regulations, and provide transparent
billing information to patients and insurance companies. It also helps ensure that patients receive
the necessary care while managing the financial aspects of healthcare provision.
Staffing Challenges
Staff retention and recruitment has been an issue for Behavioral Health in recent years. The division
has experienced a higher rate of turnover within the County workforce, as depicted in Graph IV.
This is not specific to Deschutes County, as shortages in behavioral health workers are a common
concern across the country.2
2 Behavioral Health Workforce Report to the Oregon Health Authority and State Legislature, Feb. 1,2022, OHSU
Behavioral
Health has
seen turnover
rates higher
than the
County
workforce in
four of the
last five years.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 6
Culture
The division's clinicians are focused and dedicated to serving their clients. They operate in a
challenging environment, dealing with the often-unpredictable emotions, thoughts, and behaviors
of individuals. Most clients live in a state of perpetual instability, where constant threats to their
most fundamental needs require clinicians to coordinate with multiple service providers in addition
to individual therapy. Many clients experience co-occurring disorders, as mental health issues
intertwine with substance abuse or physical health problems, further complicating the treatment
process. Despite these challenges, each clinician's impact on a client can be life-changing, making a
positive difference in the lives of individuals, families, and the community.
1.2. BACKGROUND ON PRODUCTIVITY MONITORING SYSTEMS
System
A productivity monitoring system is a tool used by organizations to track and measure the
productivity and efficiency of their employees and the services they provide. The system typically
consists of three phases: data collection, analysis, and response. It is one of many tools used by
organizations to measure overall performance towards its goals and objectives.
Productivity monitoring may include the following:
• Inputs – resources of the organization
Example: tracking employee payroll hours
• Workload – amount of work to be done
Example: measuring the number of client appointments
• Output – services provided
Example: counting the number of client assessments completed
• Adjustments – corrections to workload based on difficulty of the work
Example: factoring in client acuity (severity of illness) or adjusting appointment for
client no-shows/ late cancelled appointments
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 7
• Efficiency – typically workload divided by the inputs
Example: the divisions service hour tool (see below)
• Outcome – did the output achieve the intended result?
Example: measuring the reduction in suicide rates due to crisis interventions
• Productivity – analysis of the combination of efficiency and outcome
Example: overall assessment of cost per client case alongside client satisfaction levels
to gauge productivity
Workload measures and productivity measures are quite different. Workload measures focus on
quantifying the amount of work, tasks, or activities that need to be accomplished within a specific
timeframe. These can be captured as either volume or distribution at the individual, team, or
division level. Productivity measures, on the other hand, focus on assessing the efficiency and
effectiveness of the work being performed. These measures analyze the output or results achieved
in relation to the input or resources used.
Department’s System
Behavioral Health’s productivity measures are derived largely from an electronic health records
system, EPIC, to document client services and record client care time. The division has created a
service hour tool which measures the efficiency of an employee’s time spent on client care,
recorded in EPIC, as a percentage of the employee’s payroll hours, recorded in the County’s time
management system. The service hour tool adjusts for clients failing to show up for appointments
and includes documented non-billable client care time. Most teams within the division use the
service hour tool calculation to measure efficiency, however the ACCESS team counts units of
services provided to reflect each clinician’s output.
2. Findings and Observations
The audit included limited procedures to understand the systems of internal control around
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 8
clinician job performance and the supporting systems. No significant deficiencies were found in this
audit. A significant deficiency is defined as an internal control issue that has the potential to
negatively impact the entity's capacity to achieve its objectives, safeguard assets, and provide
comprehensive and accurate information in line with financial, compliance, or performance
reporting goals. The findings noted were primarily compliance and efficiency matters.
Audit findings result from incidents of non-compliance with stated procedures and/or departures
from prudent operation. The findings are, by nature, subjective. The audit disclosed certain policies,
procedures and practices that could be improved. The audit was neither designed nor intended to
be a detailed study of every relevant system, procedure, or transaction. Accordingly, the
opportunities for improvement presented in the report may not be all-inclusive of areas where
improvement may be needed and does not replace efforts needed to design an effective system of
internal control.
Management has responsibility for the system of internal controls, including monitoring internal
controls on an ongoing basis to ensure that any weaknesses or non-compliance are promptly
identified and corrected. Internal controls provide reasonable but not absolute assurance that an
organization’s goals and objectives will be achieved.
2.1. FINDINGS
Behavioral Health’s system of productivity measures needs additional
development.
Current clinician productivity measures are ineffective.
The division set individual productivity targets for clinicians based on their team and role.
However, targets determined by the service hour tool's calculated percentage seem to be low.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 9
Image I
Excerpts of
performance
measures
reported by the
division3.
For instance, crisis clinicians are expected to spend over 30% of their time on client care.7
Despite the low target threshold, 73% of all clinicians (including both efficiency and output
productivity) were not meeting their target as of February 2023. Conversations from
supervisors and staff indicate confusion about the division’s expectations, and there are no
consequences for employees who do not meet the target.
The division has multiple performance measures, some effective measures are highlighted in
Image I. Useful performance measures, including productivity, are ones which will help the
division identify if they are moving in the right direction. Crafting meaningful performance
measures entails a systematic approach, starting with a clear definition of the division's
overarching objectives, followed by identifying key data sources essential to achieving these
objectives and selecting appropriate analytical formulas that align with the nature of the
division's operations. Subsequently, the measures must be monitored for accuracy and
analyzed for valuable insights. If the analysis reveals negative outcomes, such as poor
productivity, it serves as a strong indicator for the necessity of implementing changes and
improvements.
3 FY24 Health Services Budget Committee Briefing
7 Client care calculation includes billable hours, non-billable hours, and credit for no-show clients.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 10
Graph V
Average
composition of
clinician efficiency4
as measured by
the service hour
tool.
Clinician Quote:
“It’s hard to set a
standard for an
unpredictable
crisis.”
Ineffective productivity measures can have several effects on an organization:
• Resources may be misallocated based on incomplete information.
• Employee motivation can be reduced without a sense of purpose and direction.
• Employee evaluations may not accurately represent individual contributions.
• Promoting output over outcome can occur.
Since 2015, the division has measured clinician productivity. In 2021, they created the current
service hour tool to improve the measurement and set new clinician targets. Even though the
division started with a clear objective while developing the tool, the low target levels suggest
the design was misaligned with the nature of the operation. The service hour tool derived
targets may not be realistic for some clinicians, like case managers and crisis workers. These
clinicians don't have much control over how much time they spend with clients, which can
make it hard for them to reach the targets in their roles. Additionally, the recent trend in
employee turnover has made supervisors reluctant to correct underproducing clinicians to
avoid potential loss of staff. Consequently, there is a need for a comprehensive reassessment
of productivity measurement methodologies to ensure alignment with the changing demands
of clinical care and to empower employees to meet realistic and achievable targets.
It is recommended Behavioral Health go through the process to develop appropriate
productivity measures as well as clarify expectations for staff.
4 Limited to those clinicians included in the audit but excluding the ACCESS team who have output targets not measured by the service hour tool.
The service hour tool
indicates the
average efficiency of
clinicians at 33% of
their worked hours.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 11
Graph V
(repeated)
Average
composition of
clinician efficiency5
as measured by
the service hour
tool.
Graph VI
Composition of
clinician activities
as reported during
the study.
The service hour tool does not reflect the time clinicians spend on client care.
A one week work study8 provided insight into clinician workload and utilization of time. The
study revealed comprehensive client care requires more time than the current service hour
tool measures. The EPIC data, on which the service hour tool is based, lacks a record of the
time spent on client care documentation. Shown in Graph VI, client care accounted for 76% of
their workload compared to the 28% average calculated by the service hour tool.
Accurate productivity measures are vital for effective resource allocation, capacity planning,
and workload optimization. They help managers understand distribution of work, enable
realistic goal setting, facilitate efficient time management, and contribute to high-quality
outcomes.
5 Limited to those clinicians included in the audit but excluding the ACCESS team who have output targets not measured by the service hour tool.
8 See Appendix A for methodology and audit procedures.
Client care in work study – 76%
Client care in service hour tool – 28%
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 12
During interviews, 40% reported not documenting all non-billable client care and 20% of
clinicians mentioned working on billable documentation outside of work hours. This suggests
there are internal pressures to complete billable documentation without reporting the payroll
hours, leading to potential falsification of measures derived from the service hour tool.
Additionally, this situation increases the risk of creating a work-life imbalance for the
employee and raises potential labor law issues for the employer.
The current measure does not provide managers with a complete picture of the efforts of
clinicians. Several key data points are missing or lack sufficient controls ensuring accuracy.
Currently, the division is not tracking time spent on documenting client care. The service hour
tool measure is dependent on data from EPIC to measure efficiency, which is susceptible to
several problems:
• Clinicians have discretion in documenting non-billable services.
• Delays in documentation hinder the tool's ability to provide timely data.
• The tool relies on a third-party system to access payroll hours, requiring manual
adjustments by the division, which impacts its accuracy.
It is recommended Behavioral Health review the accuracy of the data contributing to
clinician workload measures.
The caseload distribution system is not functioning effectively.
Conversations with clinical staff on some teams suggested a perception of unequal
distribution of cases within the team. There were notable disparities in BHS-II caseloads
among four of the eight teams, with some clinicians carrying double the number of cases
than others. Clinicians assigned to the Crisis and Access teams do not carry cases as their
primary function but may see regular clients as part of their assigned duties.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 13
Diagram II
Example of two
teams with
contrasting BHS-II
caseload
distributions.6
Source: Division
reporting as of June
2023.
The division is required by law and contract to enroll all eligible individuals regardless of
capacity. The divisions Intake and First Treatment Appointment Procedure outlines how new
cases are distributed through open first treatment appointment (FTA) slots in BHS-II
schedules. The team supervisor has the authority to modify the distribution to accommodate
client complexity9 or clinician feedback on caseload levels. Balancing caseloads involves
equitably distributing work to prevent individuals from becoming overwhelmed or
underutilized, considering factors such as client complexity, staff experience, and resource
allocation.
When the division consistently assigns a disproportionately high volume of work to certain
employees compared to others, those employees face a higher risk of burnout and fatigue.
This imbalance can result in resentment, decreased cooperation, and reduced teamwork,
ultimately impacting productivity across the team.
Balancing caseloads among staff to accommodate the continuous influx of new clients into
their team poses a challenge for supervisors, particularly for those teams serving broader
6 Newer employees, who usually build caseload gradually, were excluded from the results as were part-time clinicians.
9 While individual client complexity does impact clinician workload, no significant correlation was found between individual clinician caseload and their average client
acuity within the same team.
The Adult
Outpatient team
consists of 10
clinicians with
some clinicians
carrying twice the
caseload of other
teammates.
CSS has team
of 5 clinicians
with very
similar case
distribution.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 14
populations. Teams such as CSS provide treatment to clients with serious persistent mental
illness. These clients frequently receive ongoing County services, which makes it easier to
manage clinician caseloads since clients are less likely to be discharged. However, regardless
of the team’s purpose, clients who are not participating in services should be discharged
according to division policy. Nevertheless, BHS-IIs might choose to keep them active,
anticipating potential crises and avoiding the resulting reenrollment back into services.
Furthermore, apart from active supervisor monitoring, there are no controls within EPIC’s
scheduling feature to prevent BHS-IIs from using existing cases to fill FTA slots.
The 2012 internal audit report, Client access to services, partially raised this concern and noted
the new electronic record system could provide data to aid in the development of a caseload
distribution system. The division has several productivity measures in place for caseload
distribution, but it appears the division is not effectively using the measures to adjust
practices.
It is recommended Behavioral Health strengthen the controls for first treatment
appointment assignment and client discharge.
It is recommended Behavioral Health use the caseload measures to promote a more
equitable distribution of work within teams.
Specialization limits the efficiency of the division.
The division has created specialized teams to support specific client populations, expanding
services but restricting the division’s flexibility to adapt as the community’s need for service
change. The preference of individual therapists to work with specific demographics or
specialize in certain treatment techniques also creates boundaries in placement options for
potential clients seeking care.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 15
Clinician Quote:
“Matching clinical
techniques to
client needs leads
to more efficient
outcomes.”
The purpose of a Community Mental Health Program is to provide a system of appropriate,
accessible, coordinated, effective, efficient, trauma-informed behavioral health and substance
use treatment services to meet the needs of community members10. Equitable access to care
is a key objective in the division’s strategic plan, ensuring vulnerable populations have access
to necessary care regardless of their socio-economic background.
Specialization can negatively impact efficiency as it results in bottlenecks, narrowing of focus,
and a lack of versatility. The bottlenecks are most evident for adult clients who are currently
scheduling FTAs with a therapist for up to two months in advance at SOCO11. The division is
actively looking into delays with FTAs, and initial findings indicate that clients are frequently
being rescheduled to earlier appointments.
Specialized care is a common practice within the healthcare field. It’s important to note, while
specialization can have limitations, it also plays a crucial role in delivering advanced and
specialized care to patients. Care provided by a community program is generally not able to
be as specialized as private providers. The key lies in finding the right balance between
specialization and the need for comprehensive, coordinated care to ensure optimal
productivity and patient outcomes.
The 2012 internal audit report, Client access to services, provided several strategies to counter
the effects of specialization and bottlenecks. One strategy suggested, “Staffing can be more
flexible to be able to target areas that are taking too long to obtain service.”
It is recommended Behavioral Health re-evaluate the division’s flexibility to actively respond
to the broader needs of the community.
10 Health Services Budget Reference Manual FY2024
11 Behavioral Health First Treatment Appointment Overview Dashboard – Previous 12 months report
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 16
Opportunity exists for Behavioral Health to improve practices.
Technology can be leveraged more effectively.
Writing clinical notes from scratch significantly increases the time and effort required to
complete documentation. Clinicians often must rely on their own initiative to discover
shortcuts for documenting into the EPIC system. Many clinicians have created templates
outside of EPIC, which they must manually copy into the system, adding extra steps.
Productivity relies heavily on speed. Completing tasks quickly allows for more availability to
perform others. It is essential to consider technology as a means to enhance productivity. The
EPIC system offers smart tools that can substantially reduce the clinical writing process. These
smart tools provide clinicians with pre-written documentation formatted for convenience.
Many therapy techniques follow a repeatable structure, making them ideal for an organized
format meeting the legal requirements for clinical documentation.
The division currently utilizes smart tools within EPIC. Some examples are:
• Smart forms are used for client assessments.
• Smart text is used for goal, intervention, response, progress, and plan (GIRPP) notes.
• Smart links are used for client screenings.
• Smart lists are dropdown menus of pre-programmed options, such as a scale of 1 –
10.
• Several clinicians have smart phrases approved by the division for regular usage to
document client encounters.
Compared to technology-enabled processes, manual processes are more prone to errors,
resulting in delays, rework, and reduced productivity. Repetitive manual documentation
consumes valuable time that could be better utilized in other tasks.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 17
Only the system administrator can create smart text, links, and forms in EPIC. The promotion
of smart phrases as a productivity technique relies largely on the clinician’s team and is not
widely endorsed by the division. Engaging employees in the process of designing and
evaluating smart tools can also increase their acceptance and effectiveness.
It is recommended Behavioral Health create a comprehensive library of division-approved
smart tools to improve clinical documentation efficiency and provide training to clinicians
on how to use them.
The onboarding procedures for new therapists could be strengthened.
During interviews, 50% of clinicians who had not interned within the division expressed
concern over receiving extensive EPIC training before understanding how to use the system in
their job. 23% of clinicians also believed they would benefit from more direct supervision and
job shadowing to develop a better understanding of the position’s requirements. Although
direct supervision occurs during onboarding, only 6 out of 125 (5%) new BHS-I and BHS-II
employees have received two-month evaluations since 2019.
The division’s Employee Orientation and Onboarding Procedure mandates new employees to
spend their first two weeks attending Welcome Week, receiving EPIC training, and shadowing
staff. Upon completion of these activities, the employee is activated in EPIC. Additionally, the
procedure recommends conducting two-month employee performance evaluations.
Employee evaluations serve as a tool to enhance employee engagement, satisfaction, and
retention.
Unless they were previously interns with the division, new employees often undergo
substantial training on an unfamiliar medical system. They are left to familiarize themselves
with the system while on the job, resulting in delayed proficiency and hindering productivity.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 18
Graph VII
Distribution of
interns by job
class.
New employees also miss the opportunity for formal supervisor feedback that effectively
communicates expectations.
The current onboarding procedure prioritizes clinicians receiving caseloads over adequate
training time. There is a sense of urgency within the division to swiftly train staff and engage
clients because of the demand for services. The procedure for supervisors to provide two-
month evaluations is only recommended, not required, demonstrating a lack of support to
assure new employees get the feedback they deserve.
It is recommended Behavioral Health evaluate the onboarding content, procedures, and
supervision to better integrate new employees.
The division’s policy on supervision of interns lacks clarity.
In 2023, the responsibility of supervising
interns was distributed among one manager,
four supervisors, one lead worker, one nurse,
and five BHS-IIs. However, during interviews,
BHS-IIs handling these extra duties
mentioned they received no additional
compensation or adjustments to productivity
expectations.
The Behavioral Health internship program has been highly effective in recruiting individuals
for the division, with many current clinical staff having started as interns. While their
contributions to client care remain the primary value, clinician job descriptions offer an
optional duty for intern supervision. The division policy assigns the responsibility for
supervising interns to clinical supervisors or lead workers only.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 19
The discrepancy between policy and job descriptions creates confusion and potential conflicts
regarding the responsibility for overseeing interns. Staff members are given additional duties
but are still expected to produce at the required levels, leading to additional stress.
The division’s policy lacks clear guidance on the methodology for distributing intern
supervision duties or the role of clinical staff in supervising them.
It is recommended Behavioral Health clarify the policy for supervision of interns.
The availability of vehicles is an obstacle for some clinicians.
Clinicians assigned to the downtown Bend locations12, whose primary responsibilities involve
travel, face challenges in scheduling vehicles to fulfill their duties. With only 32 available
vehicles to accommodate the needs of 62 clinicians working at those locations13, the average
daily utilization over the last three months based on a five-day workweek was quite high at
79.38%. However, some clinicians' work schedules may not align with that standard
workweek, so the utilization rate could be higher.
The County's Light Fleet policy, GA-22, lays out a fleet management strategy including
guidelines for acquiring, retaining, and replacing light vehicles. The strategy includes viable
solutions to address short-term needs by utilizing a motor pool, commercial rental cars, and
employee mileage reimbursement.
When vehicles needed to travel to clients are unavailable, it hampers the clinician’s ability to
efficiently provide essential services to clients. Time spent seeking alternative travel
accommodations directly impacts clinician productivity, as well as leading to delayed or
12 These include the Wall Street Services Building, Mike Maier Building, and Downtown Clinic.
13 The audit work was limited to clinicians and did not calculate the number of other staff assigned to the downtown Bend locations.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 20
reduced assistance for those in need.
The division has faced challenges in coping with the growing staff size and ensuring sufficient
fleet resources. Limited fleet parking at the downtown locations poses a constraint,
potentially making it difficult to accommodate additional vehicle purchases to meet demands.
The downtown campus fleet parking includes vehicles from the State of Oregon DHS office
and the nearby County departments. Due to the ongoing limitations, a County downtown
campus parking study is currently underway, which may alter the number of fleet spaces in
the future. Data on the utilization of the other six County departments with fleets located in
downtown Bend was unavailable.
It is recommended Behavioral Health coordinate with the Road and Facilities Departments
to explore solutions for the downtown Bend locations’ fleet needs.
This could include a location-based motor pool incorporating other departments nearby.
2.2. OBSERVATIONS
Methods utilized by highly productive clinicians that could be promoted to
increase productivity for all clinicians.
Top performing clinicians have effectively applied the following strategies to use their time to the
fullest. All clinicians reported using aspects of these strategies, but not consistently and proactively.
The strategies are ranked in order of prevalence:
1. Schedule dedicated time during the day to focus on documentation and establish
clear boundaries around that time.
This strategy recognizes the extensive documentation requirements for clinicians and
highlights its importance. By allocating specific time within the workday for
documentation, the potential for working outside of paid hours is reduced.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 21
2. Anticipate the documentation requirements of annual assessments and schedule
accordingly.
As one clinician stated, "Annuals are awful," emphasizing their significant time
commitment compared to other services, approximately 15 times longer than typical
documentation. By making scheduling adjustments to accommodate this commitment,
the assessments can be less overwhelming.
3. Document encounters as soon as possible after completing them, even if they are
only partially documented.
By minimizing the time between the encounter and documentation, the need to recall
session details or refer to notes is reduced, enhancing both quality and efficiency. The
division's policy of completing documentation within one business day emphasizes the
importance of acting promptly.
4. Stay concise with documentation.
Adding unnecessary information from a clinical perspective by overwriting the details
of the encounter provides little benefit to the client.
5. As previously recommended, utilize smart phrases to reduce the writing burden.
Most clinicians employ smart phrases in some form, with the top performers
incorporating them regularly and effectively to enhance the conciseness of their
documentation.
6. Utilize smart phrases for non-billable events.
Since there are no regulations pertaining to non-billable documentation, frequently
recurring events can contain nearly identical content.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 22
3. MANAGEMENT RESPONSE
Health Services
Department,
Janice Garceau,
Health Services
Director
And
Holly Harris,
Behavioral
Health Director
September 6, 2023
To: Aaron Kay, Performance Auditor; Elizabeth Pape, Deschutes County Auditor
From: Holly Harris, Behavioral Health Director; Janice Garceau, Health Services Director
Re: MANAGEMENT’S RESPONSE to 2023 Behavioral Health Practices Improvement Audit
Enclosed you will find our response to the internal audit conducted by the County Internal Auditor’s
team during summer 2023. Overall, we are in full agreement with and support of the findings of this
report. The Auditor assigned to this project conducted a process that felt thoughtful, thorough, data-
based and collaborative. His treatment of staff throughout the process was respectful and we believe
contributed to sound results that go deep into many of the phenomenon we have observed and
struggled to address over the years. The findings validate clinician experience, illuminate some of the
core challenges of documenting clinician productivity and provide a framework and recommendations
to begin problem-solving.
A couple of realities have bearing on DCHS’ ability to fully address all of the recommendations. Services
are captured through processes driven by Center for Medicaid Services (CMS) and Coordinated Care
Organization (CCO) practices related to antiquated and burdensome Medicaid billing and
documentation requirements. A great deal of work that clinicians must perform is not billable and as a
result not easily captured. The audit time study demonstrated how that manifests, finding that
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 23
Health Services
Department
continued
clinicians spend 75% of their time doing work on behalf of clients, but can capture far less than that in
the productivity tool. This has morale and fiscal consequences.
In addition, the complex structure of CMHP funding mechanisms in Oregon – CMS to CCOs to CMHPs
with a dash of direct OHA contracts, Federal Grants and commercial insurance – results in a morass of
fidelity requirements, conflicting compliance guidance and practice hoops to jump through that often
slow and impede the work. Primary to this report is the phenomenon of fidelity, grant and contract
requirements that set caseloads/workloads for some teams against a backdrop in which the CMHP
must serve all eligible members within 7 days.
We share these realities in order to set realistic expectations around our ability to directly impact some
of the system drivers of impacts on clinician productivity. In fact, the time study’s revelation of how
much time our staff manages to find ways to serve within this context was ultimately reassuring. They
are our heroes and we hope this audit’s ultimate result will be to help us better align our processes to
meaningfully incentivize what they do every day on behalf of the community.
Recommendation: It is recommended Behavioral Health go through the process to develop
appropriate productivity measures as well as clarify expectations for staff.
a) Management position concerning recommendation: Concur
b) Comments: It was clear to us before the audit that time spent on behalf of clients was not
being captured in EPIC. The time study confirmed this for us with data. It is also clear that we
need to rethink how clinician’s work is measured and evaluated while continuing to find a way to
hold staff accountable to agency expectations regarding caseload management and
documentation standards. Our plan to move forward on this recommendation is to formulate a
workgroup comprised of representation from staff, leadership, and administrative services to
create a new process and target based on clinical outcomes. We plan to immediately stop
evaluating staff on the current productivity targets.
c) Estimated date completed January 2025.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 24
Health Services
Department
continued
d) Our current targets are intended to help the organization remain fiscally viable by leveraging
reimbursement and especially enhanced reimbursement such as PPS payment through CCBHC.
Moving away from a traditional productivity target poses the risk of financial losses. However,
until we land on a tool that adequately and accurately reflects the work of staff, that risk must be
balanced against integrity of the data and staff morale.
Recommendation: It is recommended Behavioral Health review the accuracy of the data
contributing to clinician workload measures.
a) Management position concerning recommendation: Concur
b) Comments: Similar to the comments above, it is clear that time spent providing care for
clients is not accurately reflected in EPIC. In addition, time spent documenting services that is
done outside of work hours is misrepresenting the outputs provided by the service hour tool
since that is based on payroll. The DHCS Data Analytics Team will conduct a review of this and
provide an analysis of the impact. The administrative burden associated with documentation is
likely contributing to this phenomenon. We are currently working internally to find efficiencies in
the system and will be working to implement smart tools within EPIC to assist clinicians in
documenting more easily.
In addition, DCHS is actively engaged at the State level in a collaborative project with the OHA to
Tackle Administrative Burden (TAB). The TAB workgroup developed recommendations provided
to Legislators and OHA in January 2023 and is now working actively with OHA staff to carry out
focused efforts to reduce administrative micro-regulation that gets in the way of care.
c) Estimated date completed January 2025.
d) Estimated cost to implement recommendations are unknown but could include
enhancements to EPIC to allow more flexibility to make need changes as well as Data Analytics
time to complete the analysis of the current system.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 25
Health Services
Department
continued
Recommendation: It is recommended Behavioral Health strengthen the controls for first
treatment appointment assignment and client discharge.
a) Management position concerning recommendation: Concur
b) Comments: The audit clearly revealed that FTA’s are not consistently utilized across the
Department which is also contributing to imbalanced caseloads. Leadership will evaluate the
current process and work with Supervisors to ensure FTA’s are being utilized for their intended
purpose. We will also be analyzing what is available to us in EPIC to limit access to those slots.
c) Estimated date of completed July 2024.
d) Estimated cost to implement recommendation is unknown at this time but a there could be a
cost associated with any adjustments to EPIC.
Recommendation: It is recommended Behavioral Health use the caseload measures to promote
a more equitable distribution of work within teams.
a) Management position concerning recommendation: Concur
b) Comments: The data revealed in the audit related to this recommendation validated what
staff reported in their interviews. Inconsistencies in caseloads amongst teams can lead to
burnout and team morale issues. Correcting the FTA issues will likely impact this
recommendation, but we will also work with supervisors to develop a strategy for ensuring a
balanced distribution of cases and require that team member caseloads be balanced. Managers
and Supervisors will work to create that strategy and process for continued monitoring.
c) Estimated date of completed July 2024.
d) No estimated cost is anticipated.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 26
Health Services
Department
continued
Recommendation: It is recommended Behavioral Health re-evaluate the division’s flexibility to
actively respond to the broader needs of the community.
a) Management position concerning recommendation: Concur
b) Comments: We do not agree that we can reduce specialization to the degree recommended in
the report, as DCHS is mandated by rules and contract to provide specialty behavioral health
care to certain sub-populations of clients. For example, those with Serious and Persistent Mental
Illness (SPMI) and forensic involvement, or children with serious emotional disturbance.
However,, we agree that we can and should improve our ability to enhance the competencies of
clinicians who have a desire to do so or who work in a setting that requires them to serve a
broader span of individuals, i.e. our outpatient hub locations and Crisis teams. We also agree
that by doing this, we could more efficiently utilize the resources available to us when a need
arises that was unexpected. DCHS uses a training through consultation model that could easily
be utilized to focus on specialty populations (i.e. young children, older adults, psychosis, etc.) to
expand the skills of clinicians not comfortable working with some individuals. In addition, we can
utilize the specialists we do have to provide consultation to others who may be taking cases that
are outside of their comfort zone.
c) Estimated date completed July 2024 and ongoing.
d) Estimated cost to implement recommendation is minimal but could include loss of revenue
while staff attend training.
Recommendation: It is recommended Behavioral Health create a comprehensive library of
division-approved smart tools to improve clinical documentation efficiency and provide training
to clinicians on how to use them.
a) Management position concerning recommendation: Concur
b) Comments: This recommendation is extremely helpful and we completely agree. We will
immediately begin work on creating a library of smart tools based on what clinicians are
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 27
Health Services
Department
continued
currently using that is working well. We are also exploring many other smart tools within EPIC
that could help increase the efficiency of documentation.
c) Estimated date completed January 2025.
d) Estimated cost to implement recommendation is unknown but there could be a cost to make
enhancement to EPIC to make this work possible.
Recommendation: It is recommended Behavioral Health evaluate the onboarding content,
procedures, and supervision to better integrate new employees.
a) Management position concerning recommendation: Concur
b) Comments: We acknowledge that the amount of information to onboard a new employee is
extensive and that we could improve our process to check in earlier with new staff on their
progress implementing and understanding all that they have learned in the first two months. We
have developed a revised two month check in to improve onboarding of new employees early in
their process. The two-month Job Duties & Expectations document and process is now complete
and embedded in New Supervisor Training scheduled for all leadership in October and
December.
c) Estimated date completed January 2024.
d) No Estimated cost to implement recommendation.
Recommendation: It is recommended Behavioral Health clarify the policy for supervision of
interns.
a) Management position concerning recommendation: Concur
b) Comments: This work is already underway. Stipends for staff who supervise interns will be
available in the next few months. In addition, the current policy regarding supervision will be
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 28
Health Services
Department
continued
amended to include a section regarding supervision of interns.
c) Estimated date completed January 2024.
d) Estimated cost to implement recommendation is significant, but is currently being paid for out
of workforce incentive funds through OHA. It is estimated to be approximately $50,000-70,000
per year, a fraction of the total behavioral health budget. We include these efforts as part of a
multi-pronged strategy to improve recruitment and retention of staff.
Recommendation: It is recommended Behavioral Health coordinate with the Road and Facilities
Departments to explore solutions for the downtown Bend locations’ fleet needs.
a) Management position concerning recommendation: Concur
b) Comments: We have known of this issue for some time and have elevated the parking issues
downtown for years. We appreciate the creative problem solving of this recommendation and
fully support its implementation. Health Services will immediately begin coordinating with Road
and Facilities on this matter.
c) Estimated date completed July 1, 2024 but is really outside of Health Services control and
more likely up to Road and Facilities to move this forward.
d) Estimated cost to implement recommendation is likely significant but unknown. I would defer
to Road and Facilities for this information.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 29
Appendix A: Objective, Scope, and Methodology
The County Internal Auditor was created by the Deschutes County Code as an independent office
conducting performance audits to provide information and recommendations for improvement.
“Audit
objectives” define
the goals of the
audit.
i. OBJECTIVES and SCOPE
Objectives included:
1. Identify causes and potential reasons for performance below established divisional levels with
some clinicians.
a. Develop criteria: Assess for clinicians (with good documentation) the skills and approaches
supporting reasonable service hours. Productive clinicians should be meeting the > 50%
service hour target as currently measured.
b. Assess the how and why clinicians are falling short of the service hour target. To a lesser
extent, why they may or may not have adequate documentation. Understand the role of
supervisors and supporting systems and workflow in supporting these clinicians.
2. Be aware of any issues with compliance with federal and state regulations and requirements, as
may be applicable.
Scope and timing
The overall assessment work commenced in March and ran through August 2023. The assessment
included clinicians classified as Behavioral Health Specialist I, Specialist II, Leads, and Supervisors
providing outpatient services. Audit work was limited in the following areas:
• Division derived measures that were applicable to the audit objective;
• Clinician selection was developed in discussions with the division after analysis of metrics and
work performance;
• Sample size for both the criteria group and the group with lessor service hours; and
• Time studies comprised one week of data collection from all working clinicians and
supervisors.
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 30
ii. METHODOLOGY
“Audit procedures
are created to
address the audit
objectives.”
Audit procedures included:
• Interviews of selected divisional management and staff;
• Developing an understanding client workflow;
• Developing an understanding and analyzing current metric tools and service hours across
Health Services;
• Developing and collecting a tool to capture clinician work tasks and hours for a one-week
period. Analyzing the resulting data including comparison to current metric tools;
• Selecting clinicians to be used for best practices as the top performers. Identifying
supervisors for these clinicians;
• Selecting clinicians having difficulty meeting the expected metrics to understand obstacles.
Identifying supervisors for these clinicians;
• Developing an understanding of how clients are managed with the systems and anticipated
workflow;
• Developing an understanding through interviews on how clinicians perform and document
work;
• Developing an understanding through interviews on how supervisors oversee and manage
work;
• Assembling and developing themes for potential improvement areas; and
• Interviews and observation of employees, assistance from IT staff in Health Services, and
other procedures as deemed necessary.
We conducted this performance audit in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and conclusions based on our
audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.
(2018 Revision of Government Auditing Standards, issued by the Comptroller General of the United States.)
Health Services – Behavioral Health Practices Improvement #22/23-9 September 2023
Page 31
{End of Report}
Please take a survey on this report by clicking on the attached link:
https://www.surveymonkey.com/r/2223-9
If you would like to receive future reports and information from Internal Audit or
know someone else who might like to receive our updates, sign up at http://bit.ly/DCInternalAudit.