• Table of contents

Overview of the Users’ Guide

If you have any questions relating to the Web Decision Support Tool, please email: support@amhocn.org.

Background and purpose

Clinicians in public sector mental health services in all Australian states and territories are collecting information on outcomes for consumers, via a range of standardised measures that make up the National Outcomes and Casemix Collection (NOCC). These data are being collected with a view to monitoring and improving the quality and effectiveness of services, and are aggregated at the service level, then at the over-arching organisation level, then at the state/territory level and then nationally. The Australian Mental Health Outcomes and Classification Network (AMHOCN) has been charged with the task of managing, analysing and reporting, and supporting and encouraging the use of these data, in a manner that is useful to a range of stakeholders.

Key among these stakeholders are the individual clinicians who are collecting the data. Clinicians lead busy professional lives, and have shown good will and commitment in collecting outcome data to date. In order for this to continue, however, there is recognition that the data they are collecting must have direct utility for the clinical decisions they make regarding the care of individual consumers. For this reason, AMHOCN has developed a ‘Decision Support Tool (DST)’, that provides clinicians with information about consumers’ mental health status at a single point in time, and with information on whether their mental health status changes during the course of an episode of care. Clinicians can enter information about an individual consumer, and see how this consumer compares with other similar consumers in the same type of setting across the nation.

The first version of the DST was released in February 2006, the second in September 2006. The current deployment is the first web-based release, and has been released under the revised name of ‘Web Decision Support Tool (WDST)’ to distinguish web versions from earlier stand-alone products. Work is currently well advanced for a second web release, and AMHOCN continues to seek feedback for that and subsequent versions. It is anticipated that the functionality provided by the WDST will evolve and that the other clinical, consumer and carer measures that comprise the overall NOCC suite will be incorporated.

Structure of the document

This Guide has been written primarily from the perspective of an individual clinician wanting to compare the clinical profile on an individual consumer with similar characteristics and treatment settings to those in the national data set. There are, of course, other perspectives such as those of a service manager, or policy planner. The basic principles of operating the DST are similar for all these perspectives.

Operation

The NOCC WDST is located at wdst.amhocn.org. After accessing this link you will see:

WDST as it initially appears
WDST as it initially appears

At the top of the main part of the screen you will see four drop down boxes (Jurisdiction, Age Group, Level of Analysis and Service Setting) representing the top-level criteria by which data are partitioned. You need to make a selection from each of these partitions.

Jurisdiction

For most users, Jurisdiction is fixed to National. For those users where other options are available the appropriate option should be selected.

Age Group

Age Group has three levels:

  • Child and Adolescent
  • Adult
  • Older Person

Selecting any of these will bring up a new drop down menu - Measure - from which it is possible to select from a list of outcome measures for that Age Group. These options are:

  • HoNOSCA, CGAS, FIHS, SDQ-PC, SDQ-PY and SDQ-YR for ‘Child and Adolescent’;

  • HoNOS, LSP-16, Basis-32, K10+L3D, K10+LM and MHI-38 for ‘Adult’; and

  • HoNOS65+, LSP-16, Basis-32, K10+L3D, K10+LM, MHI-38 and RUG-ADL for ‘Older Person’.

After a measure is selected, another new drop down box - View - appears and a selection must be made. Note that Measure and View are also partitions, hence a selection is mandatory.

An example of the WDST just prior to View selection
An example of the WDST just prior to View selection

A selection from the Age Group drop down menu also brings up a heading for Age Range. By default, all ages for the specified group will be included, however it is possible to further reduce those who will be included in the analysis. To do so, click on the “edit selection” link.

WDST about to restrict age range
WDST about to restrict age range

In the above example, which is already restricted by Adult Age Group, HoNOS Measure and Total Score View, clicking the link will bring up a list of 6 age ranges. Initially, all 6 ranges will have a tick in the checkbox, indicating they are selected. However it is possible to change the ranges that will be included in the analysis. Each checkbox can be individually toggled off or on to create a selection such as the one below that will exclude all age ranges except 25-34 and 35-44. The “select all” link will put a tick in all the checkboxes, while the “clear” link will remove the tick from all checkboxes.</p>

WDST having selected a restricted age range
WDST having selected a restricted age range

Level of Analysis

At this point it is possible to collapse the age range selector (by clicking “collapse” on the screen shown above) or to continue directly to the next partition: Level of Analysis. Selecting Collection Occasion brings up new boxes for Occasion and Individual Comparison Status Score. Occasion is a required field in this context, and you must choose from:

  • Admission
  • Review
  • Discharge

Individual Comparison Status Score is an optional field used to highlight a particular score in the distribution. Typically this is to see where an individual’s score falls compared to the rest of the population. Choosing Episode Transition as the level of analysis brings up a Transition box rather than an Occasion one. The options in this case are:

  • Admission/Review
  • Admission/Discharge
  • Review/Review
  • Review/Discharge

The optional Individual Comparison Status Score is also available for a transition. The example below uses collection occasion (admission) as the level of analysis. It also uses the optional individual comparison; the report will now highlight the HoNOS score 16.

WDST with Level of Analysis options selected
WDST with Level of Analysis options selected

Service Setting

The final partition is Service Setting. The options in this case are Inpatient, Residential and Ambulatory. Selecting from Service Setting does not initiate any sub selections. The example in the next section illustrates an Inpatient selection.

Sex and Diagnosis

These final fields are optional, however they can be restricted using exactly the same mechanism as for Age Range described above. The levels of stratification for diagnosis are based on Summary Diagnosis Groupings that were developed originally for the Mental Health Classification and Service Costs (MH-CASC) Project. The example below shows part of a screen showing selections having been made from Sex and Diagnosis, in expanded view:

Part of WDST screen having selected Service Setting and restricting to a subset of Sex and Diagnosis
Part of WDST screen having selected Service Setting and restricting to a subset of Sex and Diagnosis

Query Submission

Click the “Run Query” button at the bottom of the screen to submit your query. There will likely be a short wait while the results are generated. During this time you will see an indicator - Wait indicator - and a message to “Please Wait”. The results of the query will look something like the following:

Main part of a query result screen
Main part of a query result screen

Sharing and saving queries

Queries can easily be shared between users or saved for later analysis. There are two methods of sharing/saving the results of queries. The first method allows you to save a link which will return you to the query results page, the second allows you to download the results of a query so that they can be opened up in a spreadsheet application such as Excel.

On the top left hand corner of the query result screen you will find a link called ‘Bookmarkable link’. Clicking on this link will take you to the same results page, but the URL will now contain all the parameters of your query. You can now save this URL as a bookmark or copy and paste it in order to share or save it for later analysis.

2. Download the results of the query

Beside the ‘Bookmarkable link’ link is a is ‘Download CSV’ link. Clicking on this link will download a CSV file. This file can be opened up in a spreadsheet application such as Excel. It contains all the parameters from your query and the statistical results as well as a table containing the data from the chart.

Examples

Two examples are provided below. The first considers an individual consumer’s mental health status at admission to inpatient care. The second considers the change in mental health status during the course of an episode for the same consumer.

Example 1: Mental health status

Imagine you are a clinician working in an adult inpatient unit, and you are interested in comparing one of your consumers, Kim, at admission with the national aggregate. Kim is a female, aged 41 and has a diagnosis of schizophrenia. Using the specification section of the WDST, you select the following:

Jurisdiction National
Age Group Adult
Measure HoNOS
View Total Score (12 items)
Age Range 35-44
Level of Analysis Collection Occasion Status
Collection Occasion Admission
Service Setting Inpatient
Sex Female
Diagnosis Schizophrenia, Paranoia and Acute Psychotic Disorders

Your selection should look like this:

Specification for the query
Specification for the query

The results:

Results from the query
Results from the query

This tells you that there are 2,066 valid observations for female consumers aged 35-44 with a diagnosis of schizophrenia, paranoia and acute psychotic disorders at admission to adult inpatient settings in the NOCC dataset at this time. The median HoNOS total score (that below which 50% of observations fall and above which 50% of observations fall) is presented under it’s alternate name: 50th Percentile. The value in this example is 13.0. Other percentiles are similarly displayed. The mean HoNOS total score for consumers like this, calculated by dividing the sum of the total scores by the number of observations, is 13.7. The standard deviation around this mean, which is a measure of variance, is 6.6.

Suppose that Kim’s total HoNOS score at admission is 20. Type ‘20’ into the blank white box after Individual Comparison Status Code, and click ‘Enter’. You will see that the Percentile returned is 84.6, and you will also see her score represented on the histogram with her peers. The chart in the result should look like this:

Chart from Individual Comparison Status Score query
Chart from Individual Comparison Status Score query

Below the chart, you can see a description describing Kim’s score on this measure compared to all similar consumers.

As the text that follows the histogram says, 84.6% of assessments for female consumers with the same age, gender and diagnosis as Kim’s across the country score 20 or less on the HoNOS at admission to an inpatient setting; 15.4% of assessments for similar consumers score more than 20. So, Kim’s assessment is in the top 15.4% of all similar consumers in terms of severity. The histogram itself shows this information pictorially, demonstrating with the grey bar the point at which Kim’s score falls relative to the distribution of scores for all consumers with similar characteristics. The grey horizontal bars with numbers to the left and the right give you a guide as to the number of observations that make up each bar on the chart.

Supposing that you were interested in how Kim’s scored on the Behavioural Problems Subscale. First, click the “Return to Query” link. When the old screen appears, clear the value of 20 from the Individual Comparison Status Score box, then select ‘Behavioural Problems Subscale’ from the View drop down box, leaving all other partition and stratification settings intact. You should then see a screen like this:

Specification for Behavioural Problems Sub-scale query
Specification for Behavioural Problems Sub-scale query

An edited version of the result appears below:

Simplified result from Behavioural Problems Sub-scale query
Simplified result from Behavioural Problems Sub-scale query

Again, you can enter Kim’s status score on this Subscale to compare with the national aggregate. Similarly, you can explore the remaining three subscales of the HoNOS as well as each of the individual 12 items.

Example 2: Change in mental health status

You successfully treat Kim during her inpatient stay. At discharge, Kim has a HoNOS total score of 6. Remembering that her admission score was 20, this means that her change score is 14. You can determine how this level of change compares with the norm, by changing the parameters on the WDST to consider change scores, rather than mental health status. To do this, use the “Return to Query” link, then change Level of Analysis to Episode Transition and View to Total Score. You will notice that this results in the Occasion box being replaced by the Transition box. You are interested in how Kim’s transition from admission to discharge compares with that of her peers, so select Admission/Discharge. The screen should now look like something this:

Specification for the query
Specification for the query

The results:

Simplified results from the query
Simplified results from the query

If you look at the change score statistics that are generated for consumers like Kim, you will see that there are 1,765 observations contributing to the analysis, that the median HoNOS total change score is 7.0 (this is indicated as the 50th percentile). This is interpreted as 50% of these consumers experience greater improvement than a change of 7, and 50% experience less improvement. The mean change score is 7.8 and the standard deviation is 7.0.

To examine how Kim’s HoNOS total score compares with this overall picture, click “Return to Query” type 14 (Kim’s change score) into the Change Score box, and click “Run Query”.

Results from the query
Results from the query

You will see that Kim rates in the 84th percentile so, as the text below the histogram says, 84.8% of consumers like Kim shift 14 points or less on the HoNOS between admission and discharge from an inpatient setting, and 15.2% have a change by more than 14 points. In other words, Kim’s improvement is well above average.