Risk Assessment Tools, Revised Models and Implementation Plan
Written by Alana Dass
RNR Defined and In Context:
As of 2018, the RNR (Risk-Need-Responsivity) model of assessing people held for sentencing is still one of the baseline methods of determining which facilities people are sent to once sentencing occurs. From research done in 2013, RNR standards assess a person’s likelihood to recidivate based on their life history of antisocial behavior, antisocial personality pattern, antisocial cognition and antisocial associates,[1] where “antisocial” (antisocial personality disorder, “sociopathy”) is defined within the DSM-V diagnostic criteria (2012) as “significant impairments in personality functioning manifest by A.) Impairments in self functioning [...] B.) Impairments in interpersonal functioning.”[3]
“Impairments in self functioning” is characterized by “self-esteem derived from personal gain, power, or pleasure”[3] and “Goal-setting based on personal gratification; absence of prosocial internal standards associated with failure to conform to lawful or culturally normative ethical behavior.”[3]
“Impairments in interpersonal functioning” is characterized by “Lack of concern for feelings, needs, or suffering of others; lack of remorse after hurting or mistreating another.”[3] and “Incapacity for mutually intimate relationships, as exploitation is a primary means of relating to others, including by deceit and coercion; use of dominance or intimidation to control others.”[3]
The intended purpose of the RNR tool is to “Work with the moderate and higher risk cases”[2] and “Keep low-risk cases out of intensive correctional services”[2] such that people with low-risk scores do not enter an environment where people with higher risk scores can influence (raise) the risk of their lower-scoring counterparts while individuals of all risk levels involved are given access to the level of care that they need. In other words, using RNR as a risk assessment tool necessitates that ATIs and a network of health support exists as diversions. Full, nationwide implementation of RNR is restricted by a lack of available and adequate services to fulfill the needs of varying risk levels as well as stigma and cost-effectiveness surrounding rehabilitative alternatives to incarceration.[11] In Tompkins County, however, RNR can be instituted because facilities like REACH Medical, Cayuga Health and various nonprofits are willing and able to assist formerly-incarcerated people in reentry. In other words, the foundations of community-based care are already in place. With the coming improvements in inter-organizational communication, SAMHSA’s Integrated Health Solutions[12] can be functional and efficient within the county; public knowledge of the community health services will be increased and organizational overlap can be worked around. One suggestion for facilitating greater inter-organizational communication would be to develop a form of social media specific to the county’s organizations for the sake of having all relevant voices in one place, though this would necessitate a role in which someone spends their working hours maintaining their respective organization’s communications through this social medium.
Statistical and Variable Qualities of RNR and Other Risk-Assessment Tools:
The success rate of the RNR tool is approximately 70%[14] for predictions of recidivism. Given the likelihood of erroneous results, other tools and risk-assessment paradigms have been created to address the gaps that RNR either fails to consider or reflects inaccurately due to bias in reporting. As a result, the risk-assessment tool COMPAS[16] was developed by Northpointe, Inc., a private software company. COMPAS is a 137-question risk assessment[15] that intends to assess a person’s risk to community safety, risk of recidivism, risk of not appearing for court appointments and some pointers about stable living conditions both in their past and at the time of assessment. The model of COMPAS used by probation in the state of New York “adheres to the three core principles of the risk-needs-responsivity model.”[18]
COMPAS, however, is debatably flawed since it inconclusively displays racial biases[13] in assigning higher risk and recidivism predictions to black people compared to white people processed through the system, even though recidivism rates for formerly-incarcerated black people decreased more than that of white people.[17] The controversy surrounding COMPAS is that ProPublica, the research company that showed the racial disparities in COMPAS data, combined the medium-and high-risk categories and contrasted that combined data with the low-risk data.[1] At the same time, the presence of the observed bias, even without using a method like ProPublica’s, exists due to the fact that the actuarial calculations are made from local public data.[1] Like COMPAS, all actuarial tools are susceptible to the eventual development of biases that are already pervasive within the justice system; the ability of any actuarial risk assessment tool to ascertain an accurate recidivism rate for any given individual depends on the extent of human knowledge pertaining to behavioral analysis and cognitive psychology. Despite the methods ProPublica used for the aforementioned controversy, their investigation still yielded accurate results about COMPAS’ 80% error rate in predicting violent crimes and 61% success rate in predicting recidivism for nonviolent crimes.[17] ProPublica acknowledged that COMPAS’ errors in calculations can be partly attributed to data about an individual being entered inaccurately; in other words, COMPAS made predictions based on information limited to the knowledge of the people using it (administrators may not have known about prior criminal records in other states or other attributes affecting risk).[17] In the words of Nathan James, a crime policy analyst for the Congressional Research Services,
“there is a limit to how accurately recidivism can be predicted given the current level of knowledge about the causes and correlates of criminal behavior. Second, well-validated instruments may show similar levels of performance because they are tapping “common factors” or shared dimensions of risk, even though the instruments utilize different items or have different approaches.”[1]
For the state of New York, a 2012 study of COMPAS implementation in probational contexts provided valuable insight into the reliability of COMPAS. COMPAS was evaluated based on the predictions it made on its Recidivism Scale component. One of the initial observations is that “With respect to offender age at assessment, the study found that the Recidivism Scale somewhat UNDERestimated the likelihood of rearrest for offenders 16 to 18-years-old and substantially OVER-estimated the likelihood of rearrest for offenders in their mid-forties and onward,”[18] which seems congruent with predictions accounting for individuals’ criminal history. At the same time, the tool accurately predicted recidivism rate for the majority of 16,303 study cases that were assessed, resulting in an AUC value of 0.71 (approximately 71% success rate of prediction).[18] The study cases are not representative of the entire report, however. Recidivism rate AUC values for different categories of crimes varied (0.68 AUC for Penal Law and VTL subgroups as well as underestimations within certain penal law cases and overestimations in a few VTL cases)[18]
Implementation and active use of any actuarial tool within a criminal justice context carries the risk of accuracy stagnation (due to the algorithms’ dependence on public data). It is imperative within current systemic contexts that an actuarial tool’s use is confined to providing suggestions and insights based on trends in public data and is not the sole basis on which judgements are made.
Revised Risk-Assessment Tool -- The ACE Scorecard Metric:
In order to strengthen the effectiveness of actuarial tools for the sake of reducing incorrectly-assigned recidivism likelihoods, the current ways of using these tools should be reconsidered. More precisely, actuarial tools can be used to approach a form of individualized justice if they were removed from criminal justice settings and placed in clinical settings instead. Consider a tool that measures the economic needs of the person awaiting trial as well as the person’s history and severity of trauma in addition to the level of risk that the person poses to community safety: a person with a longstanding need for mental health support would best be served with resources and a place of rehabilitation away from the community until their risk level (risk to self and risk to community) decreases. Based on the sample assessment that is publicly accessible, New York state’s COMPAS-Probation (2012) does not include a series of psychometric questions within the assessment to gauge status of mental health beyond substance use, history of diagnosis and history of psychotropic drug use.[18] While the tool itself is not intended to be a means of mental health diagnostics, it cannot account for the status of an undiagnosed individual’s mental health as it is. It also must be acknowledged that COMPAS-Probation was developed at some point before 2007[18] and so the language used to refer to certain mental illnesses (developmental disabilities and neurological conditions, particularly) is not reflective of recent knowledge or the DSM-V (2013).
To increase the effectiveness of COMPAS as an instrument of suggestion (not just risk), psychometric analysis can be added. Adverse Childhood Experience (ACE) and resilience metrics, in particular, will provide deep insight on the history of an individual’s trauma and may inform possibilities as to the basis of their actions (cognitive behavioral science). To explain, the ACE scorecard metric is a questionnaire that gauges an individual’s level of trauma (both major isolated events and prolonged trauma) sustained in their formative years.
A licensed psychologist or social worker in an independent public organization (in this case, Tompkins County Mental Health) would be responsible for administering risk assessment, which could be a revised version of RNR/GLM/COMPAS with special attention to ACE; details about this will be shown at the end of this section. Administration of the assessment would be subject to a code of ethical/privacy standards much like how HIPAA operates, meaning that no one outside of the administering party and the person being evaluated can see the contents of the assessment. Once the results are processed and a series of scaled numerical scores are obtained, the professional judgement or interpretation of the administrator is relayed to the criminal justice system (not necessarily the numeric scores). Complete implementation, like RNR, would require extensive coordination between the criminal justice system and local nonprofit partners to ensure that all people with "low risk" are diverted and that resources are invested in people with "high risk." To ensure that this process of assessment is free from administrator bias towards the person being assessed, the process of administration will include a standardized, equitable outline.
Issues With the Revised Model and Counters:
To address the nature of these loopholes lending to exploitation, accountability must be generated and enforced on the side of the risk-assessment tool’s administrative institution. To accomplish this, a set of ethical guidelines based on HIPAA laws would act as a system of checks and balances to make sure that institutional misuses of the risk-assessment tool carries consequences for the evaluator. At the same time, instituting an ethical guideline to follow further ensures the protection of privacy for the person being evaluated since access to information is limited between institutions. In the event of a high-risk result, the individual would need to remain under supervision until the court date, but being detained in a jail or prison for holding is potentially detrimental to mental health. To address this, a transitional care facility that is affiliated with the county should be established as an extension of Tompkins County Mental Health. If people could be diverted away from jail or prison and instead held in a facility where there is access to necessary resources, individuals who scored within low-risk could address their needs easily and individuals who scored within high-risk are in a safe location with the same access to resources (intervention can begin before the person begins the process of incarceration). Additionally, depending on the size of the facility and allocation of resources, the larger implication of transitional care facilities within the county extends to include assisting homeless people (reducing crowding of jails).
—————
References
1: James, Nathan. "Risk and Needs Assessment in the Federal Prison System." Congressional Research Service, July 10 (2018). https://justiceroundtable.org/wp-content/uploads/2018/07/Risk-and-Needs-Assessment-in-the-Federal.pdf
2: Bonta, James, and J. Stephen Wormith. "Applying the risk–need–responsivity principles to offender assessment." What Works in Offender Rehabilitation: An Evidence‐Based Approach to Assessment and Treatment (2013): 69-93.
3: American Psychiatric Association., & American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Arlington, VA: American Psychiatric Association. https://www.psi.uba.ar/academica/carrerasdegrado/psicologia/sitios_catedras/practicas_profesionales/820_clinica_tr_personalidad_psicosis/material/dsm.pdf
4: Subramanian, Ram, et al. “Recalibrating Justice: A Review of 2013 State Sentencing and Corrections Trends.” Vera Institute of Justice, July 2014.
5: Henry, Matt. “Risk Assessment: Explained.” The Appeal, Tides Advocacy, 25 Mar. 2019, theappeal.org/risk-assessment-explained/.
6: “Federal Prisons: Information on Inmates with Serious Mental Illness and Strategies to Reduce Recidivism.” U.S. Government Accountability Office (U.S. GAO), U.S. Government Accountability Office (U.S. GAO), 15 Feb. 2018, www.gao.gov/products/GAO-18-182.
7: “Mental Illness.” National Institute of Mental Health, U.S. Department of Health and Human Services, Feb. 2019, www.nimh.nih.gov/health/statistics/mental-illness.shtml#part_154784.
8: Bronson, Jennifer, et al. "Drug use, dependence, and abuse among state prisoners and jail inmates, 2007–2009." Washington, DC: United States Department of Justice, Office of Juvenile Justice and Delinquency Prevention (2017).
9: “More Imprisonment Does Not Reduce State Drug Problems.” The Pew Charitable Trusts, The Pew Charitable Trusts, 8 Mar. 2018, www.pewtrusts.org/en/research-and-analysis/issue-briefs/2018/03/more-imprisonment-does-not-reduce-state-drug-problems.
10: “NIH Fact Sheets - Addiction and the Criminal Justice System.” National Institutes of Health, U.S. Department of Health and Human Services, report.nih.gov/NIHfactsheets/ViewFactsheet.aspx?csid=22.
11: “Risk-Needs-Responsivity (RNR) Simulation Tool.” Center for Advancing Correctional Excellence (ACE!), George Mason University, 2013, www.gmuace.org/research_rnr.html.
12: “What Is Integrated Care.” What Is Integrated Care / SAMHSA-HRSA, US Department of Health and Human Services, www.integration.samhsa.gov/about-us/what-is-integrated-care.
13: Larson, Jeff, et al. “How We Analyzed the COMPAS Recidivism Algorithm.” ProPublica, ProPublica, 23 May 2016, www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.
14: Edward J. Latessa and Brian Lovins, “The Role of Offender Risk Assessment: a Policy Maker Guide,” Victims and Offenders, vol. 5, 2010, p. 212 (hereinafter “The Role of Offender Risk Assessment”).
15: Yong, Ed. “A Popular Algorithm Is No Better at Predicting Crimes Than Random People.” The Atlantic, Atlantic Media Company, 29 Jan. 2018, www.theatlantic.com/technology/archive/2018/01/equivant-compas-algorithm/550646/.
16: Angwin, Julia. “Sample-COMPAS-Risk-Assessment-COMPAS-‘CORE.’” DocumentCloud, DocumentCloud, 2011, www.documentcloud.org/documents/2702103-Sample-Risk-Assessment-COMPAS-CORE
17: Center, Electronic Privacy Information. “EPIC - Algorithms in the Criminal Justice System.” Electronic Privacy Information Center, EPIC, 2019, epic.org/algorithmic-transparency/crim-justice/.
18: Lansing, Sharon. “New York State COMPAS-Probation Risk and Need Assessment Study: Examining the Recidivism Scale’s Effectiveness and Predictive Accuracy .” Division of Criminal Justice Services Office of Justice Research and Performance, Sept. 2012.