Pretrial Risk Assessments

We believe that jurisdictions should work to end secured money bail and decarcerate most accused people pretrial, without the use of “risk assessment” instruments.

The extraordinary measure of pretrial detention should be treated as a last resort and should only be imposed upon an accused person after they’ve received a thorough, adversarial hearing that observes rigorous procedural safeguards respecting individual rights, liberties, and the presumption of innocence.

In light of the concerns raised in this document, we urge jurisdictions to reconsider their use of risk assessment tools. Pretrial “risk assessment” instruments – although they may seem objective or neutral – threaten to further intensify unwarranted discrepancies in the justice system and to provide a misleading and undeserved imprimatur of impartiality for an institution that desperately needs fundamental change.

Where these tools are used, in order to reduce the harm they can cause we urge the following:

Principle 1

Pretrial risk assessment instruments must be designed and implemented in ways that reduce and ultimately eliminate unwarranted racial disparities across the criminal justice system. Those engaged in the design, implementation, or use of risk assessment instruments should also test ways to reduce the racial disparities that result from using historical criminal justice data, which may reflect a pattern of bias or unfairness.

Principle 2

Pretrial risk assessment instruments must be developed with community input, revalidated regularly by independent data scientists with that input in mind, and subjected to regular, meaningful oversight by the community. The particular pretrial risk assessment instrument chosen should be trained by, or at least cross-checked with, local data and should be evaluated for decarceral and anti-racist results on a regular basis by the local community, including people impacted by harm and violence, and people impacted by mass incarceration, and their advocates.

Principle 3

Pretrial risk assessment instruments must never recommend detention; instead, when a tool does not recommend immediate release, it must recommend a pretrial release hearing that observes rigorous procedural safeguards. Such tools must only be used to significantly increase rates of pretrial release and, where possible, to ascertain and meet the needs of accused persons before trial, in combination with individualized assessments of those persons. Risk assessment instruments must automatically cause or affirmatively recommend release on recognizance in most cases, because the U.S. Constitution guarantees a presumption of innocence for persons accused of crimes and a strong presumption of release pre-trial.

Principle 4

Neither pretrial detention nor conditions of supervision should ever be imposed, except through an individualized, adversarial hearing. The hearing must be held promptly to determine whether the accused person presents a substantial and identifiable risk of flight or (in places where such an inquiry is required by law) specific, credible danger to specifically identified individuals in the community. The prosecution must be required to demonstrate these specific circumstances, and the court must find sufficient facts to establish at least clear and convincing evidence of a substantial and identifiable risk of flight or significant danger to the alleged victim (or to others where required by law) before the exceptional step of detention of a presumptively innocent person, or other onerous supervisory conditions can be imposed. All conditions short of detention must be the least restrictive necessary to reasonably achieve the government’s interests of mitigating risks of intentional flight or of a specifically identified, credible danger to others. Any person detained pretrial must have a right to expedited appellate review of the detention decision.

Principle 5

Pretrial risk assessment instruments must communicate the likelihood of success upon release in clear, concrete terms. In accordance with basic concepts of fairness, the presumption of innocence, and due process, pretrial risk assessment instruments must frame their predictions in terms of success upon release, not failure. Further, such tools should only predict events during the length of the trial or case – not after the resolution of the open case.

Principle 6

Pretrial risk assessment instruments must be transparent, independently validated, and open to challenge by an accused person’s counsel. At minimum, the public, the accused person, and the accused person’s counsel must all be given a meaningful opportunity to inspect how a pretrial risk assessment instrument works. The accused person’s counsel must also be given an opportunity to inspect the specific inputs that were used to calculate their client’s particular categorization or risk score, along with an opportunity to challenge any part – including non-neutral value judgments and data that reflects institutional racism and classism – of that calculation.

Signed by:

African American Ministers In Action
Alternate ROOTS
American-Arab Anti-Discrimination Committee
American Civil Liberties Union
Amistad Law Project
Arab American Institute
Asian Pacific American Labor Alliance
Bend the Arc Jewish Action
Black Alliance for Just Immigration
Black Lives Matters Philadelphia
The Bronx Freedom Fund
Brooklyn Community Bail Fund
Brooklyn Defender Services
The Center for Carceral Communities
Center for Democracy & Technology
Center for Justice Research – Texas Southern University
The Center for Media Justice
Center for Popular Democracy
Center on Race, Inequality, and the Law at NYU Law
Chicago Community Bond Fund
Civil Rights Corps
College and Community Fellowship
Color Of Change
Colorado Freedom Fund
Connecticut Bail Fund
Cville Immigrant Bond Fund
Data & Society
Dauphin County Bail Fund
Decarcerate Tompkins County
Defender Association of Philadelphia
Defending Rights & Dissent
Denver Justice Project
Eastern Iowa Community Bond Project
Electronic Frontier Foundation
Ella Baker Center for Human Rights
Entre Hermanos
Essie Justice Group
Families for Justice as Healing
Fight for the Future
Free Press
FreeThe350BailFund
Global Justice Institute
Government Information Watch
The Greenlining Institute
Helping Educate to Advance the Rights of Deaf Communities (HEARD)
Humanizing AI in Law Research Group, MIT
Immigrant Family Defense Fund
Impact Fund
Impact Justice
Insight Center for Community Economic Development
The Institute of the Black World 21st Century
Jewish Council for Public Affairs
Juntos
Justice Strategies
Kent County (Michigan) Immigrant Bond Relief Fund
LatinoJustice PRLDEF
The Leadership Conference Education Fund
The Leadership Conference on Civil and Human Rights
Local Progress
Madison County Bail Fund Inc.
Massachusetts Bail Fund
The Mass Liberation Campaign
Media Alliance
Media Mobilizing Project
Mijente
Minnesota Freedom Fund
Movement Voter Project
MoveOn
NAACP
NAACP Legal Defense and Educational Fund, Inc.
National Action Network
National Association of Social Workers
National Bail Out
National Center for Lesbian Rights
The National Council for Incarcerated and Formerly Incarcerated Women and Girls
National Council of Churches
National Employment Law Project
National Hispanic Media Coalition
National Law Center on Homelessness & Poverty
NETWORK Lobby for Catholic Social Justice
New America – Public Interest Technology
New America’s Open Technology Institute
Northwest Community Bail Fund
Oakland Privacy
One Pennsylvania
Open MIC (Open Media and Information Companies Initiative)
OVEC-Ohio Valley Environmental Coalition
People’s Action | Mass Liberation Project
People’s Paper Co-op
The People’s Press Project
Philadelphia Bail Fund
Philadelphia Community Bail Fund
Philadelphia Red Umbrella Alliance
PolicyLink
Portland Freedom Fund
POWER Interfaith
Prison Policy Initiative
Progressive Leadership Alliance of Nevada
Project SAFE
Public Defender Association
Public Knowledge
Reclaim Philadelphia
Reentry Think Tank
Richmond Community Bail Fund
Robert F. Kennedy Human Rights
Silicon Valley De-Bug
Southern Center for Human Rights
Southerners On New Ground
Southwest Workers Union
Texas Organizing Project
Tucson Second Chance Community Bail Fund
United Church of Christ, OC Inc.
Urbana-Champaign Independent Media Center
VietLead
Voice of the Experienced
Washington Lawyers’ Committee for Civil Rights and Urban Affairs
Washington Square Legal Services Bail Fund
Young Women’s Freedom Center
215 People’s Alliance

Principle 1

If in use, a pretrial risk assessment instrument must be designed and implemented in ways that reduce and ultimately eliminate unwarranted racial disparities across the criminal justice system.

When considering whether and how to adopt pretrial risk assessment instruments, jurisdictions and stakeholders must acknowledge that the criminal justice system in the United States, since its inception, has allocated benefits and burdens on the basis of race. History confirms the point, as do decades of research and data.

While the methods of arrest, prosecution, and punishment have evolved over time, the imbalance in the allocation of burdens has not. Persistent racial disparities plague the system today. Armed with that knowledge, jurisdictions must work to remedy known, unwarranted racial disparities in the administration of criminal law. In this context, the term “racial disparity” refers to unjustifiable differences in the rates of contact by a racial group with any stage of the criminal justice system that are attributable to explicit bias, implicit bias, socioeconomic inequality, or facially race-neutral policies that produce a disparate racial impact. The national movement to reduce race and wealth-based inequities in pretrial detention will be of limited utility if the types of disparities that jurisdictions seek to remedy simply re-emerge at another point in the system.

Against that historical backdrop, it is imperative that pretrial risk assessment instruments, if used at all, be designed to help meet the goal of reducing racial disparities in the criminal justice system. If a tool cannot help achieve that goal, then it is not a tool that the justice system needs. All pretrial risk assessment instruments must be designed, tested, calibrated, and implemented with the goals of exposing, documenting, reducing, and ultimately eliminating unwarranted racial disparities across the criminal justice system.

Pretrial risk assessment instruments must be designed so that no racial group bears the undue burden of errors made by the instrument. Several measures of fairness – such as error rate balance and predictive parity – currently exist. As innovation progresses, other measures will be developed. Jurisdictions should choose the measure, or combination of measures, that will allow them to best avoid burdening one racial group with erroneous predictions or sanctions. Community input and independent data scientist support in a jurisdiction’s choice of a fairness measure is essential.

Those engaged in the design, implementation, or use of risk assessment instruments should also test ways to reduce the racial disparities that result from using historical criminal justice data, which may reflect a pattern of bias or unfairness. One approach is exploring a corrective weights design, which counteracts biased outcomes by accounting for the effects of systemic racism characterized by patterns of over-policing and other racially disparate criminal justice practices that drive systemic inequities.

The methods chosen to address racial disparities should be jurisdiction-specific, as the causes of disparity may vary over time and by location. Ideally, jurisdictions would use the risk assessment process to engage in research and analysis of the root causes of racial disparities in their criminal justice apparatuses. Critically, the means to address those disparities may fall outside the scope of the criminal justice system, and require an investment in education, public health, economic opportunities, or various other social services.

Jurisdictions should collect and maintain quantitative data on race across all stages of the criminal justice system in order to implement this principle. When possible, that quantitative data should be audited by an agency or institution independent from actors within the system to avoid biased statistical reporting. Quantitative data should also be supplemented by qualitative accounts from individuals who have come into contact with the criminal justice system by virtue of an arrest. These data sets should inform the design, implementation, and use of pretrial risk assessment instruments.

Pretrial risk assessment instruments are not a panacea for racial bias or inequality. Nor are they race-neutral, because all predictive tools and algorithms operate within the framework of institutions, structures, and a society infected by bias. Those facts weigh heavily against their use. However, in those instances when jurisdictions commit to employing a risk assessment instrument as part of the pretrial ecosystem, doing so with an eye toward eliminating unwarranted racial disparities advances the cause of justice for all.

Principle 2

If in use, a pretrial risk assessment instrument must never recommend detention; instead, when a tool does not recommend immediate release, it must recommend a pretrial release hearing that observes rigorous procedural safeguards. Such tools must only be used to significantly increase rates of pretrial release and, where possible, to ascertain and meet the needs of accused persons before trial, in combination with individualized assessments of those persons. Pretrial risk assessment instruments, when used to assign community supervision, should only be used to assign the least restrictive and onerous forms of pretrial supervision. Risk assessment instruments must automatically cause or affirmatively recommend release on recognizance in most cases, because the U.S. Constitution guarantees a presumption of innocence for persons accused of crimes and a strong presumption of release pretrial.

Pretrial risk assessment instruments, if used at all, should only identify groups of people to be released immediately. Additionally, such tools may be used in combination with an individualized assessment to determine the specific needs of an accused person pretrial, with the goal of finding non-punitive, non-onerous direct supports and alternatives to detention that would encourage the accused person’s appearance at trial, and would reduce their risk of being arrested again during the length of the trial.

Pretrial risk assessment instruments must be carefully crafted to avoid encroaching upon the limited scope, or prohibition, of preventative detention and over-conditioning of release in each jurisdiction. Given these limits, pretrial risk assessment instruments need not forecast future arrests at all. Such an approach would alleviate the most egregious forms of bias, and acknowledge the fact that bail has historically existed only for the purpose of ensuring appearance at court. Additionally, the design of any tool should always give far greater weight to the avoidance of false positives than to the avoidance of false negatives, as the harms of detaining the wrong person pretrial are far greater than erring on the side of release.

If jurisdictions seek to use such tools, any scores that do not support or recommend release needn’t be shown to individual judges. Rather, as discussed in the preamble, they need only trigger individual hearings. To the extent that tools involve risk scores being given to individual judges for decisionmaking and value-balancing, the results of pretrial risk assessment instruments should never be framed in a way that recommends detention or that labels an individual as “high risk,” as this masks an inherent value-balancing in the guise of purportedly neutral data. An algorithm cannot determine what level of risk warrants detention or release. Such determinations are fundamentally political and not empirically derived, and they must be explicitly identified as such. Moreover, unquestioning reliance on risk assessment instruments – when determining a defendant’s pretrial liberty – directly violates the constitutional requirement of individualized determinations.

Data used in the development of risk assessment instruments must be reviewed for accuracy and reliability. Data collection must include a transparent and periodic examination of release rates, release conditions, technical violations or revocations, and performance outcomes by race to monitor for disparate impact within the system.

Defense counsel must be included in the process of selecting, calibrating, designing, shaping, and testing a pretrial risk assessment instrument and included in the ongoing evaluation of the tool. Tools of this nature should be overseen and used only toward reducing or ending monetary bail without triggering more expansive forms of supervision and surveillance, such as the use of electronic monitoring. Instead, such controls should only be imposed as an actual alternative to incarceration for someone who has otherwise been lawfully determined eligible for possible detention. Groups with a stake in increasing detention, or in causing high money bail amounts to be set, should not be involved in tool selection.

If the use of a particular pretrial risk assessment instrument by itself does not result in an independently audited, measurable decrease in the number of people detained pretrial, the tool should be pulled from use until it is recalibrated to cause demonstrably decarceral results.

Principle 3

Neither pretrial detention nor conditions of supervision should ever be imposed, except through an individualized, adversarial hearing. The hearing must be held promptly to determine whether the accused person presents a substantial and identifiable risk of flight or (in places where such an inquiry is required by law) specific, credible danger to specifically identified individuals in the community.

The prosecution must be required to demonstrate these specific circumstances, and the court must find sufficient facts to establish at least clear and convincing evidence of a substantial and identifiable risk of flight or significant danger to the alleged victim (or to others where required by law) before the exceptional step of detention of a presumptively innocent person, or other supervisory conditions can be imposed.

All conditions short of detention must be the least restrictive necessary to reasonably achieve the government’s interests of mitigating risks of intentional flight or of a specifically identified, credible danger to others.

Any person detained pretrial must have a right to expedited appellate review of the detention decision.

Extensive evidence demonstrates that pretrial detention and intensive supervision can and do derail the lives of accused people, imposing enormous costs on them, their loved ones, and society at large. Detention and intensive supervision are extraordinary and toxic sanctions to impose on any person based on an unproven allegation of criminal conduct. These deprivations cannot be justified by merely resorting to generalities or statistical estimates. Only an individual, adversarial hearing should ever be considered a constitutionally sound basis for detention pretrial.

At timely, individualized hearings, counsel must be provided to represent accused persons. Public defenders or court-appointed counsel shall be present to represent those who cannot afford their own attorney. The court must ensure that defense counsel has the time, training, and resources to learn important information about the client’s circumstances that may not be captured in a pretrial risk assessment instrument and adequate opportunity to present that information to the court. The court shall administer an “ability to pay” inquiry prior to setting any financial conditions of release, and only after concluding financial conditions are least restrictive. When an accused person establishes that they receive means-tested public benefits, currently held or recently held in correctional facilities or residential facilities that serve mental or physical illness, or have an income below 250 percent of the federal poverty guidelines, when adjusting for household size, said individuals shall be determined ineligible to pay. When an accused person suggests that they do not meet the above-mentioned criteria, but otherwise believe they are unable to pay the amount established by the court, said individuals shall receive a substantive ability-to-pay hearing, including inquiry into income, debt, and public benefits, as well as an affirmative opportunity for accused individuals to provide evidence of indigence. Prospective future employment, access to illiquid assets, the ability to borrow money from family members, or the ability to loan money from a for-profit surety company shall not be considered as contributing to an accused individual’s present ability to pay.

In all cases, the hearing shall commence with a presumption of release on recognizance. If the prosecution seeks an order of pretrial detention – which must be done transparently, and not through the imposition of unaffordable money bail – the burden of proof shall be on the prosecution to establish fact-specific clear and convincing evidence of “relevant risks,” defined as a substantial and identifiable risk of intentional flight or, when explicitly permitted by law, a specifically identified, credible danger to a specific person (rather than the amorphous notion of “community safety”).

Detention should only be an option in specific and narrow circumstances; in all other cases, only release on recognizance and, where necessary, non-monetary, non-onerous conditions of release, should be available. In the specific and narrow circumstances where pretrial detention is a possibility, this determination may only be made at the conclusion of an individualized, adversarial hearing following the appropriate procedures, and if the court finds at least clear and convincing evidence of a relevant risk that no condition or combination of conditions of release or support services could mitigate.

Prior to setting any nonfinancial conditions on an arrestee’s pretrial release, the court must determine that those conditions are the least restrictive necessary to reasonably assure that any relevant needs are served or risks present are mitigated. In all cases, the presumption will be of release on no conditions. Any conditions set must be tailored to the individual arrestee: uniform or blanket conditions of release shall not be permitted.

Principle 4

If in use, a pretrial risk assessment instrument must be transparent, independently validated, and open to challenge by an accused person’s counsel. The design and structure of such tools must be transparent and accessible to the public.

The design of pretrial risk assessment instruments must be public, not secret or proprietary. This means adopting local legislation or enforceable regulations that enforce transparency by sharing the data and design with that specific jurisdiction, in addition to reporting requirements throughout the implementation process. The source code and training data (appropriately anonymized) should be made public. And all tools and their documentation must be clear about the source data and code underlying each conclusion in any report; in other words, whether any given conclusion is empirically derived or based on a political, moral or personal choice or assumption set by criminal justice decisionmakers or other government officers.

At minimum, the public, the accused person, and the accused person’s counsel must all be given a meaningful opportunity to inspect how a pretrial risk assessment instrument works, including access to:

A complete description of the design and testing process used to create a tool;
All input factors a pretrial risk assessment instrument uses;
The weights assigned to those factors;
The thresholds and specific data used to determine “low,” “medium,” or “high” risk calculations or risk scores; and
The outcome data used to develop and validate the tool.

The accused person’s counsel must also be given an opportunity to inspect the specific inputs that were used to calculate their client’s particular categorization or risk score, along with an opportunity to challenge any part – including non-neutral value judgments and data that reflects institutional racism and classism – of that calculation.

In addition, the factors considered by a pretrial risk assessment instrument must comport with local and state law regarding the presumption of release.

Principle 5

If in use, a pretrial risk assessment instrument must communicate the likelihood of success upon release in clear, concrete terms.

In accordance with basic concepts of fairness, the presumption of innocence, and due process, pretrial risk assessment instruments must frame their predictions in terms of success upon release, not failure. For example, a tool could say that, in the past, people similar to the accused person have had an 80 percent likelihood of appearing for all of their court dates over 18 months of hearings. Communicating and framing a pretrial risk assessment instrument’s prediction in negative terms (that is, the likelihood the accused would fail to appear upon release) may unnecessarily lead decisionmakers to perceive and treat the accused more harshly. To the extent possible, the tools should also provide the likelihood of success when certain supports or services are provided, consistent with identifying and addressing genuine needs rather than purported risks.

Further, such tools should only predict events during the length of the trial or case – not after the resolution of the open case. Every tool should thus have a temporal cutoff for its prediction period – for example, six months – which local stakeholders may wish to track speedy trial laws.

Principle 6

If in use, a pretrial risk assessment instrument must be developed with community input, revalidated regularly by independent data scientists with that input in mind, and subjected to regular, meaningful oversight by the community.

A pretrial risk assessment instrument must be designed with input from members of the local community, including, but not limited to:

Independent data scientists not paid for by the designer of the implemented tool;
Survivors of harm and violence and people impacted by mass incarceration;
Decisionmakers and actors throughout the criminal justice system, including public defenders, judges, and district attorneys; and
Community groups focused on racial justice.

Revalidation of the tool should include a check for predictive validity – including understanding mean score differences across race/gender/protected characteristics that could result in disparate impact on those communities. Revalidation should include independent auditing for the decarceral and racially equitable results our communities want and need. Validation and revalidation should take place annually at the least, with the ideal being quarterly.

The people described above should be convened into a funded and staffed community advisory board, supported by data scientists to understand how algorithms work, how they are trained, how they weight their factors, and how pretrial decisionmakers use the algorithms in real decisions. This board should be able to host regular hearings educating it and the public on how algorithms are used in pretrial decisionmaking, and potentially in other criminal justice, community well-being, or public sector decisionmaking contexts. This funded board could be convened through acts of city councils, mayors, or state bodies, depending on local laws and jurisdictional practices. This funded board should have the ability, in concert with elected officials, to pause or roll back the use of a pretrial risk assessment instrument if that tool is not meeting, in practice, the decarceral or racially equitable goals set out by the funded board and other stakeholders.

Judges and their officers – including bail administrators and magistrates, pretrial services staff, and anyone who touches the pretrial risk assessment instrument – should receive regular training and retraining on the use of the tool, and should be overseen in their use of the tool for decarceral and racially equitable results.

The particular pretrial risk assessment instrument designed or chosen should be trained by, or at least cross-checked with, local data and should be evaluated for decarceral and racially equitable results on a regular basis by the local community, including the funded board consisting of key groups mentioned above.