Workplace AI Risk Assessments: Five Questions Employers Should Be Asking Now

For business leaders, that creates a practical challenge: An organization cannot manage workplace AI risk if it does not know where AI is being used, what decisions it influences, or whether appropriate safeguards are in place.

As discussed in our recent alert, California's Workplace AI Laws Raise the Bar for Employer Accountability, employers face growing expectations to understand how AI is being used in the workplace and to maintain appropriate oversight of AI-assisted employment decisions.

The right place to start is a Workplace AI Risk Assessment. This should be more than an inventory of AI products. A meaningful assessment identifies where AI influences employment decisions, examines the data and processes behind those decisions, evaluates the risks, establishes appropriate safeguards, and assigns responsibility for ongoing oversight.

In performing a Workplace AI Risk Assessment, employers should be able to answer five questions:

1. Where Is the AI?

Start by identifying where AI is actually being used. Do not limit the review to standalone generative AI tools. AI may already be embedded in systems used for recruiting, scheduling, performance management, employee monitoring, workforce planning, attendance, security, and other employment functions. Existing platforms may also acquire new AI capabilities through product updates or additional features. Third-party systems belong in the inventory as well. Using an outside vendor does not eliminate the employer's need to understand how the technology operates within its employment processes.

For each system, identify:

• what the system does;
• where and by whom it is used;
• whether it affects applicants, employees, or both;
• who owns the system or vendor relationship; and
• which employment processes the system touches.

2. What Employment Decisions Does It Influence?

Finding the technology is only the beginning. Next, determine what the technology actually does. An AI system may organize information, rank candidates, generate a score, identify patterns, flag behavior, recommend an outcome, or otherwise influence an employment decision. Those functions do not present the same level or type of risk.

For each system, trace the output to the employment process in which it is used. Pay particular attention to tools that influence consequential decisions involving hiring, promotion, compensation, performance, scheduling, discipline, termination, or workforce reductions.

The practical inquiry should focus not only on what a vendor says the product does, but on how the organization actually uses its output. Employers should know where along that spectrum each system falls.

3. What Data Does It Use?

Once you know what the system does, understand what goes into it. Identify the information the system collects, receives, or analyzes; where that information comes from; what the system generates from it; and where those outputs go next.

This is particularly important when technology evaluates applicants or employees. The underlying information may include application materials, attendance information, productivity data, performance information, communications, location information, or other workforce data. A system may also create new information through scores, rankings, predictions, classifications, or recommendations.

Then assess the risks associated with the data and the resulting outputs.

For systems that affect employment decisions, the assessment should address at least:

• whether the data is appropriate for the employment purpose;
• whether inaccurate or incomplete data could produce an adverse result;
• whether the system creates potential discrimination or disparate-impact concerns;
• whether privacy or confidentiality concerns are implicated;
• whether accessibility issues could affect applicants or employees; and
• whether the system generates information or inferences that the organization should not be using in employment decisions.

Features involving sentiment analysis, behavioral insights, employee scoring, or similar inferences deserve particular scrutiny because the label attached to a feature may reveal little about what information the technology actually analyzes or produces.

4. Where Is the Human in the Loop?

Human oversight should mean more than placing a person at the end of an automated process. For each significant AI-assisted employment decision, determine when human review occurs, who performs it, what information the reviewer receives, and whether that person has authority to question or reject the AI-generated result.

Then test whether that review is meaningful. A manager who receives a recommendation without understanding its basis may be poorly positioned to challenge it. The same is true if the organization expects managers to follow system recommendations as a matter of practice or if reviewers do not have access to the information necessary to independently evaluate the result.

Managers using AI-assisted employment tools should understand the role the technology is supposed to play, its limitations, what they are expected to review, and when a decision should be escalated. The objective is independent human judgment.

5. Could We Explain and Defend the Decision?

Finally, consider what would happen if an AI-assisted employment decision were challenged. Could the organization identify what role AI played? Could it determine what information the system considered? Could the decision-maker explain why the recommendation was accepted or rejected? Could the organization identify the human who reviewed the result? And would its records support that explanation?

The time to answer those questions is before a challenge arises.

Employers should establish documentation practices proportionate to the significance of the decision. For higher-risk uses, that may include documenting the AI system involved, the relevant output, the supporting information reviewed by the decision-maker, the human review performed, and the basis for the ultimate decision.

If the organization cannot reconstruct and explain an AI-assisted employment decision, that is a governance problem worth addressing even before considering the applicable legal requirements.

A Risk Assessment Is a Process, Not a One-Time Exercise

Answering these five questions is the beginning, not the end. AI systems and their uses change. Vendors add functionality. Business units find new uses for existing technology. Employees adopt new tools. A system that initially presents limited employment risk may later become part of a consequential hiring, performance, discipline, termination, or workforce-planning decision.

A Workplace AI Risk Assessment therefore should not become a spreadsheet that is completed once and forgotten. Assign responsibility for maintaining the inventory and managing identified risks. Establish a process for reviewing significant new AI tools before deployment. Reassess existing systems when their functionality or use changes. Train the people who use AI in employment processes. And establish a path for escalating concerns when a system produces a result that appears inaccurate, inconsistent, or difficult to explain. Without clear ownership, even a well-designed assessment will quickly become outdated.

The goal is not to eliminate AI from the workplace. It is to ensure that the organization understands where the technology is being used, what it influences, what risks it creates, and who remains accountable for the decisions that follow.

A Workplace AI Risk Assessment provides a practical framework for doing that work: Find the AI, understand what it does, understand the data, put meaningful human judgment around consequential decisions, and make sure the organization can explain the result.

For assistance conducting a Workplace AI Risk Assessment, evaluating employment-related AI systems, reviewing vendor relationships, or developing workplace AI governance procedures, please contact Jennifer K. McCarty, Dean J. Shauger, or any member of Baker Donelson's Artificial Intelligence or Labor & Employment teams.