Artificial Intelligence: Capabilities, Limits, Ethics, and Evaluation

Artificial intelligence is a family of computational methods that detect patterns, make predictions, generate content, or support decisions. The useful question is not whether an organization “uses AI,” but what specific task a system performs, what data it draws on, whose judgment it changes, and how failure would affect people. An effective guide connects technical capability with workflow, evaluation, accountability, and limits. Different tasks require different evidence: drafting a summary is unlike approving a loan or detecting a medical anomaly.

Define the task and baseline

Describe the input, desired output, user, and decision that follows. “Improve customer service with AI” is too broad; “suggest draft replies to routine delivery questions for an agent to review” gives a testable scope. Map the current workflow and identify bottlenecks, error types, and the people affected. Some problems can be solved with simpler rules, better search, or clearer forms. Estimate the cost of errors as well as potential time savings.

Set a baseline before comparing a system. Measure task completion, accuracy, time, rework, accessibility, and user satisfaction with consistent definitions. Identify rare but consequential cases. A tool that works on common requests may still fail on a complaint, an ambiguous instruction, or a person using an unfamiliar dialect. Define the intended population and environment, because performance in a vendor demonstration may not transfer.

Understand the system’s inputs and limits

Ask what information enters the system, where it came from, and whether the user has authority to submit it. Training data, retrieval documents, prompts, and live records each introduce different quality and privacy risks. For predictive systems, examine how labels were defined and whether historical outcomes reflect unfair processes. For generative systems, distinguish fluent language from verified facts. A confident answer may be unsupported or omit an important condition.

Document what the system can and cannot access, retain, or change. An assistant that drafts text under human review has a different risk profile from one that sends messages or updates records automatically. Check whether the model or vendor may change the behavior without a clear approval process. Plan how users will recognize uncertain outputs and verify consequential claims against authoritative sources.

Evaluate performance in the actual workflow

Create a test set representing ordinary tasks, edge cases, and groups whose experience may differ. Keep evaluation data separate from examples used to tune prompts or configure the system. Measure useful outcomes and error types, not a single aggregate score. A false denial and a false approval may have different consequences. Review the distribution of errors and whether a person can detect and correct them before harm occurs.

Include usability and accessibility. If staff spend longer checking unreliable suggestions than writing responses themselves, nominal generation speed is misleading. Test with real workflow constraints, appropriate consent, and protected data handling. Compare against the current process or a simpler alternative. Record the version, settings, data window, criteria, and reviewers so the result can be repeated or challenged.

Assign governance and human oversight

Name the owner of the use case, data, quality, security, and incident response. Define who can approve deployment, who can pause it, and what evidence triggers a review. Human oversight must be a feasible action, not a phrase in a policy. If reviewers handle hundreds of outputs under time pressure, design the interface to surface uncertainty and give them authority to override. Train users on the system’s failure modes and on the meaning of their review.

Assess privacy, security, fairness, transparency, and external obligations in the actual jurisdiction and context. Limit sensitive data, control access, log changes appropriately, and establish retention and deletion practices. Invite affected people and domain specialists into risk identification. NIST’s AI Risk Management Framework describes governance, mapping context, measuring performance, and managing identified risks as continuing activities rather than a one-time certificate.

Pilot and monitor after launch

Begin with a bounded pilot and pre-set success and stop criteria. A customer support team might compare agent-reviewed AI drafts with its existing process on response time, factual correction, customer satisfaction, and repeat contacts. Watch for changed behavior: users may over-trust polished wording, while customers may ask questions the pilot did not include. Record exceptions, overrides, incidents, and workarounds. Monitor drift when data, products, or policies change.

Update the system only through a documented process with regression checks. Provide a fallback when the service is unavailable or unreliable. Review whether benefits reach the intended users and whether harms concentrate in a subgroup. A system that saves average staff time but produces confusing responses for people with accessibility needs may require redesign before expansion.

Make a justified decision

Conclude by comparing the proposed use with the baseline, its costs, and its residual risks. State what evidence supports the claim that it helps and what remains uncertain. If reliable human review or suitable data are unavailable, narrow the task or defer deployment. If the pilot succeeds, expand in stages with named owners and review dates. AI is most useful when its role is clear, its outputs are tested, and people can question, correct, and stop the process it influences.

When presenting an AI proposal, distinguish a prototype from a dependable service. A small demonstration can show feasibility, but real operations add changing inputs, adversarial use, access needs, outages, and people who rely on the result. Budget for ongoing tests, support, and escalation. Explain the boundary between the system’s suggestion and the authorized human decision. The record should make it possible to investigate a complaint and remedy a harmful outcome, including the ability to withdraw an output or correct a downstream record.