A preliminary maturity instrument for AI leadership
Thirty five items. Seventy points. One number you can put in front of your board.
This assessment awards points for evidence rather than for intention, which means most organizations score lower than they expect on the first attempt. That is the design. A low score is not a finding about your organization. It is a measure of how much of your practice is currently written down, and honesty is the only starting position that leads to credible, measurable oversight.
This is a preliminary instrument. It establishes a position quickly and internally. It does not verify one, and a self assessment is not evidence of anything to an outside reviewer. Treat the result as a starting point for the conversation, not as a conclusion about the organization.
Score each item zero, one, or two. Score it as an auditor would score it, not as you would describe it in a board meeting. The difference between a one and a two is evidence. If you cannot produce the document, the log, or the record within a working day, the item is a one.
Score it alone first, then score it again with your counterparts in the room. Technology, security, data, artificial intelligence leadership, and the operational owners of the affected functions will each score domains you cannot see from your own chair. The gap between the two totals is more informative than either number. In most organizations the functions closest to governance score lower than the functions closest to delivery by ten points or more, and that gap is the actual finding.
| No. | Domain | Points |
|---|---|---|
| 01 | Mandate and accountability | 10 |
| 02 | Data security | 10 |
| 03 | Data governance | 10 |
| 04 | Governance and ethical use | 10 |
| 05 | Inventory | 10 |
| 06 | Deployment controls, business and administrative | 10 |
| 07 | Oversight and audit operating system | 10 |
Who should see the completed assessment. This is a cross functional instrument and it does not belong to one office. The seven domains sit across compliance, technology, security, data, artificial intelligence leadership, and operations, and each of those executives holds part of the answer. Include clinical leadership wherever an administrative tool reaches into clinical workflow, which is more often than the org chart suggests. Circulate the completed assessment to all of them. A score that only one function has seen is a score that only one function will act on.
Include your counsel, internal or external, in that circulation. Several domains cover ground counsel is already accountable for, including vendor terms, data use, patient disclosure, and how tools are classified. Counsel may also have a view on how an internal assessment of controls should be handled and retained, and that view is easier to act on before the assessment is completed than afterward.
One item decides more than the other thirty four. Item 7.5 asks whether anything has ever actually been stopped, paused, or removed from production. A program that has never stopped anything has never been tested, and an untested program is a description rather than a control. Score that item honestly before you score anything else.
Most healthcare organizations scoring themselves honestly for the first time land between twenty and forty. That is not a grade and it is not a finding. It is the ordinary consequence of adopting artificial intelligence faster than any governance function was funded to follow. What separates organizations from here is not the score, it is whether the next ninety days are planned or improvised.
Find your tier. Do these four things. Do not start anywhere else.
| Tier | The next ninety days |
|---|---|
| Ad hoc 0 to 24 |
Name a single accountable executive in writing this month. Build the inventory before you build anything else, including policy, because policy written without an inventory governs an imaginary organization. Stop new deployments until intake exists. Report the honest position to the board rather than the improving position. |
| Documented 25 to 41 |
Test one control end to end and see whether it holds. Close the gap between the written policy and the observed practice on the three highest risk tools. Give the governing body written authority to stop a deployment, and then let it stop one. Set thresholds so that a metric triggers an action rather than a conversation. |
| Operating 42 to 55 |
Move from go live validation to continuous real world monitoring by site and by setting. Establish independence between the party that operates a tool and the party that reviews it. Extend coverage to the tools that entered through partners and affiliated sites. Retain supporting records on a defined schedule rather than by habit. |
| Mature 56 to 70 |
Have the position reviewed by an independent party, because a self assessment is not evidence of anything. Organize the supporting file so that it can be produced rather than assembled. Then turn the same discipline outward and make it an advantage in contracting, in diligence, and in the market. |
Run this assessment twice. Once alone, and once with technology, security, and operational leadership in the room. Where the two totals disagree by more than ten points, you do not have a governance problem. You have a shared understanding problem, and it has to be resolved before any control you write will hold.
Most oversight programs fail at the first step, which is not policy. It is classification. Two tools that look identical to the person using them can carry very different risk, and until a tool is classified, every control applied to it is a guess. Run every tool through these three questions in order and record the answer in the inventory record itself.
The question to ask the people using the tool. Can the person operating a given tool tell you whether it is assistive or autonomous? If the answer is no, the classification exists on your side of the organization only, and the person whose name goes on the output is carrying that risk without the information.
Two companion resources.
The AI inventory record template. The field structure behind a defensible inventory, ready to populate rather than design. This is the first action for any organization scoring in the lower two tiers.
Twelve questions every healthcare board should ask about AI. A governance brief written for directors, covering business, administrative, and clinical artificial intelligence, and the patterns of liability behind them. Written to be handed to a committee without a covering explanation.
Both available at hlthworks.com
After the assessment
HLTHWORKS builds artificial intelligence governance, oversight, and audit readiness for medical groups, health systems, health plans, payers, and the vendors and investors serving them. Our executives have operated inside these institutions rather than advised them from outside, which is why our work arrives as controls that can be executed rather than frameworks that cannot.
An independent read of your position across all seven domains, with evidence tested rather than described, delivered as a board ready finding.
Discovery, classification, and an inventory record built to be produced on request rather than assembled under pressure. Template available at hlthworks.com.
The oversight operating system: thresholds, monitoring, independence, escalation, and a supporting record organized to be produced on request.
Hear the conversation behind this assessment. Episode 5 of Redesigning Healthcare: Unfiltered, Medical Groups and AI: Building the Right Foundation, with Tim Burke, Chief Compliance Officer of Radiology Partners.
This preliminary maturity assessment is provided for general informational purposes. It is not legal advice, it does not create an attorney client relationship, and completing it does not create a professional relationship with HLTHWORKS. It is not an audit, an accreditation, a certification, or an opinion, and it does not measure compliance with any statute, regulation, accreditation standard, or contractual requirement. Scores and tiers are descriptive of program maturity only and are not findings, conclusions, or representations about legal or regulatory position. Requirements applicable to artificial intelligence in healthcare vary by jurisdiction and change frequently. HLTHWORKS makes no warranty as to accuracy, completeness, or fitness for any purpose, and accepts no liability for actions taken or not taken in reliance on this document. Consult your own legal counsel and compliance leadership before relying on any position described here.