CQC inspections

How the CQC AI Auditor works: an AI CQC audit for GP practices

CQC AI Auditor team · · 9 min read

In short: CQC AI Auditor is an AI CQC audit for GP practices, built by PCMS, a primary care consultancy. There are two levels. The self check is a structured set of yes and no questions that scores what you tell us. The AI assisted audit reads the evidence your practice already holds, interviews you on it the way an inspector would, corroborates your answers against your documents and scores every one of CQC's 34 quality statements from 1 to 4, using CQC's own method. You get a report with the reasoning, the gaps and a Path to Outstanding. Your first self check is free.

If you have been looking for a CQC audit tool for your GP practice, you have probably met three kinds: a CQC self assessment checklist, a mock inspection, or a consultant's visit. People ask us which one this is. Is it a mock inspection? Is it a checklist? Does it just read my policies? It is an audit, built for CQC inspection preparation, and this post walks through what happens from creating an account to opening the report.

Two levels: start with what you think, then find out if you are right

Level one, the self check, answers the question of where you believe you stand. It is a structured set of yes and no questions covering your policies, your processes and each quality statement. No documents to upload, nothing to prepare. Your answers are scored against the framework and you get a rating and a report, in around one to two hours.

We are open about its limit. The self check scores what you tell us, not what your evidence says. It is still a useful CQC readiness check, because it is quick, it is built around all 34 quality statements, and it shows where you are confident and where you are not. Your first one is free when you create an account, with no card needed.

Level two, the AI assisted audit, answers the harder question: can you show it? You upload the evidence you already hold. The AI reads it, then interviews you with questions drawn from your own documents. Your answers are corroborated against what those documents actually say, cross referenced across the evidence set, and scored against the single assessment framework.

The line we use on the site is the simplest way to put it: the self check tells you where you believe you stand, and the AI audit tests it against the evidence you actually hold. The How it works page sets out both levels side by side.

What the AI assisted audit does with your evidence

Four steps, in order.

It reads. The evidence you upload is read against all 34 quality statements before you are asked anything. That is where the gaps a report names come from: a safeguarding policy that still names another practice, or a significant event log that records a discussion but no decision.

It interrogates. Then it interviews you. The questions are not a generic list; they come from your own documents. It asks, it listens, and then it asks the follow up. Vague answers get pushed on. If you say you monitor same day demand, it will ask what percentage of appointments are same day, what your DNA rate is, and to see last month's access data. That is deliberately uncomfortable, because the alternative is finding out on the day.

It cross references. Your answers are corroborated against what your documents actually say and cross referenced across the evidence set. Saying it and showing it are treated as different things. This is the step that separates an audit from a questionnaire.

It scores. Each quality statement gets a score from 1 to 4, using CQC's own descriptions: evidence shows significant shortfalls, some shortfalls, a good standard, or an exceptional standard. Statement scores roll up to a percentage and a rating for each key question using CQC's published thresholds and both of CQC's limiting rules. Our How we score page sets out the arithmetic, and CQC's version is on its How we reach a rating page.

Reading, questioning and corroborating is how a real assessment works. It is the difference between what you believe and what you can show.

The interview, and why it is the part people remember

A worked example from the site, illustrative rather than a real practice. The Auditor asks how the practice manages same day demand against pre bookable appointments, and what data it uses to monitor it. The practice answers that it offers a mix of same day and routine appointments, and patients can book online or by phone. The Auditor replies that this is high level, asks for the split, the DNA rate and last month's access data, and marks the question as required for scoring under Equity in access. The current assessment reads: score 2, descriptive but no operational data or evidence.

That is what an inspector does. A good general answer on access scores a 2 until there is data behind it.

What you get back

A written report you can read, act on and hand to your partners:

  • Every quality statement, individually, scored 1 to 4 with the reasoning written out. Not a colour and a number.
  • Gaps, named specifically. The actual thing, such as "safeguarding policy does not name your practice" or "quarterly infection control audit has no follow up recorded", rather than "improve governance".
  • An action plan where a statement falls short, written as what to evidence, not just what to fix.
  • A Path to Outstanding for each area already at good: what a practice would need to demonstrate to move beyond it.
  • A breakdown across the five key questions, and the overall picture.
  • PDF and Excel export, so it can go to partners or into a meeting.

The structure, with illustrative figures, is on our sample report page.

If you have no evidence for something

All 34 quality statements are covered. If you do not have evidence for one, the audit moves on and scores what you can show. You are not blocked, and you are not marked as complete on something you have not evidenced. The report tells you which is which. The most useful thing an audit can tell you is where the gap is, and a tool that refused to finish until every box was filled would hide exactly that.

How long it takes

The self check takes around one to two hours in total. For the AI assisted audit, the tool shows an estimate on each domain before you begin it, usually one to two hours, covering uploading evidence, AI review and follow up questions. That is the top of the range for a domain you are coming to cold. If your documents are already sorted, a domain takes well under an hour. Progress is saved, so you can work through it a domain at a time.

A note on the word domain. The product calls the five areas domains. CQC's word is key question. They are the same five things: safe, effective, caring, responsive and well-led.

And if you get stuck, we will get on a call with you. Practice managers are busy and a compliance tool that leaves you on your own is not much use.

What happens to your documents

For an NHS practice this is usually the first question asked of any supplier, so it has its own page. The short version: files you upload are stored so the audit can read them and are deleted automatically 60 days after the audit completes. The text extracted from them, your scores and your report are kept, because the report is built from them and you come back to it. A file that could not be read is not deleted, because it would be the only copy of that evidence. Documents are processed by our AI provider through its API as part of running your audit. The Trust and data page and the privacy policy carry the detail.

Why we built it

CQC AI Auditor was not designed as a product. PCMS built it for our own team to use on client inspections, because we needed it. Doing this properly means reading everything a practice holds against all 34 quality statements, then testing what it says. That takes a long time however experienced you are, which is why a full audit tends to happen once and then not again for a year or more. We built this to do the same work in a fraction of the time, without losing the rigour. Having used it on real assessments, we decided to make it available to practices directly.

Speed is the point. Practices assessed under CQC's Returning to Good and Outstanding programme get at least five working days' notice, which is enough time to be told and not enough to find and fix something that drifted eighteen months ago. When the work no longer takes days, you can run it before the notice call, and again whenever something changes.

What we do not claim

  • It is not a prediction of your CQC rating. No tool can give you one. It gives you our assessment of your evidence against the single assessment framework, while you still have time to act on it.
  • It is not a replacement for judgement. It supports a consultant and your own professional judgement. It does not replace either.
  • It is not a mock inspection. No third party can fully replicate a CQC assessment. It is an audit of your evidence, done the way an assessment reads evidence.
  • We are independent of CQC and not endorsed by it. We use CQC's published framework, thresholds and wording because that is what you are assessed against, and when CQC changes its approach, we will change with it.

Our promise on the scoring

If CQC publishes a lower overall rating for your practice than our audit gave you, we will refund your fee in full. Three things to know: your CQC assessment needs to take place within six months of your report being issued, you need to be able to show you acted on the recommendations in your report, and it applies to the AI assisted audit, the one where we assess your evidence. Claims are made within 60 days of CQC publishing the rating. The full refund terms are on the site.

Where to start

Create an account and run your first self check free. When you want to know whether the evidence backs it up, the AI assisted audit is the next step. Prices for both are on the pricing page.

Frequently asked questions

Is CQC AI Auditor a mock CQC inspection?

No. It is an audit of the evidence your practice holds, done the way an assessment reads evidence: it reads, questions, corroborates and scores. No third party can fully replicate a CQC assessment.

What is the difference between the self check and the AI assisted audit?

The self check scores your answers to structured yes and no questions and needs no documents. The AI assisted audit reads your uploaded evidence, interviews you on it and scores what you can show.

Does it cover all 34 quality statements?

Yes. If you have no evidence for one, the audit moves on and scores what you can show, and the report says which is which.

Can it predict our CQC rating?

No, and no tool can. It gives you a structured assessment of your evidence against the framework.

Ask us

We do not run comments on Insights. If you have a question about how the Auditor works, ask on our LinkedIn page or reply to the newsletter.

Sources