How Do You Attest to What Isn’t There? The Access Review Population Problem

By Nicholas Hunt — Chief Identity Officer, From the CIdO

How do you attest to what isn’t there?

Every access review ends with a signature on a list. When the list is wrong, the signature certifies the error, and the finding gets written up as a reviewer problem: a terminated user nobody caught, a revocation nobody evidenced. The remediation plan writes itself. Retrain the managers, add a second sign-off.

Before you approve it, ask whether this was a reviewer failure or a population failure. A population failure means the listing was incomplete, the revocation never executed in the app, or the proof of removal doesn’t exist. Those three are what the auditor tests, and reviewer diligence fixes none of them.

What the auditor is actually testing

Auditors don’t take your user listing on faith. When they rely on information the company produces, AS 1105 directs them to “[t]est the accuracy and completeness of the information, or test the controls over the information” (PCAOB, 2024; AS 1105). PCAOB staff spell out what that means for a control that runs on a report: its effectiveness “depends in part on the controls over the accuracy and completeness of the system-generated data or reports” (PCAOB SAPA 11, 2013). An access review runs on a listing, so the listing is under test too.

For a connected app, the auditor can test the controls over the integration and rely on them for the period. For an app outside the IGA, each export is its own piece of evidence. Audit-firm guidance tells teams to retain “Pre- and post-parameters” and the original reports, and warns that “changes need to be confirmed if access was removed” (Schneider Downs, 2025). That work repeats per app, per cycle. ESG/Omdia research (2025) found 54% of available apps adequately integrated with IGA, so nearly half aren’t (Dark Reading). The argument applies to apps in audit scope; your SOX boundary is narrower than your app estate.

The evidence chain has three links

Listing → decision → removal. The reviewer owns only the middle link. On a disconnected app, the other two are manual.

Microsoft Entra brings disconnected apps into access reviews as CSV files uploaded when the review starts. The listing is point-in-time at export; whatever changes in the app during the campaign never reaches the reviewer. Every row must also “match a valid Microsoft Entra user,” so an account that maps to no one (an orphan, a shared login, a local admin) can’t enter the review at all. For a denied decision, Microsoft’s instruction is to “Remove access from your own system and then patch each decision item to indicate success or failure for removal.” Its sample record of that removal reads "applyDescription": "ServiceNow ticket created" (Microsoft Learn, 2026). SailPoint’s documented fallback has the same shape: when it can’t write to the source, “it creates a task, assigned to the source owner” (SailPoint).

On a disconnected app, a revoke is a ticket, not a removal. The IGA records an assertion that someone made the change. You can still prove the removal, but only by hand, one revocation at a time. Microsoft offers a Logic App to automate the upload and the apply results, but you build and maintain it, which is the connectivity work I’m arguing for.

Take an illustrative regional bank: about 6,000 employees, an IGA running periodic certifications, and a treasury payments app, call it PayHub, that is in SOX scope with no SCIM or usable API.

PayHub outside the IGAPayHub connected
ListingAdmin exports a CSV the day the campaign opens; the auditor tests who ran it, the parameters, and whether the record count ties to the appIGA aggregates on schedule; the auditor tests the controls over the integration
Four “revoke” decisionsFour tickets to the app owner, marked applied when someone marks themFour deprovisioning operations; the next aggregation confirms them
Evidence of removalA ticket closure plus a re-pulled listing or screenshot, per revocationThe system record of the change and the reconciled account state
Leaver checkHR leaver list against the CSV; one leaver still active becomes the exceptionThe leaver event drives removal; the residual check runs against aggregated data
OrphansUnknown, because PayHub has never been correlated to identitiesAn uncorrelated-accounts list to resolve before the audit

The finding from the left column reads “terminated user retained access.” It looks like a reviewer miss. The cause is the column it sits in: PayHub is outside the IGA, so the leaver event never reached it.

Concede the reviewer, then check the population

Rubber-stamping is real. PCAOB staff say of management review controls generally that “Verifying that a review was signed off provides little or no evidence by itself about the control’s effectiveness” (PCAOB SAPA 11, 2013). My claim is narrower. I haven’t found a study that measures what share of access-review findings trace to the population rather than the reviewer, so this is a diagnostic, not a statistic: when a finding repeats, check the population before you blame the reviewer.

Fix it in order

  1. Connect. Bring in-scope disconnected apps under automated, bi-directional connectivity, so the IGA aggregates their accounts and writes changes back. StackBob.ai is one vendor working this layer. It extends governance to applications that lack SCIM, APIs or connectors (StackBob), works behind existing IGAs such as SailPoint and Microsoft Entra ID (StackBob docs), applies the changes the IGA provisions in the app, and hands the app’s accounts back to the IGA for aggregation and correlation (StackBob docs). (Disclosure: I’m its co-founder and CEO.) It isn’t an access-review product; your IGA still runs the campaign, on aggregated data instead of hand-pulled exports.
  2. Reconcile. Correlate each newly connected app’s accounts to identities and work the uncorrelated list (SailPoint). If you’ve never reconciled an app, you’ve never checked it for orphans. Find them before the auditor’s sample does.
  3. Automate revocation. Once the IGA can write to the app, a revoke executes and the next aggregation shows it happened. The evidence of removal stops being a screenshot.
  4. Then target and shrink the reviews. With a complete population underneath, risk-based scoping and smaller campaigns become defensible. That review-design argument is its own piece; it holds only once the population does.

Start with last cycle’s exceptions

You can test this on your own data before the next campaign. Pull last cycle’s access-review exceptions and tag each one with the app behind it. Mark which of those apps sit outside the IGA, then mark which link failed: listing, decision or removal. If the exceptions cluster on disconnected apps and on the first and third links, you have a population failure, and your remediation plan should say so.

Why this audit season

The amended AS 1105, which emphasizes “the importance of controls over information technology,” applies to audits of fiscal years beginning on or after December 15, 2025 (SEC, 2024). For calendar-year filers, the year-end audits landing January through March 2027 are the first under it. It isn’t an access-review rule, but it leans harder on IT controls, and systems findings already concentrate on user access. Among companies disclosing systems-related material weaknesses in KPMG’s 2025 study, “A vast majority reported issues in managing user access, including inappropriate access rights, failure to timely manage terminated users, and inadequate access reviews” (KPMG, 2026).

When those findings land, someone will propose more reviewers. A second sign-off reinforces the middle link of a chain that broke at the ends. A remediation plan that adds reviewers instead of connectivity buys next year’s finding, with more signatures on it. Connect the population first, and the reviewer finally has something worth attesting to.


References

  1. “Spotlight: Inspection Observations Related to Auditor Use of Data and Reports” — PCAOB staff — April 2024 — https://assets.pcaobus.org/pcaob-dev/docs/default-source/documents/data-and-reports-spotlight.pdf
  2. “AS 1105: Audit Evidence” — PCAOB — current (amended 2024) — https://pcaobus.org/oversight/standards/auditing-standards/details/AS1105
  3. “Staff Audit Practice Alert No. 11: Considerations for Audits of Internal Control Over Financial Reporting” — PCAOB staff — 2013-10-24 — https://pcaobus.org/Standards/QandA/10-24-2013_SAPA_11.pdf
  4. “User Access Reviews: 16 Tips to Meet Auditor Expectations” — Schneider Downs (Dave Snyder) — 2025-03-19 — https://schneiderdowns.com/our-thoughts-on/user-access-reviews-tips-to-meet-auditor-expectations/
  5. “Identity Governance and Administration, App Proliferation, and the App Integration Chasm” (ESG/Omdia research, 2025) — Dark Reading (Todd Thiemann, ESG/Omdia) — 2025-11-14 — https://www.darkreading.com/identity-access-management-security/identity-governance-administration-app-proliferation-app-integration-chasm
  6. “Include custom data provided resource in the catalog for catalog user Access Reviews” — Microsoft Learn (Entra ID Governance) — updated 2026-08-13 — https://learn.microsoft.com/en-us/entra/id-governance/custom-data-resource-access-reviews
  7. “Understanding Certifications” — SailPoint product documentation — accessed 2026-11-06 — https://documentation.sailpoint.com/saas/help/certs/understanding_certifications.html
  8. “Correlation” — SailPoint product documentation — accessed 2026-11-06 — https://documentation.sailpoint.com/saas/help/accounts/correlation.html
  9. StackBob.ai (homepage; product claims as stated by StackBob) — StackBob — accessed 2026-11-06 — https://www.stackbob.ai
  10. “Supported IGA platforms and capabilities” — StackBob help center — 2026-09-22 — https://help.stackbob.ai/en/articles/17061307-supported-iga-platforms-and-capabilities
  11. “Set up SCIM provisioning from SailPoint” — StackBob help center — 2026-09-25 — https://help.stackbob.ai/en/articles/17085173-set-up-scim-provisioning-from-sailpoint
  12. “Order Granting Approval of Amendments Related to … Technology-Assisted Analysis of Information in Electronic Form” (Release No. 34-100774) — U.S. SEC — 2024-08-20 — https://public-inspection.federalregister.gov/2024-18986.pdf
  13. “Trends in material weaknesses — KPMG LLP 2025 study” — KPMG LLP — 2026 (study performed summer 2025 on Audit Analytics data) — https://kpmg.com/us/en/articles/2026/trends-material-weaknesses.html
LinkedIn

Leave a Reply

Your email address will not be published. Required fields are marked *