Here is a vendor-neutral, 10-point yes/no checklist for evaluating any AI tool you are considering for your tax practice. Score your current stack in fifteen minutes. The questions are about control, pricing honesty, language support, and whether client documents leave the office — not about who has the glossiest keynote or the most venture funding. If a product fails the first question and cannot explain why, stop decorating the rest of the scorecard.
This checklist comes at the end of a 20-post series on private AI in tax practice. If you want the definitions first: what private AI is. If you want the framework: Level 1, 2, and 3.
The 10 questions
| # | Question | "Yes" means | Why it matters |
|---|---|---|---|
| 1 | Can client documents be processed without leaving the office? | A Level 3 path exists | This is the fundamental question. Everything else is downstream. |
| 2 | If data leaves, is every recipient named — including subprocessors? | The vendor relationship is WISP-able as Level 2 | Un-named recipients are un-inventoried disclosure targets. |
| 3 | Can you export everything you put in? | No hostage format — your data is not held captive | Vendor lock-in with client data creates a switching cost tied to return information. |
| 4 | Is pricing published without a mandatory demo? | You can budget before a sales conversation | Demo-gated pricing signals that negotiation, not transparency, drives the business model. |
| 5 | Is AI a separate mystery subscription, or part of the core product? | "No" (not separate) is better for most firms | Separate AI subscriptions become shadow IT when staff find them too cumbersome to use. |
| 6 | Does it work in Spanish as a real tax workflow, not a toy language pack? | Bilingual client base is a first-class user | If Spanish is an afterthought, bilingual firms will default to consumer translators — Level 1. |
| 7 | Is there a physical data-control option (local inference)? | Safe vs. vault is your choice, not the vendor's default | The option to keep files local matters even if you choose cloud — it means the architecture supports it. |
| 8 | Are knowledge prompts and document jobs separated in the UX? | Fewer accidental uploads of client files into the wrong channel | One paste box for everything is how SSNs end up in consumer chat logs. |
| 9 | Can you explain the tool in one sentence to a client who asks? | Trust survives the dinner-table test | If the explanation requires a paragraph about cloud tenancy, the client will not be reassured. |
| 10 | Does the vendor's story match what is actually live — no present-tense claims for 2027 features? | You are buying a product, not a press release | Honesty about timing is a trust signal. Apply this to XyloDocs too. |
Print this table. Score your current tools: your public chat assistant, your portal's "AI features," your tax engine's helper, and anyone who cold-emailed you about an "AI operating system." Training pledges are deliberately not on the list — they are the wrong exam question.
How to tally without kidding yourself
The checklist is not a beauty contest where 7/10 is "pretty good." Some combinations reveal structural problems:
- Question 1 no + Question 7 no: You do not have private AI. You have a policy promise about a cloud service. That may be acceptable if Level 2 is your conscious choice — but call it what it is and document it in the WISP.
- Question 4 no: You cannot complete a partner vote in good faith. If you do not know the total cost before committing, you are negotiating blindly during season planning.
- Question 8 no: Your WISP will miss the real data flow. If knowledge questions and document uploads share one interface, staff will mix them — and the WISP inventory will describe the intended workflow, not the actual one.
- Question 10 no: Walk away regardless of score. A vendor that misrepresents what is live vs. what is planned is unreliable on every other answer. This applies to XyloDocs too — if we claim e-filing before it ships, that is a question-10 failure.
How to use this in a vendor conversation
Send the checklist to the vendor before the demo. Ask them to score themselves. Then verify during the demo:
- Question 1: Ask for a network diagram showing where client files travel during processing. If the answer is "to our cloud," follow up with question 2.
- Question 2: Ask for the complete subprocessor list. If they cannot provide it without "checking with legal," that is useful information.
- Question 3: Ask to export a sample file you uploaded during the demo. Verify that the export includes all metadata and is in a standard format.
- Question 4: If pricing is not on their website, ask why. "Our pricing is customized" usually means "our pricing depends on how much we think you will pay."
- Question 6: Ask for a Spanish-language demo. If the product switches to English for key workflows, Spanish is a marketing feature, not a real capability.
Where XyloDocs should score — and where it should not
Apply this checklist to XyloDocs honestly:
- Questions 4 and 6: The communication platform scores well — pricing is published at $49/month, and bilingual communication (English ↔ Spanish) is a core feature, not an add-on.
- Question 1 and 7: The communication platform is a cloud-delivered SaaS (Level 2). Local document AI is the 2027 tax-software design intent. Score XyloDocs honestly as "not yet" on local document processing until the product ships.
- Question 10: This article — and the entire series — is deliberately clear about what is live (communication platform) vs. what is 2027 (tax software, local document AI, e-filing). If we slip into present-tense claims for future features, hold us to question 10.
CRM live today: features. Tax software waitlist: /tax-software.
A printable version for partner meetings
For each AI tool your firm uses or is considering, fill in one row:
| Tool name | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Decision |
|---|---|---|---|---|---|---|---|---|---|---|---|
| (your tool here) |
Include personal chat tools that staff use informally. Shadow AI is where most Level 1 risk lives — the tools nobody wrote down.
Frequently asked questions
Is a 10/10 required?
No. A 10/10 on a tool that is not fully live would be dishonest. Prefer a truthful 6 with a clear roadmap over a fictional 10. The checklist is a decision tool, not a certification exam.
Should I share scores with vendors?
Yes. "You scored 0 on document egress control" is a better RFP question than "do you use AI?" It forces specificity. Some vendors will appreciate the clarity; others will object to the framing — that reaction is informative too.
Does this replace a lawyer's §7216 review?
No. This is an operational filter, not legal advice. Legal review of specific disclosures and consent requirements still sits with your counsel — see IRS-facing duties for the regulatory framework. The checklist helps you decide which tools to bring to that legal review.
What about tools that score well on most questions but fail question 1?
A tool that cannot keep client files local is Level 2 at best. That can be a rational choice if documented in the WISP, supported by proper consents, and contracted with named subprocessors. But it is a different posture than Level 3 — and the checklist forces you to name the difference rather than conflating "good vendor" with "private."
The bottom line
Ten yes/no questions. No brand names required. If your current tools fail "does the file leave?" you already know the central lesson of this series. Use the checklist to move from instinct to documented decision — and apply it to XyloDocs as rigorously as you apply it to anyone else.
Narrative companion: why your local tax firm doesn't need a VC-backed AI platform.
Leave a Comment
Your comment is private and will only be visible to the author. We'll send you an email confirmation.