Published: 23 May 2025 · Last updated: 15 October 2026
AI receptionist vendor checklist for Australian businesses
A practical 25-question checklist to vet AI receptionist vendors. Covers guardrails, handoff, privacy, uptime, integrations, ownership, and lock-in.
By Valory AIThe Valory AI team
Compare the workflow you intend to buy: what the caller hears, what happens next and who deals with exceptions. Use this evidence sheet during a demonstration, then work through the 25 questions with each provider.
Valory AI provides managed AI receptionists. This checklist is designed to help you evaluate any provider, including us.
Jump to: 25 questions · Six scenarios · Copyable evaluation record
Copy this evaluation record
Keep one record per scenario. Use fictional details and a test setup agreed with the provider.
Provider and proposed scope: [Service and configuration being evaluated].
Scenario: [Caller request and agreed variation].
Expected outcome: [What the caller should hear and what should happen next].
Observed conversation: [Short factual note, without private customer content].
Result checked: [Test message destination, calendar entry, connected recipient or other relevant result].
Status: Demonstrated / Not demonstrated / Failed / Outside proposed scope.
Gap and owner: [What remains unresolved and who will address it].
Next decision: [Accept this scope, request a retest, change scope or stop].
Use the status labels consistently:
| Status | Meaning |
|---|---|
| Demonstrated | The agreed scenario produced the expected result in the stated test setup. |
| Not demonstrated | There is not enough evidence to assess the result. A verbal promise alone belongs here. |
| Failed | The observed result did not meet the agreed expectation. Record the discrepancy. |
| Outside proposed scope | This capability is deliberately excluded. Record the alternative your team will use. |
Do not average a material failure away with high scores elsewhere. A useful message-taking scope can be evaluated on its own requirements; it does not need to include booking or transfers.
Filled example: an unconfirmed appointment request
This is a fictional test example, not a result for any provider.
Scenario: The provider demonstrates a request while its test calendar is unavailable.
Expected: No appointment confirmation; the request reaches the agreed test recipient for follow-up.
Observed: The receptionist explains that the time cannot be confirmed and records the request.
Result checked: The test recipient received the requested time, unconfirmed status and next owner.
Status: Demonstrated — the agreed fallback only.
Still to check: A successful booking in the intended configuration.
That fallback result does not prove normal booking or every outage path. If the receptionist instead says “You're booked” without a confirmed appointment, record a failure and resolve it before relying on that capability.
The 25-question vendor checklist
Ask for specific answers and relevant written terms, settings or demonstrations. Record anything that remains unclear.
A. Accuracy and guardrails
1. What will it handle on day one? List the exact call types and actions included, plus what is out of scope.
2. What happens when it does not know the answer? Ask how unapproved information and uncertain requests are handled, then demonstrate one example.
3. Where does the business knowledge live? Who updates it, how are changes reviewed, and which version is being used?
4. Can we approve sensitive answers and boundaries? Check pricing, cancellations, refunds and any clinical or other specialist topics relevant to your business. Define what must go to staff.
For pricing calls, distinguish approved prices, estimates and tailored quotes before agreeing what the service may say.
5. How do we test changes before they go live? Confirm the review process for prompts, knowledge, routing and connected tools, including how to undo a change.
B. Escalation and human handoff
6. What triggers escalation? Agree how the service handles uncertainty, an upset caller, billing disputes and requests covered by your urgent-call policy.
7. What does a handoff actually do? Distinguish a transfer attempt, connection to a person, captured message and callback request. Ask which options are included.
8. What context does staff receive? Inspect a fictional example showing the request, checked contact details, action taken and remaining work. Confirm the recipient and permitted access to recordings or transcripts.
9. Can the rules vary by call type and hours? Check normal hours, busy periods, closed hours and planned staff absence.
10. What happens when the next person is unavailable? Demonstrate the agreed no-answer, voicemail or closed-hours fallback. Confirm who owns the follow-up.
C. Privacy and data controls
11. Are calls recorded or transcribed, and what choices exist? Ask what callers hear about AI use and recording, what happens if they object, and which settings can be changed. Confirm your own disclosure and recording obligations.
12. What personal information is collected, and where does it go? Identify the default fields and how to collect only what the workflow needs. Ask where data is processed and stored, and which subprocessors receive it.
13. Who can access or reuse the records? Check staff and vendor access, role controls and available audit records. Ask whether the provider or its subprocessors use information for model training, and request the actual terms and available controls.
14. How do retention and deletion work? Ask about specific calls, transcripts, exports and backups, alongside your own record-keeping requirements.
15. How are incidents handled? Confirm detection, notification, support contacts and responsibilities when something goes wrong.
For further questions to take into that discussion, see privacy and call recording for AI phone agents in Australia. A vendor's settings do not establish your legal obligations.
D. Uptime and failure modes
16. What do the uptime and capacity commitments cover? Ask how uptime is measured and which carrier, model or integration failures are excluded. Confirm simultaneous-call limits and how capacity is allocated to your service.
17. What happens when a component fails or capacity is reached? Agree the fallback for answering, message delivery, transfer and scheduling, including what callers hear when simultaneous-call limits are reached.
18. How are failures detected? Ask how missed calls, tool errors and incomplete transfers become visible, and who responds.
19. Can we distinguish call outcomes? Inspect how the service reports answered calls, captured enquiries, attempted transfers, completed connections and confirmed bookings where included.
20. How do we know the follow-up reached its destination? Ask how a missing notification or unowned request is detected and recovered. A completed conversation is not evidence that the next step succeeded.
E. Integration, costs and ownership
21. Who controls the phone number, and what happens if we leave? Confirm the number's ownership, porting eligibility, process, costs and contractual restrictions in writing.
22. Can we retain our existing number and business identity? Clarify forwarding or porting requirements and who can restore the previous phone route.
If you already use an answering provider, see how to switch answering services and keep your business number for the changeover sequence, message checks and fallback arrangements.
23. Which exact actions are supported in our tools? For each CRM or calendar, distinguish reading information, creating a record, making a booking, rescheduling and cancelling. Ask whether support is native, custom or dependent on another service.
24. Who owns the data, and how can we export it? Confirm available formats, access after cancellation and any deletion or retention conditions.
25. What is the total payable for our proposed scope? Include setup, subscription, usage, messages, integrations, overages, support and exit costs. Ask for the currency, a complete written total and the assumptions that could change it. Compare providers using the same scope and usage estimate.
Six scenarios to test against your needs
Agree the scenarios and expected outcomes with the provider before testing. Use fictional contacts, approved test recipients and an isolated calendar where relevant. Do not interrupt real customers or emergency services to create an edge case.
Ask which functions the demonstration environment can prove. A browser conversation, recorded example and connected phone test may cover different parts of the workflow. Record the channel and configuration used. For example, ElevenLabs' Reception.ai testing guidance identifies functions that require phone testing; ask your provider about the limits of its own test setup.
| Scenario | What to ask for | Result to inspect |
|---|---|---|
| Ordinary new enquiry | Ask about an approved service and supply the required fictional follow-up details. | Correct request and contact details reach the intended test recipient with a next action. |
| Caller interrupts and corrects details | During an agreed phone test, interrupt an answer, then correct a fictional name and callback number. | The receptionist responds appropriately to the interruption, checks the correction and sends the final details without contradictory instructions. |
| Request outside the rules | Ask for an unapproved price, service or promise. | The receptionist explains the boundary and takes the agreed next step. |
| Requested person unavailable | Ask for a test recipient who cannot accept the call. | The agreed fallback occurs; the record distinguishes an attempt from a successful connection. |
| Successful booking, if included | Request a supported appointment in a controlled test calendar. | The appointment exists with the correct time zone and details; spoken confirmation matches it. |
| Booking cannot be confirmed, if included | Have the provider safely demonstrate an unavailable or uncertain calendar result. | No success is claimed without confirmation. The pending request has an owner, and an uncertain result is checked before retrying. |
Repeat the relevant scenarios under closed-hours rules if that coverage matters to you. Include the absence of the person who would normally receive the request.
If your callers use varied accents or often ring from noisy places, our guide to accents and background noise in AI phone calls explains what affects recognition and what a useful correction should achieve.
The accounting walkthrough describes named-staff routing, booking boundaries and refinement in one published implementation. The enquiry handoff template helps you inspect the record staff need afterwards. Neither replaces a demonstration of your proposed setup.
Decide what needs fixing before you buy
Review the evidence with the person who will receive calls and follow-up. Ask:
- Is the demonstrated scope useful? Identify the part of your actual phone workload it can handle.
- Are material gaps resolved or explicitly excluded? False completion, an incorrect recipient or an unsupported promise needs attention before relying on that path. An excluded feature needs a workable alternative.
- Is ownership clear? Name the people who maintain business knowledge, staff availability, connected tools and fallback rules, and how you request a correction.
- Do the terms fit? Check the cost assumptions, service hours, support, number and data arrangements alongside the demonstration.
An email handoff can be appropriate when it fits the response need and someone owns it. A promise to “handle anything”, unclear data terms or a workflow with no accountable next recipient needs further examination.
Agree any limited live trial separately: its scope, responsible owner, review point and how to restore the previous phone route. There is no universal number of test calls that proves reliability. Cover the material variations for the workflow you intend to use.
To define the coverage window, use the peak-period phone triage guide or after-hours call-handling guide.
After launch, keep an answered call, useful enquiry, confirmed appointment and paid job as separate outcomes. Review only records your business is authorised to access and leave unknown outcomes unknown.
For an overview of provider options, see our Australian AI receptionist buyer guide. If you are still choosing the delivery model, start with managed AI versus a platform. Once the scope is agreed, use the setup guide for the next stage.
Bring your scenarios to a walkthrough
Bring three common call types, the next person who should receive each request, and one situation where that next step is unavailable. We can work through the proposed scope, evidence to inspect and fallback your team can support.