Your signup flow has forty unit tests and none of them touch the part that actually decides whether a user gets in. SMS OTP is usually the last untested step in onboarding: it depends on a third-party carrier, a short code, a clock, and a device that is not the one running your test suite. So it gets stubbed out, and the failures that only appear in production — expired codes, lockout counters that never reset, resend buttons that loop — ship quietly.
This guide is for the teams that want to test the real path. It covers how to get numbers that receive actual SMS in CI, which OTP behaviors deserve test cases, and where the line sits between testing your own system and abusing someone else’s.
Why OTP is the most under-tested part of signup
Three structural reasons:
- The dependency is external and asynchronous. The code arrives out of band, on a device your test runner does not control, after a variable delay.
- The interesting states are negative states. Expiry, wrong codes, lockouts, resends, and rate limits are where the bugs are, and those are exactly the paths teams skip because they are awkward to trigger.
- Mocks lie about timing. A stub that returns 200 OK instantly will never reveal that your code expired in 90 seconds while your SMS route takes 3 minutes.
The measurable cost is not the test coverage number. It is the support tickets from users who never received a code, and the conversion drop you cannot see in analytics because the user simply left.
Five ways to get a number for OTP testing
| Approach | Realism | Automation | Cost | Best for |
| Mock/stub the SMS provider in code | Low | Excellent | Free | Unit tests, business logic, error branches you inject manually |
| Provider sandbox / test credentials | Medium | Good | Free | Testing your integration code against the provider’s API contract |
| One shared team phone (SIM in a drawer) | High | Poor | Low | Manual smoke tests |
| Phone farm (bank of SIMs + modems) | High | Medium | High capex + ops | Very high volume, regulated industries |
| Rented virtual/real numbers | High | Good | Low, per number | End-to-end tests in CI, multi-country coverage |
Almost every mature team ends up with a mix: mocks for the fast inner loop, rented numbers for the end-to-end suite, and one physical phone for the sanity check that the carrier path works at all.
When virtual numbers are the wrong tool
Be honest about the failure modes before you build on them:
- VoIP ranges have no SMS route. If your number type cannot receive SMS, you are testing your timeout path, not your success path.
- Some platforms block pooled ranges. Fine for testing your product. Not fine if you are trying to automate signups on someone else’s platform — see the compliance section.
- Delivery latency is real. That is a feature for testing: it forces you to handle slow delivery properly.
What a testable number pool looks like
1. A registry of numbers. One entry per number with country, type, rental expiry, and which suite owns it. Numbers are a finite resource; treat them like a connection pool, not like config values.
2. Reserve and release semantics. A test leases a number for the duration of a spec, then releases it. Two parallel CI jobs must never lease the same number — otherwise both read each other’s codes and you get flaky tests that pass locally.
3. An inbox reader. Rented numbers expose a private web inbox; read it through the provider’s API from your test harness. The shape below is illustrative — check the API docs in your WhatsNumber console for the exact endpoints:
4. A stale-message guard. Always filter by since. Without it, you will happily assert against the code from the previous test run — the single most common cause of “the OTP test passes but the product is broken.”
5. Cleanup. Release the number, and if the platform you are testing permits it, delete the test account. Leftover accounts consume your test numbers’ eligibility.
The OTP test matrix
These are the cases worth automating. Most take under a second once the pool exists; the value is that they run on every commit instead of never.
Format and input handling
| # | Case | Expected |
| 1 | Number entered in local format with leading zero | Normalised to E.164 before sending |
| 2 | Country code duplicated by the UI and the user | No double prefix; validation error if ambiguous |
| 3 | Spaces, dashes, parentheses in input | Stripped, sent successfully |
| 4 | Number with extension suffix | Rejected with a clear message |
| 5 | Non-mobile number type | Rejected, or routed to voice as designed |
Code validation
| # | Case | Expected |
| 6 | Correct code, within TTL | Accepted, session issued |
| 7 | Correct code, one second past TTL | Rejected |
| 8 | Wrong code, single attempt | Rejected, attempt counter increments |
| 9 | Wrong code × N | Locked out; counter resets only as designed |
| 10 | Code from a previous request | Rejected (old codes invalidated on reissue) |
| 11 | Code with leading zeros | Accepted — leading zeros are the classic stripping bug |
| 12 | Code with surrounding whitespace / pasted with a space | Accepted |
| 13 | Code replayed after successful use | Rejected — single use |
Delivery and rate behaviour
| # | Case | Expected |
| 14 | Resend before cooldown | Blocked with visible timer |
| 15 | Resend ×3 in a minute | Server-side rate limit, not just client-side |
| 16 | Delivery takes 90 seconds | UI still shows a usable “resend” or “call me” path |
| 17 | SMS never arrives | Clear fallback: voice call or email, not an infinite spinner |
| 18 | Same number requested from two devices | Defined behaviour, no silent account takeover |
Cases 14 through 17 are the ones that produce the support tickets, and they are the ones a stubbed provider can never validate.
Wiring it into CI without leaking credentials
- Keep the inbox API key in your CI secret store, never in the repo. A leaked key exposes message history for every number you rent.
- Tag OTP tests and run them on a schedule, not on every push. End-to-end OTP suites take 30–120 seconds per spec; nightly plus pre-release is the right cadence, with mocked tests on every commit.
- Fail loudly on timeout, with the number and the request timestamp in the message. “OTP timeout” tells you nothing; “OTP timeout on +1XXXXXXXXXX after request at 14:02:11” tells you whether to blame the route or the code.
- Cap concurrency to your number count. Ten CI jobs and three numbers means seven jobs are waiting on a lock, not testing.
- Assert on the message, not just the digits. Check the sender ID and template text occasionally — carriers and providers do change them, and your parsing will break before your users notice.
The cost model
Rented numbers are cheap relative to the alternative, but the arithmetic is worth doing:
- Numbers are billed per number, per period — a 12-month rental is typically half the monthly rate of a 1-month rental, so map the pool to your test suite’s real lifetime, not to a single sprint.
- Multiply by how many countries you support. Most teams only need one or two for the main suite and a single number for each additional locale they ship.
- Compare against the cost of one production incident in the signup funnel, or one support agent-hour per week spent on “I never got my code”.
For most teams the pool pays for itself the first time it catches a lockout counter that does not reset.
Where the compliance line sits
Testing your own product’s OTP flow with rented numbers is ordinary engineering practice. Two things are not:
- Automating account creation on third-party platforms to farm accounts, bypass limits, or scrape. That violates those platforms’ terms and, in many jurisdictions, fraud law. Rental numbers are for verifying your own systems.
- Testing with real users’ numbers or production data. Generate synthetic identities for test accounts and keep the numbers you rent clearly separated from any customer data.
A useful rule: if the service you are verifying is not yours, you are not testing — you are abusing someone else’s onboarding.
FAQ
Can I test SMS OTP without a physical phone?
Yes. A rented number with a private web inbox receives real SMS, and your test harness reads the code through an API. No SIM, no device, no human in the loop.
Do virtual numbers receive SMS the same way real numbers do?
They receive real messages, but the delivery route differs. Mobile ranges have the widest compatibility; VoIP ranges may have no SMS route at all. For end-to-end tests, match the number type to what your production traffic uses.
How many numbers do I need for a CI test suite?
Start with one per country per parallel CI job, plus one spare. Scale concurrency, not the pool — fewer numbers with a strict lease is far more reliable than many numbers shared loosely.
How do I stop tests from reading a stale code?
Filter messages by timestamp from the moment of the request, and invalidate old codes in your own system on reissue. Both, not either.
Can I automate OTP tests for a third-party app?
Not legitimately. Automating signups on platforms you do not own violates their terms. Use mocks or provider sandboxes for integrations you are building, and rented numbers for systems you operate.
What is the fastest way to get started?
Rent one number in your primary market, write a single end-to-end spec that goes from signup to authenticated session, and let it run nightly. Add cases 14–17 from the matrix next — that is where the real bugs are.
The takeaway
OTP is not hard to test; it is hard to test honestly. Mocks prove your logic; only a real number proves your delivery path. Rent a small pool, lease it properly, and assert on the timing and negative cases that a stub can never reproduce.
Get numbers for your test suite → · New to rented numbers? Start with how SMS verification numbers work.

