The product app changed. What creative must you retest?

01

What software states should you compare?

Compare the state captured during filming with the experience a customer receives now. The comparison should cover more than the app version number.

Use the following review matrix: | Compare | Recorded state | Current customer state | Retest question | |---|---|---|---| | App identity | Name, developer and store listing | Current listing and developer | Is this still the correct download? | | App release | Filmed version and build | Current production version and build | Did navigation, setup, pricing or features change? | | Firmware | Version used during filming | Version installed or required now | Does updated hardware behave as shown? | | Customer device | Phone model and OS used on set | Representative supported devices | Can customers install and operate the app? | | Market | Store country used for filming | Every campaign destination | Is the app available in each targeted market? | | Account | Existing, free, trial or paid | New-customer account | Are registration, payment or verification now required? | | Permissions | Previously granted access | Clean installation | Which prompts interrupt the demonstrated flow? | | Feature | Controls and result shown | Current app, firmware and service | Can the complete claim still be reproduced? |

The filmed state includes hidden conditions. A creator may already be signed in, have permissions granted, own a paid subscription, use saved Wi-Fi credentials, or have paired the device before recording. Those conditions can make setup look shorter than the customer journey.

Start a separate supplier-change review if fulfilled inventory may contain different hardware revisions or preloaded firmware. A software retest cannot establish that every hardware batch behaves alike.

02

Why is testing only the latest app version insufficient?

One successful test on one phone is not enough because compatibility can vary by location, device characteristics, and Android version. Google Play also supports staged updates by rollout percentage and country, meaning customers may not all receive the same release at once.

Test both a new installation and, where practical, an existing customer’s upgrade path. A previously paired device may retain settings, cached screens, permissions or firmware that a new customer will not have.

If two production versions remain in circulation, ask whether the ad is accurate for both. If it is accurate only for one, limit the claim clearly or pause the asset until the rollout stabilizes and the supported state can be verified.

1. Check the version offered to a new installation in each campaign market.
2. Check the upgrade path on an existing account and paired product where possible.
3. Review available release notes or supplier confirmation for a staged rollout.
4. Record every app and firmware version encountered.
5. Do not clear the old footage merely because one test device retains the filmed interface.

03

Which advertising claims require a new test?

Retest every express or implied claim that depends on software. An express claim is stated directly; an implied claim is the message viewers are likely to take from the words, images, timing, and omitted steps together.

The FTC’s advertising-substantiation policy says advertisers need a reasonable basis for objective claims before dissemination. Build the claim inventory from voice-over, captions, labels, calls to action, app screens, demonstrations, notifications, charts, maps, before-and-after sequences, and the order of edited shots.

Pay particular attention to words such as “instant,” “automatic,” “free,” “remote,” “real time,” “works anywhere,” and “no subscription.” Also inspect omissions. Showing one tap followed immediately by a physical result may imply that no account creation, calibration, firmware installation, accessory, permission or paid plan intervenes.

Retest remote control, notifications, schedules, automations, historical data, exports, cloud storage, multi-user sharing, offline behavior, voice-assistant integrations, AI or detection functions, and any claim about setup time, speed, range, accuracy or battery effect. Check whether each advertised function remains free or now needs a subscription, trial, credit balance or in-app purchase.

04

Which screens and setup steps should be recorded again?

Record the complete clean-install journey from app-store search to successful use of the advertised feature. Capture failures and recovery steps as well as the ideal path.

First check app-store search results, the current developer and listing, required operating-system information, pricing or subscription language, release notes, and whether the install button appears. Google explains that customers see apps compatible with their device and Play country, while compatibility can be affected by factors including Android version, device characteristics and location.

Next delete the app, reset relevant permissions, unpair or factory-reset the test product when safe, and create a new account. Record welcome screens, region selection, registration, verification, terms, privacy choices, tutorials, trial offers, payment prompts and calibration.

Retest permissions individually. Android documents runtime permissions for nearby-device Bluetooth scanning and connection in relevant app configurations. Test allowing all permissions, denying one, granting only some, and recovering after denial. An ad that says “just open the app” may need revision if the current journey contains consequential intervening steps.

Finally record power-on, reset mode, QR or serial entry, Bluetooth discovery, local-network access, Wi-Fi requirements, account binding, naming, room assignment, firmware installation, restart, reconnection and feature completion. Note any required hub, accessory or supported Wi-Fi band.

05

How should firmware changes be tested?

Test the firmware a customer receives before and after setup, including whether the update is forced. Firmware is software embedded in or controlling the product, so an app interface that looks unchanged does not establish unchanged device behavior.

NIST defines an update as a modification intended to correct security or functionality problems. For creative review, the important question is whether that modification changes any demonstrated control, indicator, alert, limit, integration, setup step or result.

Record the starting firmware, update destination, elapsed process, required customer actions, and whether the product remains usable during installation. Retest pairing, reconnection, factory reset and repair after the update. Compare older inventory with newly fulfilled inventory when batches might begin from different firmware states.

If you cannot reproduce the core advertised result after the update, stop the affected asset. Do not substitute a disclaimer for a demonstration that now depicts unavailable behavior.

06

Should the asset be kept, annotated, recut, reshot, or retired?

Choose the least disruptive route that leaves the complete ad accurate. Judge the whole sequence, not only the changed app shot.

| Route | Use it when | Required action | |---|---|---| | Keep | Setup, availability, limitations and results materially match across the test matrix | Preserve evidence and assign another review date | | Annotate | A minor variation can be prevented from misleading with a short, prominent qualification | Place the qualification beside the affected claim; use audio too when needed for a spoken claim | | Recut | Obsolete screens can be removed without inventing a smoother sequence or changing the message | Remove the shots and reassess the complete edit | | Reshoot | Setup order, permissions, interface, timing, payment or behavior changed materially | Film the current new-customer journey on representative devices | | Retire | The app is unavailable, the central feature was removed, the result fails, or a qualification would contradict the concept | Stop delivery and replace the concept |

FTC digital-disclosure guidance emphasizes whether disclosures are proximate, prominent, noticeable and understandable. A distant caption or fast footnote should not be relied on to reverse the central message communicated by the footage.

After any recut or annotation, watch the ad without production notes. Ask what an ordinary viewer would believe about installation, cost, speed, compatibility and results. Reopen the claim review if the edit still skips a material dependency.

07

What belongs in the evidence snapshot?

The evidence snapshot is a dated record of the exact product and software state represented by an approved creative asset. It should let a later reviewer reconstruct why the ad was kept, changed, or retired.

Record the creative ID, approval date, test date, product SKU, supplier, hardware revision, packaging revision and known fulfillment batch. Add the app name, developer, store URL, app version and build, plus firmware versions before and after setup.

Record the phone or tablet model, operating-system version, store country, locale, app language, network type, Wi-Fi band, Bluetooth state, required accessories, permissions, and whether the account was new or existing, free, trial or paid.

Attach store-availability screenshots, the app’s version or About screen, release notes, and an unedited recording of installation, pairing, firmware update and the claimed feature. For every claim, record the expected result, observed result, failures, known variations, disclosure text, reviewer, decision and next review date.

Use consistent asset versioning so the evidence cannot be separated from the exported ad. ATIYO can organize the roadmap, brief, brand context, assets, iterations, evidence state, decisions and reusable learnings. Media performance remains in the ad platform; ATIYO does not connect to ad accounts, buy media, calculate ROAS or know performance unless a user records it.

1. Duplicate the current creative record before changing its status.
2. Attach the evidence snapshot and identify the filmed software state.
3. Link every affected claim to its test result and supporting capture.
4. Assign keep, annotate, recut, reshoot or retire.
5. Preserve the previous decision rather than overwriting its history.
6. Set a review trigger for supplier notices, app releases, firmware changes or availability failures.