survey, interview, questionnaire, market, research, customer, satisfaction, marketing, checklist, woman, questions, answers, tablet, poll, survey, survey, survey, survey, survey, questionnaire, questionnaire, questionnaire, questionnaire
Photo by Tumisu on Pixabay

Voice of Customer

Part of Testing redesigned customer experiences

Checking feedback before and after an experience change

Compare feedback across a redesign by checking question wording, invitation rules, respondent mix and the service outcome.

This is an editorial pen name used by the publication, not a claim of an independently verified personal identity. Original editorial credit: Desk Research.

undefined

Check the two feedback sets are comparable before you read any movement between them. Confirm both cover the same customer task, question and invitation point, and show who could answer in each period. A score that shifts after a redesign may reflect a changed experience, changed respondents or a changed way of asking.

Freeze the comparison question

Suppose a service changes the message a customer sees after asking where an order is. A question about whether the message explained the next step could be asked at that point in both periods. A question about whether the order arrived belongs later and answers a different question.

Keep the exact wording, response options, scale direction and reference period with each result. If the question changes from “How clear was the next step?” to “How satisfied were you with delivery?”, do not join the scores into one trend. Keep both versions and explain the break.

Check who entered each set

Record the task, channels, eligible customers, invitation trigger, dates, invitations and valid responses where known. A post-completion survey will miss people who abandoned the earlier step. If the redesign changes who reaches the survey, post-change responses may describe a different group even when the invitation itself is unchanged.

Compare the mix of relevant case types. A period with mostly routine queries may differ from one with many delayed orders. Keep results for meaningful groups visible when the counts permit; do not present a thin subgroup as a stable population estimate.

Free-text comments can point to a failure worth checking. Their count is a count of submitted comments, not a rate for all customers.

Read the change beside other evidence

Set the two periods side by side using the same method and show response counts. Include “not sure”, missing or inapplicable answers according to a stated rule. Then check service records for the outcome the change was meant to improve. A clearer status message could receive better feedback while orders still arrive late; an accurate delivery outcome could coexist with a confusing message.

If feedback worsens, read relevant comments and cases before naming a cause. If it improves, look for contrary cases and people the invitation failed to reach. Changes in volume, channel mix, fulfilment conditions or another service rule may also explain movement. A before-and-after comparison alone cannot isolate the redesign’s effect.

Feedback Comparison Before and After Experience Change

Survey Question
How clear was the next step?
Reference Period
After receiving status message

State a conclusion the data can bear

If the evidence warrants it, a report could say that respondents invited after the message change rated its clarity more favourably in the later period, while noting any difference in case mix or question placement. It should not extend that finding to every customer without evidence for the wider claim.

End with the next decision: retain the change while checking delivery outcomes, revise wording for a group still struggling, or collect more comparable feedback. Preserve both survey versions and their collection rules so a later reviewer can understand the comparison.

More from Voice of Customer