.jpeg)
The classic situation: A redesign goes live. Three days later, conversions are up 12%. Everyone is happy. But was it the redesign? Or was it the retargeting campaign that started at the same time? Or the seasonal demand in February?
Without a clean methodology, every answer is a gut feeling. With a structured change impact analysis, you get figures instead.
A change impact analysis (CIA) is not a complicated model. The principle is simple: You mark a point in time the moment when a change went live and measure how defined metrics developed before and after.
The decisive factor is controlled comparative period. Not “last month vs. this month,” but the exact period before the event, compared with the exact same length of time after that. This is how you eliminate seasonality and general trends as much as possible.
Typical questions that you answer with this:
In bchic Analytics, Change Impact Analysis is a separate area under Business intelligence. The idea behind it: Instead of manually comparing time periods, you enter a change event and the dashboard does the work.
Step 1: Create a change event
The more precise the point in time, the sharper the separation in the data. Anyone who has installed a deployment at 2:32 PM should not enter exactly that “sometime on Tuesday.”
Step 2: Read the impact card
As soon as the event is created, bchic automatically generates a scorecard. This shows:
That's the difference between 'looks good' and 'provably better.' Learn more → Feature Page · Technical Documentation
bchic offers two measurement methods that are suitable for different questions.
Interval (phase comparison) Look exactly at the phase from the start of the change to the next change event. Ideal if you have several deployments every month and want to know how version A performs against version B in isolation without the next release diluting the result.
Cumulative (overall effect) continuously adds up the effect since the start time. Useful if you want to measure the absolute yield of a campaign: How many additional conversions did this campaign start bring us over the entire duration?
So that you don't have to start from scratch every time, there are ready-made templates in bchic:
The interval template automatically creates recurring markers e.g. weekly or monthly. Useful for teams that do regular performance reviews and don't want to add events manually every Tuesday.
The campaign template is particularly useful: It analyses the historical UTM data from the last year and automatically recognizes campaign starts. You simply select which campaigns you want to analyze from a list and the system creates the impact markers. In this way, you can also retroactively evaluate whether a campaign from three months ago really made a difference.
An underrated feature: The Goal comparison. This allows two different conversion goals to be juxtaposed, i.e. real A/B testing directly in the change impact analysis, without a separate experiment tool.
Here's how it goes:
After a change event, you can immediately see whether variant A or B has benefited more.
Did the redesign improve user engagement?
Create change event with category design. As a metric, “Avg. Select visit time” or “engagement rate.” A stable positive value over several days after launch not a one-time outlier is a reliable signal.
Is the paid campaign worth it?
Campaign start as marketing-Mark an event. Set measurement method to “Cumulative”, select “Visitor” metric. Compare the total value (absolute number of additional visitors) with ad spending this is how you get a rough CPA without third-party integration.
Monitoring performance after deployment
After every release, a Technical-Create an event. Watch bounce rate and views. A sudden increase in the bounce rate right after the event is often an early indicator of technical issues 404 pages, missing assets, or broken redirects before customer support starts ringing.
We are the only German analytics tool that enables this analysis cookieless and without consent banners. This means that the database is complete, not just the 50-60% of visitors who have agreed to a banner. A change impact analysis on incomplete data is like an A/B test with half the sample. And because bchic is built GDPR-first, no data protection officer needs to release the BI functions first. Just use.
If you connect bchic with an AI assistant via MCP integration, you can create change events directly from the chat without opening the dashboard. “Register a deployment from today at 14:00, category Technical, title Homepage Redesign v2" and bchic creates the marker automatically.
Combined with the query function, the result is a complete workflow: create an event, wait a few days, then ask “What did the redesign bring to the conversion rate?” all without changing tools.
