Sections in this category

Understanding Impact

  • Updated

“Impact” is a central concept in Sisu. This measure indicates to what degree various Subgroups in your data can affect important KPIs (Metrics) across your organization. This article describes how Impact is calculated and communicated through Sisu.

 

Related Articles:

 

Understanding Impact

If you simply want to learn which Subgroups are changing in their metric value, then you do not need impact. But to answer which Subgroups are driving your metric, you need to look at impact since it accounts for (a) the change in the metric value, (b) the size of the Subgroup, and (c) the change in its size between the two time periods

Sisu’s Analyses provide you with insights into Facts (also referred to as factors or Subgroups of data) that impact the performance of your chosen Metric

These Facts have varying degrees of impact on the overall performance of the Metric, and the impact can be either positive (the situation represented by the fact statistically “improves” the metric’s performance) or negative (that situation represented by the fact statistically “drags down” the metric’s performance).

When calculating the Top Drivers list, impact is used as one of many inputs in determining the list of Top Drivers. The Top Drivers or All Subgroups tab ranks Facts by Impact.

Impact is displayed as a positive or negative value, and the value of the impact (i.e. +3.5 or -9.5) represents the amount that the metric is affected by the Subgroup. For example, if a Subgroup’s impact is +3.5, it means that the presence of this Subgroup “lifts” the metric’s performance by 3.5 units. “Units” are the unit of measurement for the metric, (e.g., order dollar amount or customer satisfaction rate).

One more important thing to understand about impact:  it’s different for each Analysis Type and Metric Type combination. Let’s back up a bit…

Metrics can be either Numerical or Categorical in nature, and different Aggregation Methods (calculations) apply to each. Metrics are:

Metric Column from your data + Aggregation Method used for the Analysis

Impact is determined differently depending on the type of metric (numerical or categorical) and the Aggregation Method used for the Analysis, as later in this article.

 

How Sisu Displays Impact

The Top Drivers subtab on any Analysis displays Facts that impact the Metric’s performance, and the Impact on average column displays the impact value for each fact, as shown here:


Displaying_Impact.png

Which direction of the impact is considered positive -- or "good" -- depends on the goal of the metric. If the goal is to increase the metric, then positive impact has a green arrow and negative impact a red arrow. However, if the goal is to decrease the metric, then negative impact has a green arrow and positive impact has a red arrow.

In the example above, the Analysis is looking at the Average Order Value for all orders placed digitally. We can see from the facts’ impact values shown that:

  • Orders that only contain one item are negatively impacting these average order values
  • Orders in New York and Los Angeles are positively impacting these average order values
  • Orders of Energy Shots are positively impacting these average order values

 

How Sisu Determines Impact

A fact's impact represents how much that Subgroup changed the performance of the Subgroup. For General Performance Analyses, it is the presence of the Subgroup that drives change. For Comparison Analyses (Time or Group), it is how that Subgroup changed between the two time periods/groups that drive change in the Metric.

Let’s look at a few examples to help understand what this means. In the Analysis above, which is for a General Performance Analysis, orders that contain only one item (ORDER_ITEM_COUNT = 1) have a negative impact of 13.6 on average order value. If those orders were not present in the data set, average order value would be 13.6 higher. (Since this Analysis is looking at Average Order Value as the Metric, the average order value would $13.60 higher if orders containing only one item did not exist. The number value “unit” of the impact is the same “unit” used for the Metric (i.e., dollars, in this case).

 

Impact_GP_example.png

 

To calculate this impact, Sisu “removes” the rows associated with this Subgroup, calculates the average order value without that fact being present, then compares that value with the average order value with the fact. The difference determines the impact value and its direction (negative or positive).

Now let’s consider a Time Comparison Analysis. With this type of Analysis, Sisu calculates the impact on the change in the Metric’s performance between the two time periods defined. In the example below, which analyses the change in Average Order Value metric between two time periods, orders in New York Stores increased the average order value by 0.63, and orders in Los Angeles stores decreased it by 0.58.


Impact_TC_example.png

Impact works the same way for Group Comparison Analyses.

The remainder of this article describes the calculations Sisu uses to determine impact for all of the possible Analysis Type and Metric Type combinations.

 

Impact Calculations Overview

The following chart provides a quick overview of the calculations used to determine the impact of Subgroups in Sisu. The sections that follow provide greater detail for each situation.

Forumlas_Table.png

Each of these Impact Calculations are explained in the sections that follow.

 

 

General Performance Impact

 

Metric Type:  Average

For General Performance, Average Metric Type, Sisu's impact shows the contribution of the Subgroup to the overall value of the Metric. This is determined by the following formula:

GP_Numerical_Average.png

Let’s look at an example:

GP_Numerical_Average_-_New.png

How to interpret impact:  In the “ORDER_CHANNEL = digital_mobile” Subgroup, the Impact of +0.71 means that without the digital_mobile Subgroup, the Average Order Value would have been 0.71 lower (i.e., it would have been 29.7 - 0.71 = 28.99).



Metric Type:  Sum

For General Performance, Sum Metric Type, Sisu's impact shows the contribution of the Subgroup to the overall value of the Metric. This is determined by the following formula:

GP_Numerical_Sum.png

Let’s look at an example:

GP_Numerical_Sum.png

How to interpret impact:  The “ORDER_CHANNEL = digital_mobile” Subgroup contributed 3.6M (shown in the Impact column) of the 8.9M value of the overall metric. Without this Subgroup, the overall Metric would have been lower by 3.6M (i.e., it would have been 8.9M - 3.6M = 5.3M).



Metric Type:  Count (Numerical)

For General Performance, Count Metric Type on Numerical Column, Sisu's impact shows the contribution of the Subgroup to the overall value of the Metric. This is determined by the following formula:

GP_Numerical_Count.png

Let’s look at an example:

GP_Numerical_Count.png

How to interpret impact:  The first Subgroup (“ORDER_CHANNEL = digital_mobile”) contributed 117K (shown in the Impact column) of the 300.9K value of the overall metric. Without this Subgroup, the overall Metric would have been lower by 117K (i.e., it would have been 300.9K - 117K = 183.9K).



Metric Type:  Rate

For General Performance, Rate Metric Type, Sisu's impact shows the contribution of the Subgroup to the overall value of the Metric. This is determined by the following formula:

GP_Categorical_Rate.png

Let’s look at an example:

GP_Categorical_Count.png

How to interpret impact:  In the first Subgroup (“ORDER_CHANNEL = digital_mobile”), the Impact of -0.069% means that without the digital_mobile Subgroup, the Rate would have been 0.069% lower (i.e., it would have been 51.5% -0.069% = 51.4%).



Metric Type:  Count (Categorical)

Coming Soon




Time/Group Comparison Impact

 

Metric Type:  Average

For Comparison Analysis, Average Metric Type, Sisu's impact shows the contribution of the Subgroup to the change of overall value of the Metric between two time periods/groups.

Comparison_Numerical_Average.png

Let’s look at an example:

CA_Numerical_Average.png

How to interpret impact:  Changes in the “ORDER_CHANNEL = in_store” Subgroup between the two time periods contributed -0.43 of the overall change in the Metric of -1.1 (28.7 - 29.8). Had this Subgroup not seen changes between the two time periods, the change in the overall Metric would have been higher by 0.43.



Metric Type:  Sum

For Comparison Analysis, Sum Metric Type, Sisu's impact shows the contribution of the Subgroup to the change of overall value of the Metric between two time periods/groups.

Comparison_Numerical_Sum.png

Let’s look at an example:

CA_Numerical_Sum.png

How to interpret impact:  Changes in the “ORDER_CHANNEL = digital_mobile” Subgroup between the two time periods contributed -742.9K of the overall change in the Metric of -2M (2.6M - 4.6M). Had this Subgroup not seen changes between the two time periods, the change in the overall Metric would have been higher by 742.9K



Metric Type:  Count (Numerical)

For Comparison Analysis, Count Metric Type for Numerical Column, Sisu's impact shows the contribution of the Subgroup to the change of overall value of the Metric between two time periods/groups.

Comparison_Numerical_Count.png

Let’s look at an example.

CA_Numerical_Count.png

How to interpret impact:  Changes in the “ORDER_CHANNEL = digital_mobile” Subgroup between the two time periods contributed -23.7 of the overall change in the Metric of -60.8K (92K - 152.8K). Had this Subgroup not seen changes between the two time periods, the change in the overall Metric would have been higher by 23.7K.



Metric Type:  Rate

For Comparison Analysis, Rate Metric Type, Sisu's impact shows the contribution of the Subgroup to the change of overall value of the Metric between two time periods/groups.

Comparison_Categorical_Rate.png

Let’s look at an example.

CA_Categorical_Rate.png

How to interpret impact:  Changes in the “ORDER_CHANNEL = in_store” Subgroup between the two time periods contributed +0.043% of the overall change in the Metric of -0.1% (51.4% - 51.5%). Had this Subgroup not seen changes between the two time periods, the change in the overall Metric would have been lower by 0.043%.



Metric Type:  Count (Categorical)

For Comparison Analysis, Count Metric Type for Categorical Column, Sisu's impact shows the contribution of the Subgroup to the change of overall value of the Metric between two time periods/groups.

Comparison_Categorical_Count.png

Let’s look at an example.

CA_Categorical_Count.png

How to interpret impact:  Changes in the “ORDER_CHANNEL = digital_mobile” Subgroup between the two time periods contributed -12.2K of the overall change in the Metric of -31.3 (47.3K - 78.6K). Had this Subgroup not seen changes between the two time periods, the change in the overall Metric would have been higher by 12.2K.