Evaluations

[pkelleter]

hey,
first of all thanks for your work, this project seems promising to me, since so far A/B testing in SPAs often turns out to be painful.

anyhow, i do not see any kind of evaluation tools coming with your solution. are you planning to build such things - or do you have any proposal on how to tackle this issue otherwise?

thing is - A/B testing without a "result" of the test (what performed better? A or B?) is pretty much pointless in the first place :D

thanks in advance, keep up the good work 👍

[adrdilauro]

Hi @pkelleter

Usually evaluation of results is responsibility of data analysts, not developers: my side of the job is to make sure that the AB tests is implemented correctly and cleanly, and that no false positives or analysis failures can be blamed on how the test is implemented.

So, I am not going to develop an evaluation tool, also because there are plenty out there (Google Optimize, Optimizely, etc).

However, I have been exposed a lot to evaluation tools in the past, so I can give some recommendations. Another person happened to ask me the same question via email a couple of months ago, I'll paste here my reply; I hope it's helpful for you, if you have more specific questions please feel free to ask.

February mail with instructions about how to setup UA+GTM

The easiest way to measure the test is to define a new GTM variable, that reads from the cookie you have set, and then pass it to GA as custom dimension. You are then able to do all the analytics you want from inside GA.

Unfortunately I cannot give you screenshots because I don't have a personal GTM account, and the company I am contracting for at the moment doesn't have one. I found this article that explains something similar to what you need to do:

https://www.simoahava.com/analytics/simple-split-test-google-tag-manager/

You have to start reading from the point in which he starts talking about the custom dimension. The difference between what the guy of the article does, and what you need to do, is that you have to feed the custom dimension with the GTM variable. (by the way Simo Ahava's site is really good, I suggest you to read often his articles, he is nearly the only person who writes professionally about analytics for developers).

So, once you have your custom dimension set up, you can fire all the events you need in GA, and measure them by splitting the data with the custom dimension. You'll create your reports this way.

Then, when it's time to take the decision about which version is the winner, you'll need to read your reports. I guess you already have some sort of software to analyse the data: if not, remember that the main thing you need to know is when to stop the test, in order to have consistent data. If you have access to tools like Google Optimize, they are the best for this task. I don't know if at the moment there is a free version of Google Optimize.

[adrdilauro]

This article can also be useful: https://medium.com/quick-code/set-up-analytics-on-an-angular-app-via-google-tag-manager-5c5b31e6f41?source=activity---post_recommended

[martinvano]

@adrdilauro I think what OP was originally trying to ask how can we report the results from angular-ab-tests to any of existing ab testing reporting platforms? The current documentation doesn't mention any reporting whatsoever, so it's a little unclear whether there's any support for this in the library or it should be handled individually

[adrdilauro]

@martinvano

No, there is no support for reporting data in the library: the library is only meant to be used for the implementation of the test, as I said, the analytics part of it has to be handled by one of the available platforms.

The module angular-ab-tests stores the version in a cookie; it's responsibility of the analytics platform to record the value of such a cookie, and associate to any event that is relevant to the desired conversion.

On Google Tag Manager + Universal Analytics + Google Optimize, it's possible (and quite simple) to read the value of the cookie and pick up the analysis from there. I don't think the pattern followed by GTM + UA + Google Optimize is something crazy, I believe it quite reasonable that other platforms should let you do a similar thing. I only know products from the Google world though, I wouldn't swear about other platforms.

I hope this explanation is clear. Since so many people are asking about it, I should probably be more explicit in the documentation.

If you have any further doubts, again, I cannot suggest anything better than following the blog of Simo Ahava, he is the best expert I know on the analytics side.