Another new week and I’m back with another post of my series of CXL reviews. This is the first post of 2021. I’m glad that I found CXL in 2020. This year will be all about learning some more and executing what I have learned. Sharing the notes of my 5th week feels awesome as definitely a tough journey but it’s worth the efforts.
Let’s see what I learned this week.
Research and testing part — 2
Technical analysis:
The first step towards any growth process is technical analysis. In technical analysis, one needs to check if:
- Their website is working flawlessly with every single browser and device.
- To figure this out go to our web analytics tool and compare your key revenue per visitor or conversion rate. For example, chrome has a conversion rate of 7% but firefox has 3%. Then check why firefox is getting low conversion rates.
It’s all about asking the following 5 questions:
- Is anything on the website broken?
- Where is that thing?
- On which browser?
- On which devices?
- Which pages are slow?
Heuristic analysis
There are four factors on a website that matters the most:
- Relevance: If I’m googling “x” and I click on a website, do I immediately get that this is “x” if it says anything else then the website is not relevant.
- Clarity: It means do I understand specifically and clearly what this website is about. ( Who this is for, why should I buy here and not some other place. )
- Motivation: What is being done on the page to increase my motivation to take action.
- Friction: Things that make it difficult to take action like long forms etc.
Every single page should have one desired action on that page. Segmentation is a good goal for a homepage. Example if it’s e-commerce: Category page to product page to add to cart to checkout.
The desired behavioral changes happen when 3 things converge: High motivation, ability, and trigger.
Digital Analytics
- Where are the leaks?
- Which segments?
- What are users doing?
- What actions correlate with higher conversions?
See where the leaks are in your funnel. See how many different ways are there to achieve your goals. Track every micro-interaction happening on your website.
Mouse Tracking and Form Analytics:
- Where do they click?
- How far down do they scroll?
- Differences between the devices?
- Session replays
These help in understanding what interests the user more and to see how much the user is scrolling. For long sales pages, have only one background color. Some tools record the user when they’re filing a form, these tools can help you see when and why exactly the user is abandoning the form.
Qualitative surveys:
- Buyer Groups
- Which problem are they solving?
- How are they deciding?
- What’s holding them back
- What else do they want to know?
Take open surveys, Let them write the answers as we give options in surveys according to our assumptions which can be wrong.
User Testing
- What’s difficult to understand?
- What’s difficult to do?
- What goes wrong?
In user testing, completely ignore what people are saying. What people say and what they do are different worlds.
Research and Testing Part: 3
- Conduct research: The more insight you have the more successful your CRO projects will be. Both qualitative and quantitative research is crucial. Google analytics and stakeholder interviews are the CROs best friend.
- Build Hypotheses: Most CRO projects fail because the underlying optimization hypothesis is fundamentally flawed or non-existent. The better your hypothesis, the better the results. The better the insights the better the hypotheses the better the results.
- Create Treatment: Once you’ve built a solid optimization hypothesis based on insight from conversion research, you’re ready to create the treatment that you want to test live on your website or landing page.
- Test Treatment: Testing treatments live on the target audience is the only way to see if your hypothesis is right. A/B testing must follow the scientific method otherwise data will be useless.
- Analyze results: Analyze your test data and learning from your experiments is what it’s all about. If your tests don’t give you insights, there’s no point in testing at all. Every test should provide deeper insights and lead to new hypotheses.
- Follow-up experiments: CRO is an ongoing process. The real value of testing is in refining your hypothesis and constantly getting deeper insights. That’s how to move the needle and win the long term.
After finding all the possible problems in a website further segment them into these categories.
- Instrument
- Just do it
- Test / Hypothesize
- Investigate
Follow the PXL prioritization framework by CXL to see what hypothesis to test first.
An AB test is finished when:
- We have enough sample size.
- Multiple business cycles are done. As the conversion rate is different on each day.
- Statistical significance is reached.
Top 12 Testing Mistakes
- Precious time wasted on stupid tests.
- You think you know what will work.
- You copy other people’s tests.
- Your sample size is too low.
- Your test page has very little traffic.
- Your test doesn’t run long enough. ( Run for 28 days, go shorter only if there is too much traffic. )
- You don’t test a full week at a time. Don’t test for more than 28 days.
- Test data is not sent to third party analytics.
- You give up after your first hypothesis fails.
- You are not aware of the validity effect. Check if your test is working on every browser and device. Make sure that the text doesn’t have any bugs.
- You are ignoring small gains.
- You are not running tests all the time.
These were some of the major mistakes committed before doing a test so the next time you are going to run tests make sure you are not doing any of these mistakes.
Metrics of a testing program:
- Number of variants tested
- Win rate
- Average uplift per successful experiment
So this was all that I learned this week. Hope you are having a good time reading and learning from them.
See you next week:)
