Reading Time: 4 minutes

Q: Where is java not slang for coffee, and Ajax not a brand of kitchen cleanser?

A: In engineering, of course.

Cro Metrics recently won 3 Optie awards from Optimizely for our work with 99Designs and Trulia.  We didn’t win them by making simple changes like altering button text, we won them through thoughtful design and difficult-to-implement Javascript from our engineers.

Allocating your engineers to A/B test implementation means they will have less time to spend on core feature development.  However, you’d like to do both since there is significant ROI in both activities. What’s the solution?

Cro Metrics has developed a skill set to implement more complicated tests on websites, so your company can move faster, test more interesting ideas, and save your internal engineering resources for projects the marketing/product development team wants to prioritize – you CAN “have your cake and eat it too”.

Here are a few of the more complex problems we often tackle, and how we go about it:

  1. Changes to the page after it’s been loaded.
    • In JavaScript-powered A/B testing, there is often a need to “listen” for document changes and react to them without degrading performance, and without changing underlying website code.
    • SetInterval is the typical approach, but it can still cause a visible flash even for very small intervals — and the smaller the interval, the more frequently code needs to execute.
    • MutationObserver events allow instant reaction to document changes and less code execution by eliminating the need to check at intervals, but only relatively new browsers support it: most notably, only IE 11+.
    • Our solution: create an open source custom plugin that uses modern browser standards with a setInterval fallback.
      Common use cases:
    • Modal dialogs (e.g. “popups”)
    • AJAX navigation
    • Multi-page forms
    • Asynchronous Javascript (ag AJAX)
    • “Lazily” loaded assets
  2. Consistent User experience across the site, particularly when trying to target users coming from a specific traffic source.
    • Optimizely evaluates audiences per page view. A key issue with this is that experiments targeting uses based on how they arrive (e.g. referrers, Adwords campaigns, search, etc.) will only work on the landing page.

    Our solution: set cookies containing the same information used to determine a user’s arrival method (query string and HTML referrer), then use those cookie values as audience conditions.

  3. Navigation using newer Javascript frameworks such as React, Angular, etc.
    • Some frameworks use Javascript to pull up content and replace the entire current page rather than load a new page. This has advantages for page load time but is terrible for third-party Javascript. Optimizely often can’t detect the page changing, so it can’t activate experiments as needed.

    Our solution: create an experiment we call a “bootstrap” that detects a site visitor and runs continuously, detecting page change events and activating experiments manually.

  4. Javascript styling can be inefficient.
    • When making extensive changes to a page the code can get long and the commands can start piling up. This means slower execution and potentially more errors.
    • One solution, adding a <style> tag to the page containing CSS, produces better results but is difficult to work with and edit in a coding environment designed for Javascript.
    • Optimizely provides a way to add CSS to a page, but it gets added to every variation, including the original, making it not immediately accessible for making changes to only a certain variation of a test. For example:
      • Variation 2 of experiment # 123 calls for, among other things, all <a> elements to have their color set to dark blue, but to turn a dark cyan when the mouse hovers over them.
      • Typical Javascript: 
        $('a').css('color','#000099');
$('a').mouseenter(function() {
$(this).css('color','#009999');
}).mouseleave(function() {
$(this).css('color','#000099');});
// A lot more Javascript

        or:

        $('head').append('<style type="text/css">'+
'.123-2 a { color: #000099; }'+
'.123-2 a:hover { color: #009999; }'+
'/* A lot more CSS */'+
'</style>');
// A lot more Javascript
      • Optimized Javascript: 
        $('body').addClass('x123-2');
      • Optimized experiment CSS: 
        .123-2 a { color: #000099; }
.123-2 a:hover { color: #009999; }
/* A lot more CSS */
      • Result: In the end, dramatic and extensive changes can be made to the page with only two commands; one to add the class to the <body> tag and one to add the Optimizely <style> tag to the page.
      • This method can be used to restyle, reposition, or even hide elements that aren’t on the page when it initially loads.

    Our solution: Add a class to the page’s <body> tag that matches the current experiment and variation, then add CSS to the experiment CSS that looks for those classes.

  5. Bucketing users into the same experiment on different domains.
    • A recent challenge had us running a multi-page experiment on a flow that went across two different top-level domains. We found that a user could, for example, be randomly bucketed into Variation 1 on one domain and Variation 2 on the other.
    • This would create an inconsistent user experience and potentially skew results.
    • The reason for this is because Optimizely uses cookies to remember which variation a user was “bucketed” into for any given experiment. These cookies are only readable on the domain they were created on.

    Our solution: We send visitors between domains with a query string indicating what variation they were bucketed into on the first domain. On the second domain, we used Optimizely’s Conditional Activation feature to run some logic before activating the experiment. We pulled the query parameter and used Optimizely’s Javascript API to manually bucket the user into the right variation on the second domain, so when the experiment activated the cookie value was already in place, and users remained in the variation they were meant to view.

These approaches to common problems have allowed us to minimize the need for our clients to spend engineering cycles implementing tests, as well as hard-coding winning variations.

To learn more about how Cro Metrics can liberate your engineers for critical business-boosting activities, check us out at Cro Metrics.com or give us a call at 415-505-7625, we’re happy to help.

CSS Engineering Optimizely Performance