Limitations and things to be aware of

Right now, AI nodes are in beta.

Please note the following limitations:

1. We have not yet released next best page and next best node > page routing approaches
2. Only some lookback windows and frequencies are available
3. We have not released segmentation modes other than traffic source
4. Please do not throw critical production traffic at AI nodes yet
5. Do not programmatically create AI nodes. They create significant reporting load and we need time to analyse and optimise this. 

What are AI nodes?

AI nodes function like a normal rotator node but are able to intelligently route traffic to different destinations, once enough data is available.

They work by running automated reports on your data and creating routing profiles based on conversion rate or revenue per view data, using Bayesian Inference statistical methods.

They are extremely customisable and make it possible to dynamically route your traffic to the best nodes or pages on autopilot.

How do I use them?

Within the funnel builder you can create an AI node with the right-click menu or from the node palette:

Once you have created one, treat it like a rotator. Connect incoming visitors to it, then connect out to your destinations:

The first tab is exactly like a rotator:

Then, you can go to the AI node settings and advanced settings tab to configure how these AI nodes will process your data over time.

These nodes offer the most customisable intelligent routing of any performance marketing tracker, so there are quite a few settings you can use -- what works best depends on the volume and conversion behaviour of your traffic. 

To start, we recommend just using the defaults

Basic AI node settings

Here we can control how our AI nodes function. The most basic settings are what mode to use, how frequently it updates reports on your data (to determine routing percentages), and what past period it analyses:

There are two modes:

• Conversion rate
• Revenue per view

Conversion rate mode

When using conversion rate mode, the % routing to each destination will adapt based on the probability of each one being the best performer. 

This is similar to what you will get from a split-test calculator like this one. Although, we have a lot more customisation and logic behind the scenes.

In this mode, you must pick the specific event to optimise toward. This is helpful when you may have early events like leads, but where revenue data is less available or slower (e.g. you want to optimise toward leads/signups first).

Revenue per view mode

The revenue per view mode uses total revenue (across all events) and is the default. 

It will be the most effective in general situations where you have a reasonable number of conversions and revenue coming through.

With this mode, we optimise based on the monetary performance of each destination. This is not as simple as with the calculator above, so we have our own custom logic. 

We do consider event counts even when using revenue per view -- we need to consider how many events create the revenue data (more events = more confidence), to prevent runaway winners, and to consider some of your other settings.

So on revenue per view mode, you should pick the event type that has the biggest impact on your revenue -- i.e. your primary revenue creating event.

Lookback window and frequency

As for lookback window and frequency, these determine

• the report window we use for calculations and
• how frequently the reports are refreshed.

For lookback window, we recommend choosing a window within which you have a good snapshot of overall performance, ideally with 10+ total conversions.

For slower moving campaigns with less data, choose a longer lookback window. Likewise for campaigns where day-to-day or hour-to-hour performance may vary a lot.

Be careful with data periods of <24 hours if your campaign performance per variant may swing a lot depending on the hours of the day.

For frequency, this is just how frequently the report is refreshed. The value must be equal to or less than your lookback window.

For example if you set the lookback window to 6 hours, you must have a frequency of 6 hours or less -- otherwise the rolling window would have gaps in the data.

Higher frequency makes more sense when you have higher volume and more data coming in, so can make decisions faster, and don't care about the overall "performance per day" of each variant -- just what is converting best right now.

With highly algorithmic sources, rapid update frequencies may have more benefit.

Please note higher frequencies may become available later but frequency available will be tied to the addon plan purchased, as higher frequencies have scaling demand on our system.

Segmentation

Segmentation affects how reports are broken into parts to further optimise your traffic.

By default, there is no segmentation.

The only current option is traffic source, which will split your reports by traffic source, creating optimised routing profiles for each.

Do this if you're running multiple sources to the same funnel and you think each traffic source may perform differently.

Note that when segmented we still do a report without segmentation to get a default profile for the funnel itself. If the current visitor is from some segment (e.g. traffic source X), but we don't yet have a routing profile for that -- due to a report not being run yet or a lack of data, we will use the funnel-level default. If that's not available, we will use the rotator's settings.

Routing goal

Right now, the only available goal is "next best node", which will consider the data of each connected node -- so the nodes themselves are the "variants".

However, this will just route to the next nodes, which will not consider the individual performance of pages inside those nodes.

In the near future we will release the "next best page" mode will do the analysis based on all the pages that exist inside the next connected nodes, routing directly to those pages, ignoring the internal rotation of the page groups.

This is the best mode for most users, hence why it will become the default. It's just a bit more complex to get right.

However, note that this makes sense if all your connected nodes are page groups. If you have a non-page group node connected (e.g. a condition), it cannot skip past that node to a page in calculations.

In this scenario, another new mode will also be launched soon -- "best next node > best page" mode.

That mode, best next node > best page, will do two routing calculations. It will first look at the node performance and choose which one to route to based on this. Then, if a page group, it will consider the performance inside that group and route accordingly.

This mode may not result in the absolute best page destination being routed to, but allows for combinations of different node types in your funnel path, and will result in more reliable early routing if you have a lot of pages and not a lot of data.

Consider if you had 3 offer groups but each one had 10 pages. This is 30 destination page variants and you may need a lot of data before the results are statistically meaningful. 

With next best node > best page, your early routing would be base on average node performance first (which only needs 3 variants of data), with the next best page being considered afterwards -- which would use equal routings to all pages when not enough data exists.

Please note that once an AI node is running, the default routings in the rotator and subsequent page groups will basically be ignored -- it will set percentages based on the exploration weight, minimum percentage traffic and percentage caps. The default routings only get used early on when no report has been run.

As you can see, there are many options here to optimise the way these nodes route traffic through your funnel!

Advanced AI node settings

These settings are for advanced users who really want to tweak how their AI nodes function.

There's no need to touch these -- you can just use the defaults. Only adjust if you understand them and feel the need to.

We want to make these nodes simple to use, but have a lot of power under the hood to adapt to your needs.

Sensitivity factor

This determines who readily the system will allocate percentage routing to a higher performing variant (and vice versa, less to a lower performance variant).

You can think of it as a "fudge factor" that tries to pull all the values back to an average of equal amounts each.

At the lowest sensitivity, more data and stastical confidence is required for it to drive more traffic to the higher peformers.

At the highest sensitivity, even at low conversion counts (e.g. 5-10 conversions across all nodes), it may rapidly allocate >90% of traffic to the variant with the best performance.

Be careful adjusting this as it can cause runaway performance on a single variant. We do not recommend adjusting it to higher sensitivity unless you have >50 conversions across all your variants in your lookback window.

Exploration weight

This affects how much extra weight is given to variants that have very low data in your current and previous report.

Right now that is <200 visits and <3 conversions.

This is present so that when you add new variants to a page group (or connect new nodes), it senses they have no data and tries to give some weight to them.

As they surpass 200 visits or 3+ conversions, the exploration weight effect will taper off quickly.

This can be used in combination with the "minimum percentage per variant" optional setting, but is an internal default behaviour that can not be turned off.

It will also result in some see-sawing where lower performant variants will start getting allocated no traffic, then several reports later they may be allocated weight again.

This does help with preventing false negatives in the routing, but also means that you should entirely remove pages/nodes over time that you can confirm are worse performers.

Conversion sensitivity threshold

This sets a minimum conversion count for the report (total events over ALL variants), under which the system considers it to be a "low data" situation.

Below this conversion count, the system will try to prevent runaway variants and pull everything closer back to an average/equal amount per variant.

You can use this to tell the system not to be confident of the variant performance unless a minimum number of conversions is present.

It will still bias towards higher performers in all situations, just not as strongly if this conversion threshold is not exceeded.

If you have traffic that has high volume and a lot of events, you can set this higher to make the system gather more data.

If you have very low conversion counts, keep this at the default or consider lowering it.

Prior strength

This affects how much statistical meaning is placed on the previous report.

Our processing considers the current report AND the previous window.

This "prior" is used in the calculation as well. If a variant has similar performance in the prior and current report, there is more confidence in it's overall performance.

By doing this we prevent large swings in the percentage weight from one report to the next.

Setting this value higher will dampen this swinging in the data further. We would recommend leaving it on the default vaue.

If you have a situation where your variant performance can, for whatever reason, change between reports quite significantly, you may want to lower this to make the current window favoured more.

Cap variant percentage

This feature is optional and quite simple -- it will set a limit on the % weight that can go to any variant.

This can prevent runaway spend and help with fair testing.

Minimum percentage per variant

Similar to the setting above, this will try to send a minimum % to every variant available (depending on the routing mode).

This can ensure fair testing of all variants.

If the mimimum % is greater than the average, it will use the average. For example if you set a minimum of 10% but you have 15 variant destinations, it will use 1/15 weight to each of them.

As with the exploration weight feature, this helps all variants get traffic including those with low performance, those recently added, and those with zero data. Because of this, it's important that you regularly audit your AI node/funnel performance and remove any variants that you no longer want to test.

FAQ

What frequency and lookback window should I use?

Firstly, the available options may depend on the AI node plan you pay for. Greater frequencies and segmentation may require a higher tier plan.

In general, choose a window where you believe you will have adequate data to assess performance, and a frequency based on how much traffic you send > how fast you can get that data.

If you are sending 100 clicks a day from Google Ads and expect a few conversions, you can likely select last 7 days for the report and update every 24 hours. This way it will assess performance every day and look back over a significant period.

If you are sending pop traffic and get tens of thousands of views an hour, and dozens of hourly conversions, you can update more frequently and can use a shorter window e.g. 24 hours.

If you run on highly algorithmic sources where you get a lot of incoming visitors/conversions AND want to optimise quite rapdily, you may want to use our fastest report frequency and a short lookback window (e.g. hourly and look back on the past 3-6 hours).

It really depends on the situation, and what plan may be applicable. The more rapid frequencies and segmentation options will require higher plans, given the much higher loads they can put on our reporting systems.

What segmentation options are available?

Right now we have default (none) and by traffic source.

In the future we plan to add campaign ID, country, and possibly URL tracking fields but with a limit based on the top segments (e.g. by ad ID but limited to the top 100 values by visit count).

We are still developing this and need to test with more real-world data. These tighter segmentations will require a lot more traffic to work well, will require a higher plan, and will have some limitations to prevent abnormal load on our systems.

If I don't have data yet, what will the AI node do?

In this case, the node will just use the rotator settings. It's still a fully functional rotator and acts like one. It's just that if data is available, it will create and use optimised routing profiles.

Can I see the routing % data?

Not right now, but we plan to add this to the UI in the future

If I segment by something and there's not enough data yet, what will it do?

If no active segment data exists (no report has run yet or not enough data confidence), the node will use the default (non-segmented) report, and if that's not available, then it will use the default rotator settings.