Putting some spin on taking a deep breath - Rhinomed and The Turbine

Rhinomed is an ASX listed company who have recently released a product called ‘The Turbine’.  It’s basically a nasal dilator, and is marketed as a product to improve sports performance.

With my interest in endurance sports such as running and triathlon, I’ve been following their research and announcements closely. A previous invention was marketed with the same benefits, ‘Breathe Right Nasal Strips’.  Launched in the early 1990s, they were promoted as a tool to improve sports performance. Research showed no benefit while wearing them for sport. Despite this, GSK purchased the company for $566 million (no typo, that’s $566,000,000 US!) in October 2006. Interestingly, the Breathe Right website does not make any mention of benefits to sports performance and markets the strips as a sleeping aid.

The Turbine (from www.rhinomed.com.au)

The Turbine (from www.rhinomed.com.au)

So, back to Rhinomed. Their device is pictured below and is available from the website theturbine.com for $24.95 for a pack of 3. Each unit is recommended to be worn on 3 occasions only.

In June 2013, in preparation for a global release of the product, Rhinomed (then known as Consegna, their previous company name) released their first batch of research supporting their product.  This was a trial of "38 elite athletes and trainers" and recording their subjective feeling of how the unit affected their performance in running, rowing, aerobic gym classes or weight training

Their claims included:

-->    60%  of the respondants in the trial felt less fatigues during training when using the BreatheAssist Sport" (this was the name of the device before 'The Turbine'        (This means that 40% felt the same or MORE fatigued!)

--> "Close to half felt their breathing had returned to normal more quickly than usual when using the BreatheAssist Sport." (Perhaps the worst claim or spin I’ve seen – this means that over half felt their breathing returned to normal at the SAME rate or WORSE when wearing the device. SO it had an average NEGATIVE effect on recovery!)

Hardly ‘compelling results’ as the media release trumpeted in it’s headline.

Rhinomed’s main claim through 2013 was that their product delivered ‘38% more air’.  This was a claim that was frustrating to hear! For a few reasons:

{cke_protected_1}1.      {cke_protected_2}If their product was so effective, WHY didn’t they run a trial with some appropriate objective date.  Athletes don’t care about more air. They care about a) faster finishing times, b) lower effort (e.g. heart rate) at a given intensity. 

{cke_protected_3}2.      {cke_protected_4}38% more air through the nose probably doesn’t matter too much when measuring airflow through the nose. If you just open your mouth to breathe you’ll have a dramatic effect on the airflow.  

3. Finally (and most important), haemoglobin (the molecule that carries oxygen) is generally saturated at close to or at 100% during exercise. 38% more air won't make a difference to how well your body can transport oxygen (and neither will 380% more!)

For anyone with a scientific background, the research findings released to this point were at best, confusing.  Here was a product with a huge potential market reach and they kept releasing a very strange set of data to support it.

I sent some tweets to CEO of Rhinomed, Michael Johnson, letting him know this (in a concise format on Twitter of course). He let me know that there was some further research underway with some more objective data. “Fantastic!” I thought.

Fast forward to May 13, and the long awaited trial results are released.  The top of the presser announced: “Turbine delivers 6.5% increase in power”.

So, for some background, power is the gold standard of measuring improvement in cycling. Power is measured in watts. The measure of your power output has a direct and strong relationship to your performance on a bike.

As an example, an elite cyclist can produce about 6 Watts of power per kg of body weight for an hour, so about 360 Watts for a 60kg cyclist. A recreational cyclist on Beach road might be 2.5 Watts per kilogram, and a top 10% age group triathlete might be 4 Watts per kilogram.  So a 6.5% increase in power represents quite a bit.  For me for example, this might take me from 300 Watts in an hour, us to almost 320. Quite a big improvement when a full season of training may result in less improvement than this.


But, keep reading in the press release. There were 9 cyclists participating in the trial. The 6.5% increase in power represented THE BEST SIX of these only! In research, this is NOT the done thing. A vital part of conducting best practice and methodologically sound research is to include the results of ALL patients/participants who start the trial.  However here, the trial included the best results only.

It’s kind of like having a headline: “Average lottery entrant wins $2 million!” – acknowledging one lottery entrant while ignoring the other 500,000 who won nothing

To their credit, later in the press release Rhinomed did acknowledge the average improvement across ALL participants was 3.7%. A bit of simple maths fills in how the ‘mystery’ three riders performed:

Best 6 riders: 6.5% increase

Total 9 riders: 3.7% increase

Therefore, worst 3 riders would have had an average performance DROP of 1.9%. Yes – 3 of the 9 cyclists actually performed WORSE with the Turbine product in place.  Rhinomed appears to have been misleading with the release of the results.


However, does an average improvement of 3.7% still represent a significant benefit?  We need more data from Rhinomed to know about this.  Generally in research, results are presented with confidence intervals.  The default values for confidence intervals gives us a 95% chance that the true benefit of a product (such as the Turbine) is within the values given, and a 5% chance that the results are outside this range.

For example, if the results were presented as “3.7% improvement (95% confidence interval 3.5-3.9%)”, this would be a great result as we could be QUITE sure that there was a 3.5-3.9% improvement with using this product.

However if the results were presented as “3.7% improvement (95% confidence interval -15% - + 20%)”, well, we would be sure the effect of the product was in this range, but this doesn’t help us a lot as the effect could be very poor (in fact significantly worse) or very good.

Because of the small number of cyclists in the trial (only 9), the confidence intervals would likely be quite wide (or large), which makes it likely that there would be no clear benefit demonstrated from the Turbine.  We like to see the entire confidence interval lie above 0 (e.g. “1.0%-5.2%”, NOT “-2.3%-6.1%) to be confident of a benefit.

In late 2013, Rhinomed brought on Dr Mitch Anderson, professional triathlete and medical doctor, as ‘Sports Medical Advisor’ for the Turbine product. Dr Anderson has a wonderful resume including undergraduate degrees in Physiotherapy, Masters in Exercise Physiology as well as a Post Graduate medical degree. Understanding and interpreting statistics is a vital part of all of these studies and I would have hoped that Dr Anderson would have pushed for publication of transparent data.

(To his credit, Dr Anderson has tweeted regarding a 3.7% average improvement rather than the misleading and factually incorrect 6.5%.  However the press release quotes him as saying the Turbine provides a “very clear benefit”.)

3.7% is more accurate than 6.5% but still doesn't give all the detail.

3.7% is more accurate than 6.5% but still doesn't give all the detail.

 

These details such as confidence intervals are quite simple to calculate using a statistical method known as regression analysis. However you need access to all data to make these calculations. My tweets and direct messages via Twitter to Rhinomed requesting the complete data have been left unanswered.

 

The other benefit promoted in the May 13 press release is “Riders with Turbine travelled 143m further during 40 minute time trial at high intensity”.   Where exactly the figure of 40 minutes comes from is difficult to calculate!  Reading further into the trial, we have the following:

--> “Riders with the Turbine travelled 60 metres further in low intensity endurance tests compared to those without the Turbine across the 8 week program” (paraphrased)

So, just to clarify the testing procedure and results here:

--> There were 4 endurance rides, two with the Turbine and two without

-->  ‘Endurance Exertion’ was given as ’65-75%’ (although 65-75% of what isn’t stated)

--> Riders were encouraged to stay in ‘similar heart rate zones’ for subsequent tests after the first test. How large these zones are (e.g 5 beats, or 10?) isn’t stated. 10 beats can allow for a very large increase in power for example

--> The difference between groups (60 metres) isn’t mentioned as being from the total of the 4 trials, or the average, or something else, so is very difficult to interpret

--> The intensity of this trial appears to be the equivilant of these elite cyclists heading out for an easy spin. Getting home 15 seconds earlier from an easy ride doesn’t really relate back to improved performance in a race.

--> Remember the confidence intervals from above? We need this data to be confident that the improvement is actually significant rather than by chance.

--> Comparing heart rate to power output and taking an analysis of this data would have provide a more powerful insight into the results

The other claim:

--> “Riders travelled 143m further (1.2% improvement) in 10 minute high intensity power tests with the Turbine” (again paraphrased)

Let’s look into this further again

-->  The distance travelled was 12.370km vs 12.227km. This is probably for a total of 20 minutes, which would give an average speed of about 37 km/h

--> The intensity for this trial is “80-90%” (again, “80-90%” of what?). This PROBABLY is equivalent to a hard 1-2 hour race but without more quantification it is difficult to know.

--> This, admittedly is one of the better looking results on the surface, however we need to see the results from all riders before being able to be confident that there is a significant improvement.

--> Again, heart rate data that was collected has not been presented.

 

So, my summary of the press release

--> Rhinomed has been misleading in their presentation of results. An undergraduate student in their first year of studying 'Research Methods' or statistics could pick this up.

--> The trial was not blinded (e.g riders knew whether they were using the Turbine or not).  Not having blinding increases the risk of increase in effect of the device either due to placebo, or performance bias (athletes perform better to help the study obtain favourable results). Regardless of the blinding, encouraging participants with a whiteboard plastered in ‘Turbine Power Test’ isn’t the best way to encourage neutrality in rider performance! (see pic below)

--> The press release acknowledges the lack of blinding, and I admit it is somewhat difficult here (although perhaps a device with the dilating component could have been used?)

--> Any performance benefits presented are very very minor at best, and would likely be shown to be statistically insignificant with the inclusion and appropriate scientific analysis of the data.

--> There are some other obvious flaws in the presention of this research, which I won't list now as I think my point is probably made for today!

Just to clear up any confusion about what test you're doing today...

Just to clear up any confusion about what test you're doing today...


It seems like Rhinomed have ‘rolled the dice’ with this trial, and unfortunately there are no real benefits to be gained with this product. Time will tell and I look forward to seeing any future research. I’m happy to again write discussing the results of the trial if Rhinomed staff forward the full data from the trial to me.

Disclosure: I hold a very small parcel of Rhinomed stock and have since 2011. I originally purchased after seeing their product ‘VibroVein’ which has yet to be released and appears to have been dropped from their portfolio over the last year or two.