Huge downturn in CC reviewer scores (this one's for the nerds)

[hankandteri]

2 hours ago, Cruise till you drop said:

Statistics say the population of 5 to 30% would make it conclusive, which would require a survey a minimum of 6,300 peoples opinions

 

 

Statistics say nothing of the sort. In fact, statistics don't say anything at all, but statisticians do. Your source please, sir or madam. The size of your universe isn't all that relevant to the sample needed to gain insight. Plenty of clinical drug trials involve very small samples, and loads of valid market research efforts are done with relatively tiny sample sizes, depending on the objective. I've conducted a lot of that research myself over several decades. I don't pretend my little exercise is scientific, only that it appears to indicate something. Believe that there's meaning in the numbers or validity to my methodology or don't. It's all good. 

[hankandteri]

1 hour ago, Spif Barwunkel said:

I am not a numbers guy. I realize the necessity and the value of numbers and statistics within certain domains and conditions. In the cruising world when dealing with the human element there are many variables from which a skewed result can occur. Not to say that the reviews and their number ratings are not sincere, but might they be more unscientifically emotional rather than unscientifically thought out, thus more often than not resulting in negative codification and subsequent commentary?  

 

On all of my Celebrity cruises to date I have never been privy, singularly or collectively, to the personal affronts and circumstances that some folks apparently have experienced. As to the occasional disappointment, most certainly we have had a few. Therefore, reviews and downturns in the numbers are of no concern to me, at present. Not to say that can’t change, but it will be on my terms not someone else's.     

You lost me at "I'm not a numbers guy."

[NMTraveller]

11 hours ago, hankandteri said:

This one's for the other nerds on this board. If you don't like numbers, you might want to sit this one out.

 

I wondered how actual cruisers were responding to the changes Celebrity management has implemented since August, and I was pretty sure Celebrity wouldn't share that with me, so I looked at the most recent reviews here on CC. Bottom line is you can see the reaction to the changes instantaneously in the reviewer scores and those new scores are pretty ugly.

 

Celebrity's average all-time rating from Cruise Critic reviewers based on 19,000+ reviews is 4.1 out of 5, so very good. Other mass market lines like Carnival, HAL, Royal and Princess hover around 3.8 or 3.9. NCL and MSC are a bit lower (3.6 and 3.3 respectively). For comparison, Viking, a premium line, is 4.6.

 

I looked at the passenger scores for the last 60 Celebrity reviews posted on CC, (all the reviews from September and the last few posted in August), and the average score for that time period had fallen from the 4.1 baseline to 3.2. That's may not sound like a lot, but it's a brutal drop, and puts X below MSC's overall rating, so mediocre at best. Sixty reviews is a decent sample size, but I didn't have the time or interest to keep going to determine exactly when the dropoff in scores began or what the curve looks like over a larger timeframe.

 

That 3.2 average score is way lower than the overall scores for any other mass market line and on a different planet from the premium and luxury lines! To be fair, I didn't check the recent review scores for any other line to see if they had also dropped precipitously recently. I bet a few others have also had a measurable drop (looking at you NCL, Princess, and Royal).

 

A lot of the CC reviews are posted by first time cruisers and new-to-Celebrity cruisers, so it's not just loyalists reflexively opposed to change contributing to the ugly number.

 

This feedback has to be showing up in their current post cruise surveys, so if anyone in Miami knows basic math, Celebrity must know that their passengers are less than delighted by the recent changes they have made.

 

On a related note, is it time for CC to take a look at all the five star ratings they have assigned to the ships in Celebrity's fleet given the huge disconnect between the official ratings and those of their current passengers?

Hmmm.  3.2???

 

I make it a point to not book anything below a 4.0.  Whether that be restaurants, Amazon products, etc, etc.  I prefer a 4.5 to 4.8.  I remember when Celebrity used to hover around a 4.5.

[hankandteri]

53 minutes ago, NMTraveller said:

Hmmm.  3.2???

 

I make it a point to not book anything below a 4.0.  Whether that be restaurants, Amazon products, etc, etc.  I prefer a 4.5 to 4.8.  I remember when Celebrity used to hover around a 4.5.

I feel the same way when it comes to toasters, dentists, and restaurants, but if you need to see a 4.5 (or even a 4.0) rating for a cruise line on this site in order to book, your options will be limited right now. For every cruise line I looked at, at least a few people were assigning 1s in the most recent reviews. Scores like that strain credulity even in the current environment. A single 1 requires eight 5s to get an average above 4.5 with no other ratings under 5. That’s a daunting task even with a line that’s totally on its game. 

[kr00t0n]

CC reviews are in no way comparable to actual NPS scores.

 

1 country alone has had over 15k NPS scores since the start of August, and CC reviews would be from global users.

[Cruise till you drop]

7 hours ago, hankandteri said:

Statistics say nothing of the sort. In fact, statistics don't say anything at all, but statisticians do. Your source please, sir or madam. The size of your universe isn't all that relevant to the sample needed to gain insight. Plenty of clinical drug trials involve very small samples, and loads of valid market research efforts are done with relatively tiny sample sizes, depending on the objective. I've conducted a lot of that research myself over several decades. I don't pretend my little exercise is scientific, only that it appears to indicate something. Believe that there's meaning in the numbers or validity to my methodology or don't. It's all good. 

A simple google search, you can find many sites that show this range of 5 to 30%
 

I personally googled “% of a population surveyed to be statistically valid”

Edited October 11, 2023 by Cruise till you drop

[hankandteri]

3 hours ago, Cruise till you drop said:

A simple google search, you can find many sites that show this range of 5 to 30%
 

I personally googled “% of a population surveyed to be statistically valid”

Let’s pretend for a minute that our statistics/market research degree awarded by Google really gave this horrifying answer. The thing is my sample size was 100% of the population I was measuring, far above the bizarre 5 to 30% minimum stated.

 

My universe was people who submitted X reviews on CC for all of Sept. and the last part of Aug., and I measured every bit of it.

 

Then I measured that perfect sample against the average of 100% of those who have ever submitted X reviews on CC.

 

The sample you’re demanding isn’t available because your universe could only exist if 100% of X cruisers submitted reviews on CC. You can only measure a population that exists.  You could probably Google that. 
 

So, while my little exercise may not accurately reflect the hypothetical “normal” X cruiser (which was probably your main point until you went down the statistical validity rabbit hole), it perfectly captures the complete cohort I was measuring with 100% accuracy. 

 

You are saying my results aren’t statistically valid until a sufficiently large sample comes from a population that could never exist. I measured the population created by parameters I chose, and I measured every bit of it. 
 

You can argue about whether the measurement means anything or the size of the population measured. Fine, but statistical validity is not an issue. 

[Spif Barwunkel]

10 hours ago, hankandteri said:

You lost me at "I'm not a numbers guy."

Of course, I did. Simply put, Celebrity's numbers drop from 4.1 to 3.2 impresses me not. What does impress me is the service provided to us when cruising with X, not how other people attach numbers to their cruising experience.  

[hankandteri]

2 minutes ago, Spif Barwunkel said:

Of course, I did. Simply put, Celebrity's numbers drop from 4.1 to 3.2 impresses me not. What does impress me is the service provided to us when cruising with X, not how other people attach numbers to their cruising experience.  

Much better!

[wineoclock]

We will be on Beyond in the 20th 🤞in a MCSuite. 

I’ll try and remember to post and to give an accurate account as to what I think. Silhouette In June was a bit of mess service wise, (Aqua) as it was U.K. half term,  so absolutely packed to the rafters, but we still had a great cruise. If we couldn’t get a drink in one bar we just tried another or took back to our balcony. Cafe al bacio was awful but a lot of staff were sick. So excusable 

[hankandteri]

6 hours ago, kr00t0n said:

CC reviews are in no way comparable to actual NPS scores.

 

1 country alone has had over 15k NPS scores since the start of August, and CC reviews would be from global users.

If someone wants to send me the NPS raw data, I’d be happy to take a look at it, but that isn’t going to happen. My gut instinct is that it that it would tell the same tale, but no company would ever share precious proprietary data like that, especially if the story it told was bad. 

Edited October 11, 2023 by hankandteri
Typo

[hankandteri]

9 minutes ago, wineoclock said:

We will be on Beyond in the 20th 🤞in a MCSuite. 

I’ll try and remember to post and to give an accurate account as to what I think. Silhouette In June was a bit of mess service wise, (Aqua) as it was U.K. half term,  so absolutely packed to the rafters, but we still had a great cruise. If we couldn’t get a drink in one bar we just tried another or took back to our balcony. Cafe al bacio was awful but a lot of staff were sick. So excusable 

And we will be aboard Silhouette next week. Like you, I’m hoping to report that we had a wonderful experience when we return, and that the glitches you experienced in June have been addressed. Our cruise is also sold out, so the crew will be pushed. Even if things aren’t perfect (and let’s face it, no cruise is flawless in the best of times), I’m positive we will enjoy it and meet a lot of wonderful people on board. 

[Cruise till you drop]

17 minutes ago, hankandteri said:

If someone wants to send me the NPS raw data, I’d be happy to take a look at it, but that isn’t going to happen. My gut instinct is that it that it would tell the same tale, but no company would ever share precious proprietary data like that, especially if the story it told was bad. 

And my gut tells me that with a larger sample size of people that submitted reviews, that the scores were actually good.  Otherwise Celebrity would look at the results and take action.

 

Signed by

1) Glass Half Full Person

2) Sky is Not falling person

 

 

Edited October 11, 2023 by Cruise till you drop

[DaKahuna]

My apologies to one and all but this was just too good of an opportunity to pass up ....

 

[hankandteri]

1 minute ago, DaKahuna said:

My apologies to one and all but this was just too good of an opportunity to pass up ....

 

Love it. Did you know that 73% of statistics are made up on the spot?

[DaKahuna]

Just now, hankandteri said:

Love it. Did you know that 73% of statistics are made up on the spot?

 

 I always heard it was 85% but then again who really knows. 

 

[RichYak]

1 hour ago, hankandteri said:

My universe was people who submitted X reviews on CC for all of Sept. and the last part of Aug., and I measured every bit of it.

 

Then I measured that perfect sample against the average of 100% of those who have ever submitted X reviews on CC.

This 100%. It is what I wrote (or tried to) upthread, but you did so far better and more clearly than I did.

[hankandteri]

6 minutes ago, Cruise till you drop said:

And my gut tells me that with a larger sample size of people that submitted reviews, that the scores were actually good.  Otherwise Celebrity would look at the results and take action.

 

Signed by

1) Glass Half Full Person

2) Sky is Not falling person

 

 

Which raises the next important statistical topic—confirmation bias—something you see a lot of on all CC boards. 

[Team Stag]

I find post like this extremely interesting.  I do think numbers can point out trends but we also have to look closely at the source for the value of the information.

First, message boards are far from new technology.  Many of us on here are both long time cruisers and people passionate about cruising.  While we may receive some new members, most have been coming to this site for quite a while (a 1st page scan of users show only one from 2022 and the rest were 2017 or earlier).  But if someone new booked a Celebrity cruise, joined Cruise critic and then spent any time on the Celebrity boards; they would be convinced they were not going to have a good time.  I don't think I'm out of line saying this page is highly critical of Celebrity's current product. 

Second, rarely do reviewers give non-emotional reviews. Typically there is something (good or bad) that triggered them to review.  Anthony Bourdain (and many others) have often said - if you have a good experience, you tell 2 people; if you had a bad experience you tell 10.  Well with the internet / social media - your audience can quickly become more than 2 and way more than 10.  I've been on 5 Celebrity cruises the last 2.5 years & not once returned to fill out a trip report.
 

21 hours ago, hankandteri said:

Virgin Voyages has a historical average of 4.1 and their most recent 60 reviews averaged 4.5, so a significant uptick in recent passenger satisfaction. Not every cruise line is disappointing their customers right now.

When I look at data like this & have to say it is probably skewing the results some.  Virgin Voyages wasn't around pre-pandemic and many people sailing with her are getting a first look at their offering / experience.  There are concerning issues on the Celebrity board for example that is constantly getting attention - for example, "they are taking items off the menu / simplifying the menu " - well if you look at VV menus; they have already done that. But if you never were given many options on VV before; you don't notice when they are taken away.  I will also point out, almost all the cruisers I spoke to on my VV cruise, who had previously cruised before, were coming over from Carnival.  It has been a while since I've been on Carnival (so I'm not trying to bash them); but I would guess VV was a step up from Carnival.  If you found a Carnival to be a great product that you return to, then you probably are going to find Virgin amazing.  And that could influence the ratings.

Finally - we don't live in a vacuum.  Things are constantly changing.  Cruising on Celebrity a year from now may be vastly different than today.  Same can be said with any of the other lines.  I get that Virgin is the darling new kid on the block but they have just gone out for additional capital, a new CEO and delayed the launch of ship canceling 5 months of voyages already sold.  Their product also may look completely different in a year.

 

Edited October 11, 2023 by Team Stag

[Lolou127]

A very interesting thread, I was watching a YT vlog yesterday, fairly well know vloggers sailing the inaugural cruise of NCL Viva, their thumbnail said ratings of 2,6 out of 5!! They then went on to rate different aspects of the Viva, here’s where it gets interesting, the parts of the cruise that would be important to me, nice cabin/good service/great food all got great scores, the parts that wouldn’t bother me nightclubs/theaters etc got awful scores, but based on what’s important to me I’d be tempted to book, what I’m longwindedly saying is it’s all subjective and depends on what’s important to an individual 

[Traveling Fools]

@hankandteri Thank You for all the time and effort you have put into this.  It has sparked responses that I find educational and interesting.

 

Given my limited knowledge of statistical terms, I never heard of NPS until today.  I had to do a search to find out what it is and actually learned something new.  Thank you ALL for that.

 

Unless I misread the definition of NPS and is defined as:

 “Net Promoter Score (NPS) is a customer loyalty and satisfaction measurement taken from asking customers how likely they are to recommend your product or service to others on a scale of 0-10”.

 

I also understand it consists of a two-question survey, e.g.

 

1.       On a scale of 10-0, how likely are you to recommend X to someone?

2.      Why? (answered by a written response)

 

Being the simpleton I am in gathering and interpreting statistics, it is my understanding that if the person taking the survey provided a 1-10 answer to Question 1, it could be concluded they would recommend X.  Given the 0-10 scale consists of a fill-in box, the score of all respondents to that question can be rapidly calculated. 

 

However, the answer to Question 2 involves a human who reads the answer and generates one or more ratings based on any comment provided.  That begs the question of what matrix does X use to further rate the written response, i.e., Complain/Praise for food, entertainment, cleanliness, service provide by one or more crew members, etc.  It is my humble opinion, the survey could/should be expanded to include “On a Scale of 10-0 how do rate” questions for all the issues respondents include when answering Question 2.  I think those answers would provide a more germane and useful product.

[RichYak]

I don't recall ever seeing a one question survey from Celebrity. Who's getting these NPS surveys?

[hankandteri]

2 minutes ago, RichYak said:

I don't recall ever seeing a one question survey from Celebrity. Who's getting these NPS surveys?

The NPS is likely derived from a question embedded in the post cruise survey. That’s how I’d do it if I were them. 

[hcat]

We have found the recent X surveys to be much less useful  for  cruiser feedback,   compared to past surveys .  Those were very detailed by discreet areas of services, dining, experiences, entertainment, condition of ship. etc.

 

 Now the surveys are more of  a PR tool. No room for extended feedback.

 

OP started a nice discussion..but it may have run its course!

[hankandteri]

With the mentions of Gary Bembridge, Virgin, and the usefulness of Cruise Critic reviews on this thread, here’s something that combines all three. 
 

Mr. Bembridge just dropped a podcast episode that explains why Virgin, Disney, and Viking are rated so highly on Cruise Critic. He goes on to give some very interesting analysis of Celebrity at the end. His analysis is insightful but doesn’t take into account recent operational changes. 
 

Worth a listen. 
 

https://sites.libsyn.com/49893/the-cruise-lines-everyone-is-raving-about-right-now-and-why-podcast-382