Month: November 2016

I’ve been interested in the recent reports released recently by OmnicoGroup and Picsolve, two market research companies affiliated with Blooloop who publishes news and research related to the theme park and entertainment industry. Both have been publicised pretty widely on Twitter and in the mainstream media, but each explore different aspects of the current and future theme park visitor experience.

OmnicoGroup: The Theme Park Barometer

The offering from OmnicoGroup is the Theme Park Barometer, a set of questions based on another of their surveys. The full report is 15 pages long, and offers the responses to 38 questions based on 677 UK, 684 US and 670 Chinese respondents. The questions cover the full pre, during and post visit experience including what visitors do as well as what they’d like to do. Their main focus is on the online activities of visitors before and during their visit (18 questions).

Picsolve: The Theme Park of the Future

This report was based on research for Dubai Parks and Resorts, and focuses on  photos (7 questions) as well as wearables and cashless technology (8 questions). The whole report is 12 pages, and reports the results of 24 questions based on responses from 500 Dubai residents that had attended a Dubai theme park in the last year. Their questions focus almost entirely on future technology, and it seems their research serves as an exploration of some specific ideas more than a full strategic suggestion.

The results

The two surveys cover eight subjects, with four subjects covered exclusively by one survey or the other. The subjects as I can derive are:

I’ve posted the full set of responses to both surveys by subject, as it makes it a little easier to digest it all. However, despite half the subjects being covered by both surveys, it’s difficult to combine the two for many valuable insights.

Probably the most promising subject to look into here is the During Visit Online block, as it has by far the most coverage. Ignoring the different audiences surveyed, we can see that recommendation systems and information applications are highly desirable for a park, however there is still a quarter unable to connect to the internet (24%). While there is a lot of interest in the industry in selling the park online and building the post-visit experience, the two reports suggest that today’s parks may have a long way to go to satisfy the visitor’s expectations during the visit itself.

Another insight is that while Picsolve is reporting a large number of people expecting a range of advanced technology in the future, the OmnicoGroup research shows that they’re not expecting it in at least the next three years. For example, while around 90% have said they would attend a theme park offering Virtual Reality in the Picsolve report, only 65% expect to see this in the next the three years according to the OmnicoGroup report. Another example is 42% of people saying they want holographic entertainment in queues, but only 20% expected to see holographic assistants in the next three years. Admittedly having holograms in queues that just talk at you isn’t the same as the AI suggested by the term ‘holographic assistant’, but it shows that none of these expectations are for the near future.

Other than the few insights we see by crossing the two reports, there is also some evidence that despite some of their expectations of online services at the park not being met, visitors still want to engage with the park after the experience. However, the low positive response to questions asking about what visitors actually do after the experience suggests that these post-experience needs are not being met either. The same pattern holds for OmnicoGroup’s pre-visit questions, indicating that there is still a lot of low-hanging fruit to grab from visitors before and after their visit. This area should be particularly lucrative for parks considering that serving a lot of the needs visitors indicated they had, such as reserving merchandise (71%), ride times (81%) and VIP experiences (82%) really cost very little to develop and maintain compared to physical features during the visit.

Some criticisms

While I’m really interested in the results of these surveys, there are a couple of things that bug me about them as well.

First is that I don’t know how many people the OmnicoGroup report is based on (edit: since posting this OmnicoGroup contacted me with their respondent numbers – many thanks!). I may have missed this and they have been very responsive and helpful, so I can’t hold this against them.

Second, both reports ask very specific questions, then report the answers as if they were spontaneously offered. It’s a very different thing to ask ‘would you accept temporary tattoos for cashless payments’ and have 91% of people say yes than it is to say ‘91% of people want temporary tattoos for cashless payments’. My point here is not that the questions they asked were wrong or irrelevant, it’s that it is very easy to overclaim using this method. As it stands I know certain things about the theme park audience now, but I don’t know what they might have responded to any other question in the world. Given that the questions of either survey don’t cover the full process of the theme park experience (and don’t claim to), I don’t see how these reports could be used meaningfully for any business, operational or strategic decisions without it being something of a magic ball.

Finally, the Picsolve report is so focussed on specific areas of the theme park experience, I really can’t tell if it’s research or a sales pitch. Most of the images were provided by Dubai Parks and Resorts, and the final pages are dedicated to talking in general about how much the Dubai market is growing. Further to this, I don’t know how many people who live in Dubai would actually attend a Dubai Theme Park, so I’m not sure the population is really that relevant. On the other hand, a lot of what they write in this report is corroborated by the comments in the Themed Entertainment Association reports, so they’re either singing from the same songbook or reading the same reports as me.

What I learned

These reports are highly publicised and look like they took a lot of work to put together. However, something I’m learning is the importance of packaging my findings in a very different way from what Tufte would ask us to do. While statistical visualisations provide a very efficient way of communicating rich information in a small amount of time, it seems like for many people sheets of graphs and numbers is drinking from a firehose.

On the other hand, I’ve also learned that even minor crossover between two surveys can provide really valuable and useful insights, if only at a general level. I may spend more time in future looking through these results to see what else I can find, and I look forward to building up more of a database of this type of research as it’s released.

So what do you think? Have I made the mistake of looking too hard into the results, or have I missed other useful insights?

I’ve now had two attempts at predicting theme park visitor numbers, the first using Holt Winters and the second using Random Forests. Neither one really gave me results I was happy with.

Holt Winters turned out to be a misguided attempt in the first place, because most of its power comes from the seasonality in the data and I am stuck using annual measurements. Given the pathetic performance of this method, I turned to the Data Scientists go-to: Machine Learning.

The Random Forest model I built did a lot better at predicting numbers for a single year, but its predictions didn’t change much from year to year as it didn’t recognise the year of measurement as a reading of time. This meant that the ‘year’ variable was much less important than it should have been.

ARIMA: A new hope

Talking to people I work with (luckily for me I get to hang out with some of the most sophisticated and innovative Data Scientists in the world), they asked why I hadn’t tried ARIMA yet. Given that I have annual data, this method would seem to be the most appropriate and to be honest I just hadn’t thought of it because it had never crossed my path.

So I started looking into the approach, and it doesn’t seem to difficult to implement. Basically you need to at least find three numbers in place of p, d, and q: the order of the autoregressive part of the model (an effect that changes over time), the degree of differencing (the level of ‘integration’ between the other two parameters AFAIK), and the order of the moving average part of the model (the how much the level of error of the model changes over time). You can select these numbers through trial and error, or you can use the auto.arima() function in R that will give you the ‘optimal’ model that produces the least possible error from the data. Each of these parameters actually has a real interpretation, so you can actually base your ‘trial and error’ on some intelligent hypotheses about what the data are doing if you are willing to spend the time deep diving into these parameters. In my case I just went with the grid search approach with the auto.arima()  function, which told me to go with p = 0 , d  = 2 and q = 0.

The results

ARIMA seems to overcome both the lack of frequency in the data as well as the inability of Random Forests to take account of time as a variable. In these results I focus on the newly reinvigorated Universal vs. Disney rivalry in their two main battlegrounds – Florida and Japan.

Here are the ARIMA based predictions for the Florida parks:

Both are definitely improving their performance over time, but as both the Holt-Winters and the Random Forest model predicted – Universal Studios is highly unlikely to catch up to Magic Kingdom in its performance. However, unlike the Holt-Winters model, the ARIMA predictions actually have Universal overtaking Disney well within the realm of possibility. Universal’s upper estimate for 2025 is just over 35 million visitors, while Magic Kingdom’s lower estimate for the same is around 25 million. In an extreme situation, it’s possible that Universal’s visitor numbers will have overtaken Magic Kingdom’s by 2025 if we go with what the ARIMA model tells us.

The story for the Japanese parks looks even better for Universal:

In these cases we see Universal continuing on their record-breaking rise, but things don’t look so good for Tokyo Disneyland. This is really interesting because both are pretty close replicates of their Florida counterparts and both exist in a booming market. For Tokyo Disney to not be seeing at least a predicted increase in visitor numbers, something must be reasonably off. The prediction even shows a good possibility of Tokyo Disneyland beginning to get negative visitor numbers, suggesting the park’s future may be limited.

Things I learned

ARIMA definitely seems to be the way to go with annual data, and if I go further down the prediction route (which is pretty likely to be honest) I’ll probably do so by looking at different ways of playing with this modelling approach. This time I used the grid search approach to finding my model parameters, but I’m pretty suspicious of that, not least because I can see myself stuttering to justify my choices when faced with a large panel of angry executives. “The computer told me so” seems like a pretty weak justification outside of tech companies that have the experience of trusting the computer and things going well. There is clearly a lot of better methods of finding the optimal parameters for the model, and I think it would be worth looking into this.

I’m also starting to build my suspicion that Disney’s days at the top of the theme park heap are numbered. My recent clustering showed the growing power of a new audience that I suspect is largely young people with no children who have found themselves with a little bit of expendable income all of a sudden. On the other hand, Magic Kingdom and Tokyo Disney serve a different market that arguably consists more of older visitors whose children have now grown up and don’t see the fun in attending theme parks themselves.

Future things

I’ve read about hybrid or ensemble models pretty commonly, which sounds like a useful approach. The basic idea is that you make predictions from multiple models and this produces better results than any individual model on its own. Given how terrible my previous two models have been I don’t think this would help much, but it’s possible that combining different ARIMA models of different groupings could produce better results than a single overall model. Rob Hyndman has written about such approaches recently, but has largely focussed on different ways of doing this with seasonal effects rather than overall predictions.

I also want to learn a lot more about how the ARIMA model parameters affect the final predictions, and how I can add spatial or organisational information to the predictions to make them a little more realistic. For example, I could use the ARIMA predictions for the years where I have observed numbers as input to a machine learning model, then use the future ARIMA predictions in the test data as well.

Do you think my predictions are getting more or less believable over time? What other ideas could I try to get more information out of my data? Is Universal going to be the new ruler of theme parks, throwing us into a brave new unmapped world of a young and wealthy market, or can Disney innovate fast enough to retain their post for another generation to come?  Looking forward to hearing your comments.

I had a go recently at running a K-means clustering on the theme parks in the Themed Entertainment Associationreports by their opening dates and locations. This was pretty interesting in the end, and I was able to come up with a pretty nice story of how the parks all fell together.

But it made me wonder – what would it look like (and what would it mean!) if I did the same with visitor numbers?

 Competing for different audiences

Using the elbow method I described in my previous post, I again found that three or six clusters would be useful to describe my population.

Just like last time, I probably also could defend a choice of eight or even ten clusters, but I really don’t want to be bothered describing that many groups. Joking aside, there is a limit to how many groups you can usefully produce from any cluster analysis – it’s not useful if it just adds complication.

But here’s the issue I ran into immediately:

It moves clusters over the years! I shouldn’t really be surprised – it shows that these theme parks are changing the markets they attract as they add new attractions to the mix. Remember, in this exercise I’m describing audiences as observed by the parks they visit. In my interpretation of these results I assuming that audiences don’t change over time, but their image of the various theme parks around the world do change. Let’s look at the clusters:

Cluster 1: Magic Kingdom Crew

These are the audiences that love the Disney brand and are loyal to their prestige offerings. If they’re going to a park, it’s a Disney park.

Cluster 2: Local Visitors

These parks are servicing local visitors from the domestic market.

Cluster 3: The new audience

This is an audience that has only emerged recently and offering more profits, with those parks gaining their attention reaping the rewards, as seen by the membership of very successful parks in recent years.

Cluster 4: The traditionalists

This group is defined by the type of visitor that attends Tivoli Gardens. Maybe they are more conservative than other theme park audiences, and see theme parks as a place primarily for children.

Cluster 5: Asian boom market

This audience seems to be associated with the new wave of visitors from the Asian boom, as seen by the recent attention to Asian parks like Nagashima Spa Land.

Cluster 6: Family visitors

These all seem like parks where you’d take your family for a visit, so that seems to be a likely feature of this cluster.

I tried a couple of other methods- the last cluster for each park and the most frequent cluster for each park, but these really were even less informative than what I reproduced here. In the first case the clusters didn’t look much different and didn’t really change interpretation. This is probably because my interpretation relies on what I’ve learned about each of these parks, which is based on very recent information. In the second case, I reduced the number of clusters, but many of these were a single park (damn Tivoli Gardens and it’s outlier features!)

Lessons learned

This work was sloppy as anything – I really put very little faith in my interpretation. I learned here that a clustering is only as good as the data you give it, and in the next iteration I will probably try and combine the data from my previous post (some limited ‘park characteristics’) to see how that changes things. I expect the parks won’t move around between the clusters so much if I add that data, as audiences are much more localised than I’m giving them credit for.

I also learned that a simple interpretation of the data can still leave you riddled with doubt when it comes to the subjective aspects of the analysis. I have said that I am clustering ‘audience types’ here by observing how many people went to each ‘type’ of park. But I can’t really say that’s fair – just because two parks have similar numbers of visitors doesn’t imply that those are the same visitors. Intuitively it would say the opposite! I think adding in the location, owner and other information like the types of rides they have (scraping wikiDB in a future article!) would really help this.

Future stuff

Other than the couple of things I just mentioned, I’d love to start looking at the attractions different parks have and classifying them that way. Once I have the attraction data I could look at tying this to my visitor numbers or ownership data to see if I can determine which type of new attractions are most popular for visitors, or determine which attractions certain owners like the most. In addition, I can’t say I really know what these parks were like over the last ten years, nor what a lot of them are like now. Perhaps understanding more about the parks themselves would give some idea as to the types of audiences these clusters describe.

What do you think? Am I pulling stories out of thin air, or is there something to this method? Do you think the other parks in Cluster 3 will see the same success as Islands of Adventure and Universal Studios Japan have indicated they will see? I’d love to hear your thoughts.

Today Universal Studios Japan released a report showing that they had received a record number of visitors last month. The news led me to wonder – was this new record the result of Universal Studios’ meteoric rise as of late, or was it more a symptom of the renewed interest in Asian theme parks in the last few years?

Pulling apart the causes of things with multivariate regression

One of the most basic tools in the Data Scientist toolkit is multivariate regression. Not only is this a useful model in its own right, but I’ve also used its output as a component of other models in the past.  Basically it looks at how much the change in each predictor explains the change in the outcome and gives each variable a weighting. It only works when you have linear data, but people tend to use it as a starting point for pretty much every question with a bunch of predictors and a continuous outcome.

Is the Universal Studios Japan record because it is Universal, or because it’s in Asia?

To answer this question I ran a multivariate regression on  annual park visitor numbers using dummy variables indicating whether the park was Universal owned, and whether it was in Asia. After a decent amount of messing around in ggplot, I managed to produce these two plots:

In these two plots we can see that the Universal parks are catching up to the non-Universal parks, while the Asian parks still aren’t keeping pace with the non-Asian parks. So far this is looking good for the Universal annual report!

This is confirmed by the regression model, the results of which are pasted below:

In this we can see that firstly, only Universal ownership has a significant effect in the model. But you can also see the Estimate of the effect is negative, which is confusing until you control for time, which is the year*universal row of the table.  We can see here that for each consecutive year, we expect a Universal park to gain 234512 more visitors than a non-Universal park. On the other hand, we’d only expect and Asian park to have 31866 more visitors than a non-Asian park for each consecutive year over the dataset. This suggests that being a Universal Park is far more responsible for Universal Studios Japan’s record visitor numbers than it’s location. However, the model fit for this is really bad : .02 , which suggests I’m doing worse than stabbing in the dark in reality.

Lessons learned

The main thing I learned is that it’s really complicated to get you head around interpreting multivariate regression. Despite it being one of the things you learn in first year statistics, and something I’ve taught multiple times, it still boggles the brain to work in many dimensions of data.

The second thing I learned is that I need to learn more about the business structure of the theme park industry to be able to provide valuable insights based on models from the right variables. Having such a terrible model fit usually says there’s something major I’ve forgotten, so getting a bit more knowledgable about how things are done in these areas would give me an idea of the variables I need to add to increase my accuracy.

Future things to do

The first thing to do here would be to increase my dataset with more parks and more variables – I think even after a small number of posts I’m starting to hit the wall with what I can do analytically.

Second thing I want to try is to go back to the Random Forest model I made that seemed to be predicting things pretty well. I should interrogate that model to get the importance of the variables (a pretty trivial task in R), which would confirm or deny that ownership is more important than being in Asia.

What do you think? Are my results believable? Is this truly the result of the excellent strategic and marketing work done by Universal in recent years, or is it just luck that they’re in the right place at the right time? One thing is certain: the theme park world is changing players, and between Universal’s charge to the top and the ominous growth of the Chinese megaparks, Disney is going to have a run for its money in the next few years.