After our recent PR monitoring blog post got some extra attention, I thought I ought to make a post about the tool itself, how it works, its limitations and what may be missing at the moment.
This post will be updated as our monitoring evolves, and will serve as a reference to answer questions on the data collection and the meaning of some of the things we share.
TL;DR – The tool is not perfect. Some games are much harder to track than others. We constantly try to improve on all fronts and are very aware of the current weaknesses of the system.
The Media Monitor
That’s how the tool is called internally. It has evolved from the tool set that we used internally to support the work PR team carries out for our clients. Despite the first version being about a year old, a lot of the features have only been developed recently as it is not a core partof our day-to-day work. Having the tool working for that long though has proved very useful in identifying seasonal patterns (or the lack of), but we are still limited in the games we are properly tracking.
So far we have about 6,000 websites in our database (across 28 languages) – the Media Monitor is currently tracking about 3,500 of those websites.
For those websites that are not tracked, it is because of different reasons. Some websites we don’t track because it would be too complicated technically (or rather, we are using the method that allows us to track the most sites at the same time at the moment). Some websites we don’t track because we have a few bugs we need to iron out. And finally, some websites we decided not to track at all because of the nature of their content.
Every single website in the database is qualified – we keep the Alexa ranking, the language it is written in, and give every site a “media type” to allow us to analyse further the results of the work that is done. We have removed from the Media Monitor two types of websites: content farms (websites that have zero original content and generally publish automated reposts other websites’ content) and fansites (website dedicated to one or a few very specific games and don’t treat any others). In the first few months of monitoring we found such sites really skewed the results we were getting and so they had to go.
We currently have more than 950 individual trackers (also referred to as alerts sometimes). Each of these trackers is doing a full text search for key words. We have trackers for a lot of different things at the moment:
- Specific games
- Game franchises
- Hardware devices
- Video game companies
- Weird, ultra specific things that we want to track individually for whatever reason
90% of the trackers work very efficiently. They don’t return many false positives (ie: articles that use key words but on an unrelated subject) and they don’t miss much of the relevant articles.
Some trackers require us to be a little clever however. Let’s explain using the example of Thief.
Most games that are problematic are using one commonly used word. In this, on top of looking for the word Thief, we added a number of other words that need to be present in order to create a positive hit. It looks a bit like this (slightly simplified):
You NEED the word “thief” and you NEED one of the following term: xbox OR playstation OR steam
That tracker was performing very well until E3, then Uncharted 4: A Thief’s End was announced and threw up a problem.
We had to recreate the tracker for Thief as follows (this is a simplified version of it again):
You NEED the word “thief”, you NEED one of the following term: xbox OR playstation OR steam, you CANNOT have the word “uncharted”.
This is a chart showing the before and after of that tracker:
We keep a very close eye on the behaviour of the trackers, trying to keep them as relevant as possible and avoiding false positive at the same time.
Quality versus Quantity
Currently (and for the foreseeable future), the Media Monitor is only providing quantitative data. That in itself has already been very helpful to us. We see trends for what are the news the most likely to be picked up, it allows to get some benchmark on the different countries we are working on (we do PR across all of Europe, not an easy task).
We are also very aware of the questions we cannot answer with this data – but we would rather have half the answers than none at all.
Please let me know if you have questions, I will use them to update this post and clarify the notions presented here.