A game is not a good game unless it’s played fairly, and the creators of Call of Duty have taken a machine learning approach to make sure cheating behaviour in the game is quickly picked up and acted on.
Speaking at the recent Predictive Analytics World conference in San Francisco, Activision’s director of analytic services, Josh Hemann, spoke about how different classification algorithms are helping the game creator effectively referee millions of game matches played every day.
“If the community of players perceives that cheating is rampant, it just hurts the whole experience,” Hemann said. “It’s equivalent to being in an online setting where there are a lot of mean people, mean comments and jerkish behaviour. Cheating is one such behaviour in an online community where people can be anonymous and it can really hurt the health of the community.”
The aim is for the machine learning algorithms to detect certain behaviours, such as boosting behaviour, then issue the cheater a warning.
“You’re not really playing the game, you’re just doing something for your own benefit. And it actually affects the entire game,” Hemann continued.
“You have to be able to identify it. That’s where algorithms come into play for us because we can’t have referees watching every match.”
Algorithms Activision has found effective in accurately detecting this behaviour include gradient boosting machine, logistic regression and decision trees. Hemann also built different models for different game types within Call of Duty as they have their own unique player behaviour patterns and game objectives.
In addition, Hemann engineered features for the different models, looking at things like skill cadence and spatial variability.
“After every player plays a match, they have a numerical score assigned to them between 0 and 1 that comes out of this model. That value is basically 1 if the model is dead sure that you’re cheating, you’re doing this boosting behaviour,” he explained.
“With all these players and their match records having been scored, we can find those that have a really high boosting score. Then we can focus on players who have multiple events where they are doing this over and over again.
“Those are the players we can feed into downstream customer support processes so these players can be warned, contacted or in extreme cases they might have their stats reset – something to enforce that they shouldn’t be doing this.”
Hemann said one of the difficult things when working with games is that behaviours are not necessarily straightforward and easy to detect right away, which led him to take a machine learning approach.
“It’s not anything that you’d necessarily even see as an extreme value in the data, nothing would jump out at you. That’s why statistical models are needed to really detect the behaviour, which can be kind of nuanced and hard to see sometimes,” he said.
Follow Rebecca Merrett on Twitter: @Rebecca_Merrett
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.