Moderation and community specialists GGWP have unveiled Pulse, a sentiment analysis tool that “unlocks real-time insights” based on what players are actually saying in game.
The company boasts that the tool turns “previously unseen and unheard” conversations into insights they can use for actionable insights from their communities.
It analyses anonymised text and video chat to yield more honest feedback than players might otherwise feel comfortable sharing on a Discord server, and can cut through the polarised feedback shared on social media more generally.
Thatgamecompany, the studio behind successful live-service game Sky: Children of Light, is among Pulse’s early adopters, with founder Jenova Chen saying the tool helps identify “friction” inside a game so the developers can address it immediately.
In fact, the image below is from that studio, and offers an insight into how the AI-powered tool plugs into games: player mood towards particular parts of a map are flagged in real-time as positive, negative, or neutral, based on text or voice chat.
“If you think about how most studios gather feedback today, it’s a combination of either looking at the data itself, or either going on Reddit or X or Discord,” says GGWP CEO Dennis Fong.
“Part of the challenge of that, of course, is that the folks that actually participate in Reddit or Twitter tend to be quite hyperbolic in nature in terms of what they say and the way that they say it. And the feedback tends to be quite negative. It doesn’t mean that it’s not important, but it represents a very small slice of the community.
“Where does the most honest feedback come from, and how do we feed that back to developers and really get insights and understand what it is that players like and don’t like about their games?” Fong says. “That was really the starting point in some ways.”
The headstart for Pulse came from GGWP already handling pro-active moderation for a number of big publishers, including Krafton, Nexon, and Funcom. The company moderates voice chat in 12 languages, and text chat in 18.
“A lot of developers already use social listening tools to monitor Reddit and Twitter and stuff, to try to gather that feedback into a single location. What they’ve never really had access to meaningfully is in-game chat. But if you actually think about where the most honest feedback comes from, it’s not when you @ someone [asking], ‘hey, what do you think of our game?'”
“The most honest feedback would be when a player is playing your game, and what they’re saying to each other while they’re playing your game.”
Of course, the type of conversation players have in game is not necessarily going to be as coherent as a social media post, either.
“It’s not like Reddit where I’m giving you a three-paragraph response to what I like about a weapon you just released. It’s going to be a much shorter, burst-y type of conversation.”
GGWP’s background in AI-enabled moderation, though, means it’s already used to finding the desired information within conversations.
“You take a step back, part of what makes us the best-in-class from a moderation perspective – because we do work with some of the biggest companies in the world now in games – is we built our models and trained our models on those types of conversations specifically,” Fong explains.
There is something instantly alarming about the idea of in-game conversations being monitored full stop, let alone filtered by AI for specific words – but Fong says its work is GDPR-compliant, and that privacy is incredibly important to the company.
“As part of what we do at GGWP already, we have to be very privacy-compliant. We have to comply with GDPR and PIPA [Personal Information Protection Act] in South Korea and all the different privacy laws, which means that we’re already scrubbing our data, anything that has the potential to trip up data laws – because we have to comply with all of those laws wherever we operate, which is everywhere.”
“We have data deletion processes, so if a user says, ‘I want to delete my data’, there’s an automated process that helps us do that.”
Fong says the company is also deleting the vast majority of voice data automatically because it’s not relevant to their work, with around 10% or less voice chat actually being usable for feedback.
Still, privacy is the subject Fong is asked about the most. “It’s not nefarious in a traditional sense, like we’re trying to listen to what people are saying,” he explains. “We’re really just trying to help developers get a better understanding, and get a direct pipe to the feeling that’s coming from players, in the most honest, unvarnished way possible.”
Not all of GGWP’s clients are currently using Pulse to monitor feedback in this way, but it’s clearly a tool that Fong believes has a lot of potential for wider use.
“So far the feedback’s been good. A handful of our customers are using it today – we have a handful out of the few dozen of our customers that we’ve rolled it out to,” Fong says.
“Part of it is, we want to hear what they think. At a high level, we created this tool because our existing customers asked us to. ‘Hey, you have access to this data already. What can we glean from it?’ And a lot of the work was how you filter out all the noise, because it’s very noisy.”
