Matthew Tighe’s company Do Games has spent the best part of the last decade porting games to consoles, including titles like Cult of the Lamb and Art of Rally. His new venture, Gameworks, aims to take all of the knowledge gleaned from that effort and apply it to making developers’ lives a little easier.
“Pretty much every game had very similar problems,” Tighe says, “and most of them were caused not so much by technical challenges, although they could be related, it was more about the fact that the timescales were compressed, and … decisions were made somewhere in the process that were based on incomplete information or rushed.”
Tighe explains that a typical problem might be a bug found too late in development for it to be addressed properly, because the solution would involve extensive refactoring of an internal system. And whereas a game’s tutorial and early stages tend to be checked and playtested to within an inch of their lives, later stages often don’t get the same kind of attention, meaning bugs only become apparent right at the last moment.
The solution, he thinks, is to test throughout the development process itself. Tighe explains that the Gameworks platform analyses games for common visual quality performance issues and certification issues – such as using incorrect terms for joypads or controller buttons – then produces a report for the user, which might flag things like resolution or frame-rate dips. And Tighe says they’re also looking into adding things to do with memory usage, identifying where consoles like the Switch might struggle with portions of a PC game if it was ported to Nintendo’s console.
“Basically, the system we developed during our porting days for automated testing has been taken and sort of flipped around,” explains Tighe. “So instead of being focused mainly on development, it’s now focused on QA and production.”
Every time a developer runs the game, he says, the system records all of the video, inputs, and various other metrics. “We collect all this data, and then our system processes it using a combination of techniques,” he says. “It uses two or three different types of AI: So we use LLMs, we use some custom models, and we use some computer vision models. They create more data feeding into the system, and then using heuristics [in] combination with an AI model, it’s able to use that data to identify where the issues are and highlight them.”
In addition, he says, “any people that are working with the system can also add their issues in: they can clip a certain part of a video timeline for a test session.”
There are plans for the system to be integrated with tools like Jira and Trello, although at the moment Gameworks has its own internal issue system, which somewhat resembles Trello cards. “Imagine you have a section of the game where there’s a frame rate drop,” says Tighe. “You create an issue for that, and then someone could look at the cards, they’d see the details, they’d see the video, and it’s also captured the last save point. So any tester that’s using this system, they can hit a button and it will immediately start the game again and get you close to that point. Or you could share the video with your colleagues and they could comment on it.”
“What’s interesting, though, is say you have a section of the game where the frame rate drops a few times: at what point do you consider those things the same bug or separate bugs? One of the things that we are doing with an LLM is having it process the overall data that’s been captured and try to apply the logic that a human would in creating issues.” In short, the system might recognise that several frame rate drops in a row are all related to one issue rather than being separate problems, and would group them together accordingly.
“In general, it’s trying to use AI to do all this kind of stuff that people don’t like doing – all of the nitty gritty stuff – so that we can spend more time trying to make the games fun,” Tighe concludes. And importantly, he says, this isn’t about AI replacing people’s jobs.
“I think a lot of people are taking data and trying to feed it into LLMs or other models they’ve generated, and the goal is to replace something a person does, or do it better. That wasn’t our goal, because our goal started before AI really could actually affect it. Our goal is to make production easier, to try and help more games be a bit more successful, because their quality is higher, they have fewer problems, and the development’s more efficient. And AI is an enabler to do that.”
PerfCop, or Performance Copilot, is another tool that’s working along similar lines to Gameworks, trying to surface and highlight technical problems during the development process. For Ken Noland, co-founder of PerfCop maker AI Guys, AI has been a long-term fascination.
“I’ve been building games for about 25 years,” he explains. “Started out as a generalist, kind of worked a little bit in audio, and then I spent 15 years as a network programmer. And then over the last seven or eight years, I’ve been transitioning more and more into the AI side, specifically the generative AI side, because it honestly fascinated me. So I kind of got my start in generative AI before ChatGPT was actually a real thing, because of Black & White.” That 2001 Lionhead game used pioneering AI learning techniques, and the game’s AI programmer Richard Evans would later go on to work at Google DeepMind, a company founded by Lionhead alumnus Demis Hassabis.
Despite his longstanding interest in generative AI, Noland distances himself from the AI evangelists. “When the AI hype cycle began, there were people saying, like, ‘Oh, artificial general intelligence is going to be coming in the next year or next six months’, or ‘We’re all going to be out of a job in six months’ time’. And so I felt that there was kind of a need for somebody with a more realistic, more grounded approach to step in and say like, ‘No, AI is not going to be taking your job’. … AI is great for some use cases, AI is not so great for other use cases, and we tend to bill ourselves as AI realists.”
Although he doesn’t regard AI as a panacea, he does see it being very handy for things like pre-screening for code reviews. “So just making sure the code that’s being submitted actually conforms to a set of rules that are established by a technical director or development director, and ensuring that it’s well documented and that another engineer could look at the code and understand it,” he says.
“I still recommend people do actual code reviews, where they review every single line of code, and they walk it through with the developer to make sure that they understand the logic. So it’s not using generative AI to replace anyone, it’s literally just using generative AI to spot the forehead-slapping moments of, ‘Oh yeah, I forgot a semi-colon at the end of this line’.”
PerfCop, he explains, is a performance analysis tool that’s based around statistical analysis with a “thin veneer of generative AI.” The tool is designed for Unreal Engine, but Noland says it goes beyond existing performance analysis products for Unreal.
“Tools like GameBench and the Unreal Automation Test Framework are valuable, but they serve fundamentally different roles,” he says. “GameBench is focused on high-level telemetry (frame rate, thermals, device comparisons) primarily for mobile performance benchmarking. It tells you that a problem exists and on which devices, but it doesn’t help engineers understand why.”
“Unreal’s Automation Test Framework serves a completely different purpose. It’s designed to ensure the game is functioning correctly by running tests, catching regressions, and verifying that systems aren’t breaking. As part of that process, it can generate performance data and trace files, but it doesn’t analyse them.”
“That’s where PerfCop comes in. It takes the performance data produced by tools like the Automation Test Framework and performs deep statistical analysis to identify abnormal execution patterns, isolate the most impactful issues, and break them down to the function or scope level.”
The AI part only comes in at the end of the process, generating a report based on deterministic analysis. “It communicates structured findings in a way that accelerates debugging and decision-making,” says Noland. Indeed, based on feedback from early trials, AI Guys reckons that PerfCop saves around 15–70 hours of time per month per developer.
“Feedback so far has been phenomenal,” enthuses Noland. “Most of the developers who have used it say that it saves them from having to do generic reports. It saves them from having to spend six to eight hours looking over the trace file, and analysing it, and comparing previous versions, and finding out where things have changed. It automatically does all of that for you.”
Noland adds that PerfCop also has a chat interface called Sherlock, which developers can use to ask questions about the results. “It can call into the original data set and extract the actual performance metrics that are embedded in the trace file,” he explains. “You can actually ask very specific questions, like, ‘What’s going on in frame 23 with the pathfinding taking up 20% of the frame time?'”
PerfCop is typically set to run through various levels nightly or weekly, and then produce a performance report. That allows developers to “address any performance concerns week to week rather than waiting to the end,” says Noland. Like Gameworks, it’s a tool that can enable developers to keep track of problems as and when they emerge, rather than discovering some nasty surprises – like catastrophic frame-rate drops in a late-game section – during the frantic weeks ahead of a game’s launch.
More generally, Noland sees this kind of back-end application as being the future of generative AI in the games industry, rather than the more headline-grabbing uses we’ve seen. “There’s a large pushback [against] what players are calling AI slop, and I do understand why that term is getting thrown around quite a bit. However, as a development tool, that’s really where I see its strengths. As a development tool, as with PerfCop, it can help you identify your performance bottlenecks, and there are other areas that we are also looking into that could potentially change how the structure of the game operates. That’s really where I see generative AI being incredibly useful.”
