Do you think it is likely that we will start to see Large Language Models integrated in to major video games? If so, are there some examples within gaming already?
I think using LLMs to provide the dialog for NPCs in a RPG is a use case that’s just begging to happen. Ie townsfolk that don’t just give the same few replies every time, and who react to things you’ve done in the past beyond just whatever prewritten options the developer thought of.
They did this in Free City, and one of the bank tellers became indistinguishable from a player.
What is Free City? I can’t figure it out from what I found in my searches.
I think the PC vs. console divide is relevant here. I’m not sure how advanced text entry on consoles is these days, but I imagine PCs have the advantage with keyboards. Maybe if they use voice recognition on the consoles? But AAA games usually target both, and if interacting with the model is clunky for a big chunk of your market then the big developers might not use the technology.
Of course, indie devs that only target PC can go wild.Not sure about Switch, but PS5/Xbox support mouse and keyboard inputs if a game is designed for them.
Voice recognition seems more likely to me, though. PS5 already has an advantage there because every Dualsense controller has a mic in it.
Whisper from OpenAI is pretty solid for speech recognition (at least English), and it is small enough to deploy on mobile devices. If I recall correctly, both PS and Xbox controllers have mics built-in, so input device is covered.
AI could also generate dialogie options for players, though. It could operate as traditional dialogue, with AI generating responses and possible doalogue paths ahead of time so you get a “normal” experience that just changes every time
I think so. New GPUs will be able to handle AI models running locally before too long. I think this will be used for NPC behavior as a replacement for procedural quest / dialogue generation. I have seen a lot of mobile games leveraging this but they don’t seem very good yet. Models need to be trained more specifically for each game I think
You can run language models on consumer cards right now. The only thing is depending on the size of the model and the amount of VRAM on your card, you might not be able to do much else.
Yeah there is likely still a ways to go before we can run high end modern games plus a local model, but newer nvidia cards are pretty crazy. It’s probably closer than I think
