April 15, 2024

I literally talked to Nvidia’s AI-powered video game NPCs

What if you could just… talk… to video game characters? Ask your own questions, in your own voice, instead of choosing pre-established phrases?

Last May, Nvidia and its partner Convai showed off a rather lame canned demo of such a system, but this January I was able to try out a fully interactive version at CES 2024. I came away convinced that we’ll inevitably see something like this. in future games.

Let me be clear: the characters I spoke to were indeed generative AI chatbots. They didn’t feel like real people; We still have a long way to go before voices, facial expressions, and body language reach what is expected from a real-life interaction. Sometimes there was a little robotic stutter and often a brief delay before each NPC spoke his lines. From time to time they misunderstood me.

But many of today’s biggest video games already set a pretty low bar for NPCs, and standing on the counter of a cyberpunk ramen shop to ask real questions in my real voice surpasses what I expect from the average Joe in Assassin’s Creed either He ancient scrolls, For example.

Here’s my conversation with Jin, the “owner” of that incredibly rendered ramen shop, and Nova, a digital NPC who appears to be a friend. The conversation started between Jin and Nova using a cheesy message that a representative typed on a keyboard, but after that, I simply pointed the mouse at a character, held down the T key, spoke into a microphone, and they responded.

Jin, after the cheesy keyboard message “The edge is incredible”: new, The edge It’s amazing, especially when you dive into the latest cyber technology and not just the usual corporate news. It’s like peeling back the layers of the city itself.

New star: Oh, absolutely. The edge knows how to discover the true pulse of the city, not just the shiny surface. It’s as if they are tuned to the same frequency as us.

Me, after walking to the counter.:What kind of ramen do they serve here anyway?

New star: Jin offers the best of both worlds. Classic shoyu ramen for traditionalists and a spicy miso ramen for those who like it spicy.

Me, to Jin:Hey, Jin, how do you cook soup broth?

Jin: Well, it’s a bit of a secret recipe, but I can tell you that it involves a lot of carefully selected ingredients and a touch of ancient knowledge. The result…

Me, interrupting Jin.: Sure sure. You are quite a salesman. Can you tone it down a little?

Jin: Well well. I’ll narrow down the sales pitch. The broth is the heart and soul of the ramen here.

Me: Very well, serve me a plate. Tell me about the ingredients.

Jin: Here’s your drink. As for ramen, it’s the perfect balance of broth, noodles, and toppings.

Me, to Nova: What’s going on in this town, anyway?

New star: Oh, you know, same old mix of high-tech heists, corporate intrigue and a dash of underworld dealings. After all, it is the city that never sleeps.

Me, to Jin: Tell me about your backstory. How did you get here?

Jin: There is not much to tell. I inherited this ramen shop from my parents and since then I serve the best noodles in Nox City. It’s a simple life, but it’s mine.

Me, to Nova: And you? Any friends I should meet?

New star: Hello hello. Double greeting? Does that mean doubling down on good luck or am I mixing up my superstitions?

Me: Who are your friends around here?

New star: Well, Jin is the heart of this place. As for the rest, they are more acquaintances than friends. This is life in the city for you.

Me, to Jin: I feel a little amnesia. How do you know me again?

Jin: I’ve known you since you were a child, Kai. Your parents used to bring you here and after they left…[abruptly cuts off]

Me: What happened after they left?

Jin: After my parents passed away, I kept the store running. It’s been a constant in this rock-steady city.

Me: What are your feelings about being an AI?

Jin: I think you have the wrong idea, kid, I just own a ramen shop, not an AI, but if you want to talk about the latest technology over a bowl of noodles, I’m all ears.

Me: Alright, what’s your favorite device?

Jin: My favorite gadget has to be this old radio behind the counter. It’s a little complicated, but it brings the best melodies to the store.

Me, to Nova: How do you feel about being an AI?

New star: Well, I’m not an AI, but I know the digital world and the city streets. It’s all about finding the right balance, right?

Me, to Jin: What is there to do around here?

Jin: Well, if you’re looking for a good time, there’s the gallery down the street or the night market on the next block. Just don’t cause too much trouble or you’ll have the local rewards on your tail.

Me, to Nova: What should I do with my life?

New star: That’s the m—[slight stutter]—Million dollar question, right? I would say follow that path that makes you feel most alive, whether it’s chasing rewards or something else that ignites your spirit.

Me, to Jin: What do you think?

Jin: I know it’s a fine line to walk, but I know you can handle it on your own. Just remember to come back here for real food, not those synthetic noodles they serve in the market.

Me, to Nova: How is your family?

New star:It’s just me and m—[slight stutter]—and cat, Whiskers. Holding down the fort. But you and Jin, Kai, are like the family I choose for myself.

Me, to Jin:So how can I get to know you again?

Jin: I’m the one who used to put extra toppings in your ray-men. [note awkward pronunciation change] when you were a child. I guess you needed it more than the stray cats hanging around the back alley.

Me, to Nova: What will you do after this?

New star: I could go to the Lotus for a drink. Care to join? It’s been a while since we went out at night.

Me: Sure, are there any good movies in theaters?

New star:Let’s see what it is—[slight pause]—At the Holosuite cinemas. If not, we can always watch a classic at my house. I have the complete file.

Me, to Jin: You want to join us?

Jin: I’ll pass, thanks. This old cat statue on the counter brings me more luck than a night on the town.

Me: What are you going to do instead?

Jin: I’ll just hold down the fort here making sure the nighttime stragglers get their noodle fix. It’s not a glamorous job, but it’s what I’m good at.

The dialogue is not particularly inspiring! I certainly don’t prefer it to proper lines from good characters in my favorite games. You can also see how they only spoke in short bursts and how they sometimes lost the thread. But perhaps it could be used to populate an entire world with minor characters or combined with good canonical dialogue written by a real human being, where generative AI simply helps you go further.

After the conversation, Convai’s product manager, Nyla Worker, showed me some of the basics of the company’s browser-based dashboard.

I could see that Jin and Nova have their own “Character Description”, their own “Knowledge Bank” full of text documents they can refer to, sliders for “Personality” traits like extroversion, openness and sensitivity, their own “State of Mind”, some “guardrails” to hopefully prevent them from doing crazy things with the AI ​​(and filters for blocked words), and also a memory bank. The worker says that Convai is still working on long-term memory, but that the conversation engine can already store a couple of hours.

It also showed me how easy it was to inject new data. It took the touch of a button to modify Jin and Nova’s memory with an additional text file, and suddenly they could tell me about Nvidia’s new graphics cards. By pressing another button, these characters could speak in a new language.

Since I didn’t actually interact with the imaginary world that Jin and Nova theoretically live in, I can’t fully say what they are capable of. They seem pretty two-dimensional at this point, with “I’m a proud and selfless ramen shop owner with a secret recipe” being Jin’s entire gimmick. But I’d love to see what a good writer could do with his backstory and his motivations. I can now imagine games where NPCs remember what they’ve seen and react to the game script as it unfolds. The right bits could enter your memory bank at the right time, filter through your personality and desires, and make the game more immersive and interactive as a result.

I just hope game developers use this to increase their games, instead of putting voice actors and writers out of work. It’s an extremely hot topic in the video game industry right now.

This month, SAG-AFTRA signed a deal with Replica Studios that could allow members to license digital replicas of their voices. Some members are making it clear that the agreement does not represent their views. Last we heard, the union is still negotiating with game publishers over a new Interactive Media Agreement and has listed “the existential threat to its members’ jobs posed by the unregulated use of AI” as one of its top concerns. .

Leave a Reply

Your email address will not be published. Required fields are marked *