It’s a feat made all the more remarkable because the famous folk singer is dead.
National broadcaster SBS plans to use artificial intelligence (AI) to bring Kim’s voice back to life in a new program, “Competition of the Century: AI vs Human,” airing later this week.
It’s not the first time that AI has been used to revive a famous singer in South Korea.
In December, music channel Mnet aired “One More Time,” a show that used AI and holograms from deceased artists to pay tribute to their work.
Recent AI performances have thrilled fans of their music and technology, but others have voiced concerns about the ethics and legality of reviving the voices of the dead. The creation of new works – or revived voices – by AI also raises copyright issues. Who is considered the owner? The creator of the AI program or the AI system itself?
Legendary Korean singer
Kim was only 31 years old when he died in 1996 at the height of his career after a string of hits including “A Letter From a Private”, “Song of My Life” and “In the Wilderness”.
Decades later, fans still gather on a street named after him near his childhood home in Daegu city, although the tribute was mostly held online this year due to the coronavirus pandemic.
The artist’s murals peek across the street, where tourists sit on benches shaped like guitars and listen to musicians play his greatest hits.
So when national broadcaster SBS announced it was using AI to mimic Kim’s voice in a new program airing this month, it put fans in a frenzy.
“The restored voice is very much like him, as if Kim recorded him live,” said Kim Jou-yeon, a Kim fan for 30 years.
It even impressed a young Park Hye-hyun fan, who was only two years old when Kim died. “One of my wishes was to hear more songs in Kim’s voice… I had tears,” she said.
Show inspired by a Go competition
Surprisingly, Lee won one of their three matches.
Go requires two players to place black and white stones on a 19 by 19 grid. Each participant tries to claim the most territory by surrounding his opponent’s pieces so that they are removed from the board.
“Lee-AlphaGo was a big shock, followed by a brief but huge interest in AI, which is still under development somewhere, but we’re not familiar with it yet,” SBS producer Nam said in a phone interview.
Feeling that the Korean public was ready for an update, Nam began compiling the six-part AI competition series, including the AI performance of the late Kim Kwang-seok.
Recreate Kim
The audio AI company behind Kim’s reincarnated voice is Supertone, a South Korean startup founded in 2020 that provides AI audio solutions for content creators, according to co-founder and Chief Operating Officer Choi Hee-doo.
“For example, BTS is very busy these days, and it would be a shame if they couldn’t participate in content due to time constraints. So if BTS uses our technology when creating games or audiobooks or dubbing an animation, for example, they don’t necessarily have to personally, ”said Choi.
Supertone’s Singing Voice Synthesis (SVS) technology learns voices by listening to multiple songs with accompanying notes and lyrics, Choi explained.
The system learned 100 songs from 20 singers before getting 10 Kim Kwang seok songs to learn. It now knows its voice well enough to mimic the singer’s unique style and pronunciation, Choi said.
In the SBS show, the AI Kim will not compete against a human singer – he will sing a duet with it. “We’ve revived Kim Kwang-seok to showcase its capabilities,” said Nam, the show’s producer.
Instead, Ock Joo-hyun, the former lead singer of girl band Fin.KL, takes on the AI machine. Just as the AI system has taught Kim’s voice, it will be trained to mimic her too.
While some view a match between an AI singer and a human as harmless fun, others warn that the technology poses threats that must be addressed by stricter guidelines and regulations.
Potential dangers of AI
Tech-savvy South Korea is considered a forerunner in AI technology, but there is a call for more regulation.
Countries and organizations around the world are calling for legal reforms, ethical guidelines and social agreements to prevent the misuse of AI.
Heo Hoon, Supertone’s Chief Technology Officer, said he supports the laws to regulate the industry. To mitigate current risks, he said the company does not deal directly with the public but prefers to work with companies that share its ethics.
“We are clearly aware of the potential for our technology to be misused when in the hands of the public,” said Heo. “I think reviving the voice of the deceased would set off a backlash that would hopefully start a public debate and lead to final legislation.”
The company has built a level of protection into its recordings by marking the audio as produced by its AI, although casual listeners are unlikely to know.
“We have watermarking technology, which is information that is planted in audio that cannot be heard, but it does track where it was made and how it is distributed,” said Heo.
Questions about ownership
Using AI to create works traditionally produced by human creativity raises the issue of ownership. If AI makes the works, does the program own the copyright or is it owned by the programmer?
Lawyer Ko Hwan-kyoung, an expert in AI and data protection, said the problem needs to be addressed as AI becomes more advanced.
“What’s interesting about AI is that it is evolving to a level where it can write and compose by studying data,” said Ko.
Last year singer Hayeon released “Eyes on You”, a single composed by an AI program. Human producers later refined the song, according to her record label Enterarts.
“Are we going to recognize AI as a legal entity with legal personality like humans, and grant copyright?” Ko, the lawyer, asked. “We need the necessary regulations that guarantee human safety, not excessive regulations that hinder the development of AI technology.”
In the case of Kim Kwang-seok, producer Nam said SBS had received permission from Kim’s family to reproduce his voice before moving on with the show.
SBS paid a one-time fee to his family for showing his voice on the show, as with other cast members, Nam said. Portions of the show will be published on YouTube after airing, but neither SBS nor Supertone plan to release Kim’s song as a single.
“We have agreed not to officially release the songs introduced in the show,” said Choi of Supertone.
So, for now, Kim fans will have to be content playing the old songs that the real singer recorded with a microphone over 20 years ago.
Son Somi in Seoul also contributed to this story.