Data and emotion: Could Spotify's emerging AI technology change how we feel?
By Klara Lee
Fri 26 March 2021
Spotify Web Player has recently patented a voice recognition technology that's capable of tailoring bespoke music recommendations to our emotions and environment. We've long known that music has a strong impact on our mood. With the help of Artificial Intelligence, could this new music technology bring us one step closer to a world where we can manipulate ourselves into feeling how we want?
Could this benefit our lives, or will it become another powerful tool for advertisers and tech giants to target us more effectively? In this article, we’ll talk you through how Spotify’s emerging technology identifies your emotions, its plan to use this personal data to advertise to you better, and the promising future for AI and music.
How could Spotify detect my mood?
Two years in the making, Spotify has finally received the patent for an emotional speech recognition technology that could revolutionise our every day lives. By analysing tone, rhythm and speech patterns, it's able to infer your emotional state, age, and gender. It then uses this to suggest songs accordingly. The technology can also determine your personality type: ‘the tone of voice may be more upbeat, high-pitched and/or exciting for users that have been assigned the personality trait of extroversion.’
Last year, Spotify conducted a study following how people's personality traits influence their musical preferences. Introverts, for example, tend to dig deep into an artist's catalogue, ‘listening to more tracks for each artist they discovered’. Spotify can also ‘make observations’ about your environment and play music that fits your social setting, for example ‘alone, small group, party’.
The patent has the ability to categorise your mood as ‘happy, angry, sad, or neutral’, and these results can be combined with other information, like the personal data it already collects from third party platforms such as Facebook. This is then used to make ‘inferences about your interests and preferences’.
How might Spotify's voice assistant help our day to day lives?
Right now, the potential benefits of Spotify knowing your mood is to have more relevant and relatable song recommendations. But, if Spotify, or any other music streaming service, announces that it can use music and AI to manipulate how you feel, this could have a powerful impact on your life.
A positive example of this is a study that found that people who listened to upbeat music could improve their moods and boost their happiness in just two weeks. We also know that music makes you feel better the older that you get. Beyond just improving your daily mood, one study showed that music therapy can be used as treatment for emotional disorders related to neurological conditions, including ‘Parkinson’s disease, dementia, stroke, and multiple sclerosis’.
This is supported by the work of the American Music Therapy Association (AMTA), which claims that music therapy can help with managing stress, enhancing memory, and even relieving physical pain.
Similarly, The Lancet found that music can alleviate pain and anxiety amongst those undergoing or recovering from surgery, so much so that many didn’t even need pain medication. The Lancet study highlighted that pain and anxiety were particularly reduced when individuals got to choose their own songs: suggesting that tailored music is the key to high effectiveness. Spotify’s algorithm, combined with further personal information collected from other platforms, could be used to effectively regulate and change our daily emotions, as well as treat neurological conditions and physical body pain, in the long term.
Does this technological advancement pose an ethical concern?
Currently, Spotify already shares ‘user data, usage data and voice data’ with third parties for advertising purposes with over sixty companies. These include Amazon, Facebook, Google, Pinterest, Snapchat and Twitter. So, once this new technology is implemented, your voice data and therefore your current emotional state, gender, age and accent could be used by all of these advertisers to target you more strategically.
Targeting you with advertising based on your songs or voice and inferred mood could be considered unethical. Other than selling your emotional state data, the collection and selling of a user’s personal trigger or comfort song could potentially be used to target them commercially.
In a general sense, we also know that this isn't actually that far removed from what many companies have done for some time, albeit in different ways. For example, research has shown that music can influence how fast you eat your food in a restaurant or how long you remain in a shop. This is because the speed of the music played correlates with speedier spending decisions.
There are of course grave potential misuses of the ability to track the everyday emotions of people, such as to manipulate mood or the sharing of this sensitive information for commercial gain with third parties. But, the positives could also be profound, such as the ability to soothe our mental state or physical pain. As ever, it is less the intrinsic nature of the technology itself, but rather how it will be used.
Hopefully, we will see a future where companies work constructively with regulators to ensure the benefits to the user are maximised and the harms prevented.