How to break free of Spotify’s algorithm [MIT Tech Review]

August 16, 2024August 16, 2024 Tiffany Ng 0 Comments

View Article on MIT Tech Review
Since the heyday of radio, records, cassette tapes, and MP3 players, the branding of sound has evolved from broad genres like rock and hip-hop to “paranormal dark cabaret afternoon” and “synth space,” and streaming has become the default. Radio DJs have been replaced by artificial intelligence, and the ritual of discovering something new is neatly packaged in a 30-song playlist, refreshed weekly. The only rule in music streaming, as in any other industry these days, is personalization.

But what we’ve gained in convenience, we’ve lost in curiosity. Sure, our unlimited access lets us listen to Swedish tropical house or New Jersey hardcore, but this abundance of choice actually makes our listening experience less expansive or eclectic.

Most of us access music through streaming services: over 600 million of us worldwide, to be exact. And claiming over 30.5% of this population, nearly double the share of any other streaming service on the market, is Spotify. With its game-changing release in 2015 of Discover Weekly—a generated playlist that tailors song selections to a user’s listening habits—Spotify presented personalization as the remedy to our overabundance of options.

But in efficiently delivering what people seem to want, it effectively eliminated choice and removed humanity from the entire music listening—and music discovery—experience. According to a 2022 report published by Distribution Strategy Group, at least 30% of songs streamed on Spotify are recommended by AI. The success of Discover Weekly has since inspired mood-dependent playlists that change throughout the day and psychic readings based on people’s listening habits. Other streaming platforms, like Apple Music and Amazon Music, have followed suit. All these takes on personalization share a common fault: The playlists too often resemble one another, filled with songs that offer different variants of the same sound.

Glenn McDonald, a former engineer at Spotify and the self-described “data alchemist” largely responsible for developing the company’s encyclopedia of genres, believes that while accessing new music is technically easy, many of us don’t do it—mainly because we’re not sure where to start looking.

As we grow accustomed to the convenience of shuffling a generated playlist, we forget that discovering music is an active exercise.

We expect too much from the algorithm

For Spotify, McDonald says, personalization begins with breaking down songs via a data-intelligence platform that was known as the Echo Nest before the company acquired it. Through a combination of signal processing and human listening by musicologists, Spotify assigns roughly 10 different attributes to songs (e.g., key signature, danceability) before grouping them into libraries. AI-powered programs then pull from these buckets of sound to generate the personalized playlists, with parameters tailored to the habits of each user. How Spotify categorizes music determines what is made visible to us. It also shapes which niches artists fit into and how much exposure they get.

McDonald sorts our listening habits into three concentric clusters: the stuff we listen to every day, the stuff that sounds like our stuff, and everything else we stumble upon. Spotify’s auto-generated playlists mostly keep to the first bucket, occasionally venturing into the second. The third is accidental. The service never offers anything strictly different.

Spotify thinks that even if we say we want to listen to something new, we always return to what’s familiar, McDonald explains. He argues that in practice, slipping a reggae track into a playlist of “bedroom pop” (a genre that mainly features dreamy melodies and hushed vocals) often makes for an uncomfortable listening experience: “If you’re given something new, it’s odd, in the same way being teleported to random spots around the world for three minutes at a time would not be a pleasant tourism experience.”

To construct the 6,291 microgenres in its database, McDonald says, Spotify uses social data—how listeners of the same artists sort those artists’ songs and who else they listen to. He clarifies that Spotify’s genres have no absolute boundaries but reflect a loose and dynamic consensus on how users listen to music. Small clusters of overlapping listening habits define these loose categories, while constant cross-pollination creates variations on them. “Everyone understood where the center of the village was, and the further out you went, the more subjective it became,” he says of the process as he remembers it. McDonald mapped this musical landscape on his personal site, Everynoise.com.

As we grow accustomed to the convenience of shuffling a generated playlist, we forget that discovering music is an active exercise.

Our respective listening habits, when considered together, form a dynamic network that reveals how we collectively understand music. It’s a shame that Spotify’s current usage confines us to isolated algorithmic bubbles.

Context and community

Personalization, broadly speaking, has made navigating the internet’s infinite pool of content incredibly convenient. We’re served what we like, told what to buy, prompted on what to say. It’s no surprise we expect our music streaming apps to do the same. However, employing algorithms to optimize music discovery requires explicitly defining what we want, and the problem is that what we want could easily be shaped by what we encounter. Asking an algorithm to broaden our horizons is like having lunch with a friend who claims to be open to anything but vetoes everything you suggest. “Curiosity is an active mode,” McDonald says. It’s up to us to step outside our bubble.

Music enthusiasts are creating new ways to reinvigorate this sense of curiosity, building everything from competitive recommendation leagues to interactive music maps. Before streaming, discovering music was work that brought a distinctly emotional reward. “Back in college, I listened to whatever my friends were listening to,” recalls Zack O’Malley Greenburg, former senior music editor at Forbes. He describes exchanging CDs with friends, spending hours deciding which songs he liked and which he didn’t. Later, acquiring new music became an exercise in sorting through audio files on thumb drives and (illegally) downloading MP3s from questionable websites. Sharing music was a much more personal, peer-to-peer exercise, and making a mixtape for a crush was a substantial labor of love. Automated recommendation systems have replaced this social culture of sharing music. The anonymous playlists we opt into today may be edited and even shared, but the emotional stakes are much lower.

Because personally recommending songs revealed our taste, we had a vested interest in what we recommended. But the algorithm assumes no risk, simply offering what’s mathematically sound.

“What I think is missing in music streaming is why someone thinks I should like a certain song,” says Alex Keller, one of the cofounders of Music League, an online platform that allows people to submit songs to playlists that fit a certain theme. The platform has doubled its user base since last year, to roughly 130,000 monthly users.

Music League has built this loyal community by gamifying the experience of recommending music. Users can join public leagues or create private ones with themes ranging from “Best rap song” to “Horse crime.” Each league hosts multiple rounds, where participants compete by submitting and voting for songs they think best fit a prompt. A big part of the experience, Keller says, is the conversation around each submission. He describes how his experience of each song changes as users are pushed to defend their choices.

Unlike the myriad personalized Spotify playlists that instantaneously refresh on demand, leagues can be open for months at a time. There may be a long gap between receiving a prompt and submitting a song, or between listening and voting. People are encouraged not only to listen to songs from start to finish (an increasingly rare practice) but also to include liner notes alongside the songs they submit. Slowing down the process of music discovery can foster more purposeful listening.

“As an adult, music is in the background in your life,” Keller says. For him, the social focus of Music League puts it back in center stage. The collaborative recommendation process gives each song emotional weight and offers a refreshing departure from the streams of generated playlists we shuffle for ambience.

Similar to Music League is a private Facebook community called Oddly Specific Playlists, a group that connects users from all corners of the internet with playlists inspired by (as the name suggests) very specific things. With over 364,000 members, the group is flooded with requests daily; users post pieces of inspiration and attach a brief explanation of their interest in the theme. Others share relevant songs and offer personal anecdotes to color their recommendations. Requests like “Strong masculinity; healthy, not toxic; not misogynistic; bonus points for queerness” beget discussion. What could strong masculinity sound like? What does a healthy song entail?

Often, playlist requests have ventured into more somber subjects like heartbreak and grief. As users share deeply intimate stories about their relationships to specific songs, conversations develop and communities heal. The fact that members have likely never met can make the experience even more meaningful. Connecting with strangers around the world reveals the universality of even the most seemingly specific experiences and offers a unique form of validation. The discussions can also breathe new life into old songs; a request for songs prominently featuring the sound “oh,” from a member whose two-year-old was obsessed with the letter O, spotlighted “Oh! Darling’” by the Beatles.

Rather than challenging your tastes, algorithms only provide shuffled versions of what you already enjoy.

This focus on fostering organic human interaction is not new. Until 2017, Spotify actually had a chat feature, but it wasn’t used widely enough (and didn’t result in enough streams) to justify the resources required to maintain it. So instead, the company pivoted to optimizing personalization.

While Spotify’s platform evolved to make choosing music as easy as possible, the unpolished format of Oddly Specific Playlists has largely remained the same. Comments are still difficult to keep track of, and users must sift through mountains of posts to find relevant
recommendations. Despite the clunky experience, the community has been thriving since 2019.

“If a social network is any good, then it has to have some actual people putting new content into the ecosystem and organizing it in a coherent way—like someone making a hand-curated playlist,” says Kyle Chayka, a New Yorker staff writer and author of Filterworld: How Algorithms Flatten Culture. That’s just what the members of Oddly Specific Playlists do, even if the results can be hard to manage.

In his book, Chayka recounts the many hours he’s spent surfing music forums like AntsMarching.org and UFCK.org (fan sites dedicated to all things concerning the Dave Matthews Band and Pearl Jam, respectively), finding company with other posters who shared low-fidelity tapes from old concerts and fun facts about a band’s formation. These cultural rabbit holes, to Chayka, offer a form of “mutual learning” that helps us better understand what we’re consuming. If we know how an artist’s signature style came to be, for example, we’re more capable of intentionally shaping our tastes.

Slowing down with curation

In Filterworld, Chayka also outlines how algorithms have taken the place of magazine editors and museum curators as gatekeepers of culture. “I think curation is a way to resist the flattening of the internet,” he says, though acknowledging that the term itself has been watered down over the past decade.

Chayka frames curation as intentional, arduous, and finite—characteristics he deems antithetical to our relationship with algorithms. Where a curator voices perspectives that welcome discourse and discomfort, algorithms are written in fear of offending. “When a human interprets a piece of art, it adds value rather than takes it away. An algorithm has no capacity to interpret,” he adds.

Before streaming, a magazine profile on an emerging artist or a blogger’s “Songs I’m listening to” column would put musicians on your radar, inspiring deep dives into their discography. Music publications like Blender, NME, and The Source, also had great influence, the latter notably discovering The Notorious B.I.G. and highlighting him in its “Unsigned Hype” column. But, as Greenburg explains, “streaming services remove a step.” Rather than challenging your tastes, algorithms only provide shuffled versions of what you already enjoy. Like the Soylent shakes popular in the mid-2010s for supposedly offering all the nutrients you need from a meal, these personal playlists may fulfill but can never satiate.

In Filterworld, Chayka offers independent radio DJs as an antidote to the algorithmic takeover. The vaguely physical act of tuning into a radio station, like entering a concert hall, restores a tactile quality to our experience of music. When there’s a voice behind the selection of songs, we’re more likely to pay attention, Chayka insists. He describes how these DJs “utilize all of their knowledge, expertise, and experience in order to determine what to show us and how to do it.”

“When a human interprets a piece of art, it adds value rather than takes it away. An algorithm has no capacity to interpret.”
Kyle Chayka, the New Yorker

The Hong Kong–based musician known as Cehryl, who hosts the show Mystery Train on Eaton Radio, structures her shows around narratives. “I think about my shows in the same way I think about a performance,” she says. “There’s an emotional arc.” She puts her tastes first, hoping to express a unique point of view that will bring something new to her listeners.

In a world of on-demand music, the real-time format of independent radio mandates a specific sequence of uninterrupted listening. Without skips, shuffles, or the ability to pause, it gives curators the opportunity to push their listeners’ boundaries.

Creating with “algorithmic anxiety”

To Cehryl, a big part of being a musician today is grappling with the existential question of whether to make music for the algorithm. Since the popularization of streaming (and the rise of TikTok), the average length of a song has decreased from four minutes to roughly three. Artists are encouraged to put out singles or EPs instead of releasing concept albums. And in 2023, Spotify launched the Preview function, a TikTok-esque infinite-scroll music feed that showcases the “best” few seconds of each song with every swipe. The algorithm rewards relevance and instant gratification. “No long songs. No patient, drawn-out songs. You want the hook by 15 seconds in, if not earlier,” Cehryl says.

Experiencing what Chayka calls “algorithmic anxiety,” Cehryl describes a need to feed the algorithm’s perception of her: “I have often been playlisted as bedroom pop. But I don’t think I make bedroom pop.” For artists, Spotify’s genre breakdowns play a complicated role in their creative process.

Spotify’s algorithm offers loose categorizations to identify emerging genres or remodel familiar ones, but the platform’s promotion of broader, more recognizable genres makes some artists feel pigeonholed and pressures others to conform. Fitting into Spotify’s categories increases an artist’s chances of going viral on the platform, even if each stream yields only $0.003 for the creator.

Alex Antenna, who has created a website called Unchartify to offer a more manual way of navigating Spotify’s database, attributes these pigeonholes to Spotify’s push for personalization. He built his site to bypass the plethora of “made for you” playlists and highlight lesser-known corners of Spotify’s database.

“Spotify’s music database has a very rich set of various parameters, markup, and categories to classify music in a very detailed way. This is simply not exposed in the official app,” he says. He believes that even though it has a sophisticated way of sorting music, Spotify intentionally oversimplifies: Its library offers mainly personalized playlists drawing on broad categories like “metal” or “party,” many of which feature mostly “popular artists or songs you heard 1,000 times.”

Antenna points out that beyond genres such as bedroom pop or indie folk, Spotify offers a plethora of microgenres (such as “reminimal” and “sky room”) that are accessible only by name through its API. He hopes that by surfacing genres that more accurately represent an artist’s sound, a system as granular as Unchartify can combat algorithmic anxiety.

Unchartify reorganizes Spotify’s database by sorting all genres into alphabetical order—something unheard-of in today’s world of engagement optimization—and mapping them so that each album is a node connecting to a list of similar albums. Unlike Spotify’s “Fans also like” feature, which recommends similar artists without suggesting where their similarity lies, Unchartify offers a precise picture of where an album sits musically in relation to others.

Unless specifically asked, Unchartify doesn’t try to guess what you’re looking for. Instead, it gives you the tools to surf Spotify’s database systematically, as you might sift through archives in a public library. Antenna’s position reveals an important source of tension in the world of on-demand music: Making the abundance of content online digestible requires simplification, but simplification often forgoes nuance.

Beating the algorithm

Going a step beyond Antenna’s archaic decision to list genres alphabetically is Radiooooo, a self-described musical time machine that randomizes the discovery process by eliminating genre entirely.

Founded in 2012 by a group of four DJs, Radiooooo curates a selection of songs for each decade dating back to the 1900s for each country across the globe. It prompts users to select music by time periods and geographic locations rather than genres or artists—discarding any semblance of our current streaming experience and inspiring a new way of thinking about music. Radiooooo also tacks on a social component by crediting members who’ve discovered the track, joining communities like Music League and Oddly Specific Playlists in encouraging a form of crowdsourced recommendation that invites conversation and disagreement—a far cry from Spotify’s vision of optimized, unimpeded listening.

Perhaps the only way to escape our algorithmic bubbles is by building community. When we welcome diverse patterns of music consumption, we’re challenged to consider music from different perspectives, the same way independent radio stations curate to tell a story rather than cater to a demographic. There’s nothing to optimize in a community, and in turn, nothing to oversimplify.

Despite functionally contradicting Spotify’s philosophy, platforms like Radiooooo, Music League, Oddly Specific Playlists, and independent radio all complement the use of such platforms. They act as a springboard for our process of discovery, helping us step past Spotify’s insistence on personalization by directing us where to look and, most important, making it fun.

McDonald likens the functions of Spotify to Google Maps. “Google Maps doesn’t do the exploration for me, but it’s helpful if I go somewhere,” he says. Rather than taking us on guided tours, it provides the tools for us to navigate somewhere new. Much as it shows us what’s nearby and how to get there, and flags notable landmarks others have visited, Spotify helps us access most music, lists global listening trends, and introduces us to artists similar to those we already know. But it’s communities that help us home in on a destination Spotify can help us explore.