Music Recommendation by Unified
Hypergraph: Combining Social Media
Information and Music Content
Jiajun Bu, Shulong Tan, Chun Chen, Can
Wang, Hao Wu, Lijun Zhang and Xiaofei He
Zhejiang University
1
Multi-type Media Fusion
• Content analysis
–
–
–
–
–
text
Image
Audio
Video
……
• Social analysis
–
–
–
–
–
Hypergraph
Friendship
Interest group
Resource collection
Tag
……
2
Outlines
• Music Recommendation
• Social media information
• Unified Hypergraph Model
• Music Recommendation on Hypergraph (MRH)
• Experimental results
3
Music Recommendation



We have huge amount of music available in music
social communities
It is difficult to find music we would potentially like
Music Recommendation is needed!
Recommended music
by the Last.fm.
4
Traditional Music Recommendation

Traditional music recommendation methods
only utilize limited kinds of social information
 Collaborative Filtering (CF) only uses rating
information
 Acoustic-based method only utilizes
acoustic features
 Hybrid method just combines these two
5
Music Social Community
[www.pandora.com]
Social activities
Actions which between
users
users
can do on resources
6
Introduction to Last.fm
Users can listen to music.
Users
canare
bookmark
These
music tracks
resources
bylikes
tags.best.
this user
Users can
also
make
Users
friends.
can
join in
groups
7
Social Media Information in Last.fm
Memberships
Friendships
Tagging
relations
Listening
relations
Inclusion
relations
8
Social Media Information

The rich social media information is valuable
for music recommendation.
To
build the users’ preference profiles.
To
predict users’ interests from their friends.
To
recommend music tracks by albums or artists.
…
9
How About Graph Model?

Use traditional graph to model social media
information but fail to keep high-order
relations in social media information
(u1, t1, r1)
(u1, t2, r2)
(u2, t2, r1)
It is unclear
whether u2
bookmarks r1,
r2, or both.
10
Unified Hypergraph Model

Using a unified hypergraph to model multi-type objects
and the high-order relations
Each edge in a hypergraph, called a hyperedge, is an
arbitrary non-empty subset of the vertex set
Modeling each high-order relation by a hyperedge, so
hypergraphs can capture high-order relations naturally
(u1, t1, r1)
(u1, t2, r2)
(u2, t2, r1)
The high-order
relations among the
three types of objects
can be naturally
represented as triples.
11
Unified Hypergraph Construction
tag
Each type of relations
corresponds to a certain
type of hyperedges in the
unified hypergraph.
album
tag
tag
album
The six types of objects form the
vertex set of the unified hypergraph.
12
Hyperedges Construction Details
:a hyperedge
corresponding to
each pairwise
friendship
:a
hyperedge corresponding
to each tagging relation
:a hyperedge
corresponding to
each group
:a hyperedge
for each album or artist
: a hyperedge
for each track-track
similarity relation
: a hyperedge for each
user-track listening relation
13
Ranking on Unified Hypergraph
Setting a user
as the query
Track List
…
Tracks have
more strong
“hyperpaths” to
the query user
will get higher
ranking scores
14
Notation
• A unified hypergraph
•
: Vertex-hyperedge incidence matrix
15
Notation-2
•
: the degree of a hyperedge is the number
of vertices in the hyperedge :
•
: the degree of a vertex is the weight sum
of all hyperedges the vertex belongs to:
• Dv , De and W : diagonal matrices consisting of
hyperedge degrees, vertex degrees and
hyperedge weights
16
Problem Definition
• Given some query vertices from , rank the
other vertices on the unified hypergraph
according to their relevance to the queries.
• : the ranking score of the i-th object
•
: the vector of ranking scores
•
: the query vector
17
Cost Function
Vertices contained in many
common hyperedges should
have similar ranking scores
Obtained ranking scores should
be similar to pre-given labels
The optimal ranking result is
achieved when Q(f) is minimized
18
Matrix-vector Form
19
Optimal Solution
Requiring that the
gradient of Q(f) vanish
gives the following this
equation
We define
Note: all the matrices are highly sparse!
20
Music recommendation on Hypergraph (MRH)
• The offline training phase:
Constructing matrix H and W
Computing matrix Dv and De
Calculating
, where
• The online recommendation phase:
Building the query vector y
Computing the ranking results f*
Recommending top tracks which not listened
21
General Ranking Framework
Setting a user
as the query
User List
…
For friend recommendation
Group List
…
For group recommendation
Tag List
…
For topic recommendation
Track List
…
For music recommendation
Album List
…
For album recommendation
Artist List
…
For artist recommendation
22
Personalized Tag Recommendation
Tag List
Setting a user
and an resource
as the queries
…
Personalized Tag
recommendation for
the target user and
resource
23
Objects and Relations in Our Dataset
Objects
Relations
24
Compared Algorithms
Algorithms
Information Used
User-based Collaborative Filtering (CF)
R3
Acoustic-based music recommendation (AB)
R3, R9
Ranking on Unified Graph (RUG)
R1, R2, R3, R4, R5, R6, R7, R8, R9
Our proposed music recommendation on
Hypergraph method (MRH)
•
•
•
•
•
R1: friendship relations
R2: membership relations
R3: listening relations
R4: tagging relations on tracks
R5: tagging relations on albums
MRH-hybrid
R3, R9
MRH-social
R1, R2, R3, R4, R5, R6, R7, R8
MRH
R1, R2, R3, R4, R5, R6, R7, R8, R9
•
•
•
•
R6: tagging relations on artists
R7: track-album inclusion relations
R8: album-artist inclusion relations
R9: similarities between tracks
25
Performance Comparison
Comparison of recommendation algorithms in terms of MAP and F1.
Comparison of recommendation algorithms in terms of NDCG.
It is clear that our proposed algorithm significantly outperforms the
other recommendation algorithms
26
Precision-Recall Curves
Comparing
to MRH-social,
MRH
uses
similarity
relations
among
Acoustic-based
Our
proposed
(AB)
method
alleviates
works
these
worst.
problems.
That
is because
MRH-hybrid
acousticonly
The
CF
superiority
algorithm
does
ofmethod
MRH
not
over
work
RUG
well
indicates
too.
This
that
is probably
the
hypergraph
because
istracks
the
indeed a
additionally.
Wefor
find
that
using
acoustic
information
can
improve
uses
based
similarity
method
relations
incurs
the
among
semantic
gap
tracks
similarities
and
listening
based
relations,
on the
better
choice
user-track
modeling
matrix
complex
inthis
ourmusic
data
relations
setand
is highly
in
social
sparse
media
information
recommendation
result,
when
we than
onlywith
care
topCF
ranking
music tracks.
acoustic content
butare
itespecially
works
not always
much
consistent
better
AB and
human
knowledge
27
Social Information Contribution
MRH
using
listening
relations
MRH
using
listening
relations,
using
listening
The MRH
baseline
is MRH
onlyrelations
and
tagging
relations
and
relations
andsocial
inclusion
relations
using listening
relations
Comparison of MRH on different subsets of social media information in terms of MAP and F1.
There is a little improvement at lower ranks obtained by social
Tagging relations do not improve the performance. That is because there is
relations.
Intuitively,
the users’ tastes
may be inferred
from friendship
Using correlation
inclusion
relations
resources,
cantagging
recommend
music
a strong
betweenamong
listening
relationswe
and
relations,
and
and
membership
relations.
tracks in the same
or similar
albums,
well as the
tracks performed by
thus the
usage of
taggingasrelations
is limited
the same or similar artists. So the performance is improved greatly.
28
An Example
Top 5 Recommended Tracks for No.793
No. 793
User No. 793 joins in
the groups about
metal and named
The reason these
Slipknot. Users in
three tracks are
these groups are
recommended is
fans of Metallica
that user No. 793
(one of the four
likes music come
most popular heavy
from System of a
metal band. )
Down best.
and Slipknot
War?
From:
System of a Down
Know
From:
System of a Down
Dirty Window
From:
Metallica
Deer Dance
From:
System of a Down
Spit It Out
From:
Slipknot
29
Conclusion
• We use the unified hypergraph model to fuse
multi-type media, includes multi-type social
media information and music content.
Social media information is very useful for
music recommendation.
Hypergraphs can accurately capture the
high-order relations among various types of
objects.
30
31