Shuqi Dai
Deep Music Generation via
Music Frameworks
Music Framework is a hierarchical music structure representation. With Music Frameworks and new musical features, we combine music domain knowledge with deep learning, and factor music generation into sub-problems, which allows simpler models, requires less data, and achieves high musicality.
I. Controllable Deep Melody Generation via Music Frameworks
Demo
-
Example 0 contains some cherry-picked demos, Examples 1 - 6 are non-cherry-picked demos.
-
We directly use the chord progression from the original input song. In Example 0, we use the original accompaniment from the input song as well.
-
"#G" is the group number we used in the paper to identify different generating controls.
-
In Examples 1 - 6, #G group numbers 3 and 5 are control groups not using Music Frameworks.
-
All music Copyright 2021 Shuqi Dai. Please do not copy or distribute any AI-generated music in this page.
Example 0
Original Input Song1
Output New Melody
(Copy Basic Melody & Rhythm, Generate Melody)
(Generate Basic Melody, Copy Rhythm, Gen Mel)
(Copy Basic Melody, Generate Rhythm, Gen Mel)
Original Input Song2
Output New Melody
(Copy Basic Melody & Rhythm, Generate Melody)
(Generate Basic Melody, Copy Rhythm, Gen Mel)
(Generate Basic Melody & Rhythm, Gen Mel)
(Generate Basic Melody & Rhythm, Gen Mel)
Original Input Song3
Output New Melody
(Copy Basic Melody & Rhythm, Generate Melody)
(Copy Basic Melody & Rhythm, Generate Melody)
(Copy Basic Melody & Rhythm, Generate Melody)
(Generate Basic Melody, Copy Rhythm, Gen Mel)
(Generate Basic Melody, Copy Rhythm, Gen Mel)
(Copy Basic Melody, Generate Rhythm, Gen Mel)
Original Input Song4
Output New Melody
(Copy Basic Melody, Generate Rhythm, Gen Mel)
(Generate Basic Melody & Rhythm, Gen Mel)
Example 1
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
control
group
control
group
generate
generate
6
generate with Basic Rhythm Form
original
Example 2
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
generate
generate
6
generate with Basic Rhythm Form
control
group
control
group
original
Example 3
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
generate
generate
6
generate with Basic Rhythm Form
control
group
control
group
original
Example 4
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
generate
generate
6
generate with Basic Rhythm Form
control
group
control
group
original
Example 5
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
generate
generate
6
generate with Basic Rhythm Form
control
group
control
group
original
Example 6
Original Input Song
#G
Rhythm
Generated New Melody
Realized Melody
Basic Melody
generate
copy
copy
generate
generate
copy
generate
without
copy
1
2
3*
generate
copy
generate with Basic Rhythm Form
4
generate
copy
generate without Basic Rhythm Form
5*
generate
generate
6
generate with Basic Rhythm Form
control
group
control
group
original