AI Creativity

Question 1

What are the three types of art described in the taxonomy of computer-generated art?

Answer 1

C-art, G-art, and CG-art

Question 2

What is C-art?

Answer 2

Art that uses computers as part of the art-making process

Question 3

What is G-art?

Answer 3

Art that is generated, at least in part, by some process not under the artist’s direct control

Question 4

What is CG-art?

Answer 4

Art produced by leaving a computer program to run by itself, with minimal or zero human interference

Question 5

What are passive tools in the context of generative systems?

Answer 5

Tools that make no attempt to alter the user’s input, e.g., Microsoft Paint

Question 6

What are active tools in the context of generative systems?

Answer 6

Tools that actively interpret and process user inputs, adding things to it, e.g., the sketch pad mentioned in the lecture

Question 7

What is a first-order Markov model in text generation?

Answer 7

A model where the next state depends only on the current state (word)

Question 8

What is a second-order Markov model in text generation?

Answer 8

A model where the next state depends on the two previous states (words)

Question 9

What are the three parts of the example AI system for generating pop music?

Answer 9

1. Lyric generator using GPT-2 transformer model, 2. Music generator using Music-VAE auto-encoder model, 3. Singing voice synthesis using the DiffSinger model

Question 10

What is the basic process for generating text using a Markov model?

Answer 10

1. Pick a random initial state, 2. Select from possible next states, 3. If no possible next state, go back to step 1

Question 11

Name three examples of more complex generative text models.

Answer 11

Variable order Markov model, Long short-term memory network (LSTM), Transformer network

Question 12

What are some features to consider when describing a generative system?

Answer 12

System architecture, Number of agents, Roles, Environment, Corpus, Input, Output, Communication, Human interactive modality, Task, Evaluation

Question 13

How does a second-order Markov model differ from a first-order model?

Answer 13

It considers the two previous states (words) instead of just the current state

Question 14

What is the main difference between more complex generative models and simple Markov models?

Answer 14

They have a more complex method for picking the next state/output, including more complex state representations

Question 15

In the context of generative systems, what is a corpus?

Answer 15

The collection of data (e.g., text, images, music) that the system uses to learn and generate new content

Question 16

What is the main advantage that transformers add to language models?

Answer 16

They add context awareness to embeddings, allowing for a combination of contextual and sequential data

Question 17

How does self-attention work in transformer networks?

Answer 17

It creates a new type of embedding that incorporates information about other words in the context, not just the current word

Question 18

What are the two main ways of encoding input in recurrent neural networks?

Answer 18

1. One-hot encoded vectors, 2. Embeddings

Question 19

What is the ‘bag of words’ approach good for, and what is its limitation?

Answer 19

It’s good for sentiment analysis, but not great for generating text as it ignores sequence

Question 20

When was the transformer architecture first reported?

Answer 20

2017

Question 21

What are the two main components that transformers model?

Answer 21

Sequence and context via self-attention

Question 22

How many parameters does GPT-2 have?

Answer 22

1.5 billion parameters

Question 23

How many layers does GPT-2 have?

Answer 23

48 layers

Question 24

What is meant by ‘zero-shot’ concept in relation to GPT-2?

Answer 24

GPT-2 can perform tasks it wasn’t specifically trained for, outperforming some specialized models

Question 25

What is Huggingface?

Answer 25

A platform on a mission to democratize good machine learning, providing tools and models for NLP

Question 26

Why was GPT-2 initially considered 'too dangerous to release'?

Answer 26

Due to concerns about potential malicious applications of the technology

Question 27

What is the 'auto-regressive mode' of GPT-2?

Answer 27

It can generate an endless stream of words based on previous output

Question 28

How much text data was GPT-2 trained on?

Answer 28

40G of text

Question 29

What are 'attention heads' in the context of transformers?

Answer 29

Multiple projections of attention, allowing the model to focus on different aspects of the input simultaneously

Question 30

What is the main advantage of using pre-trained models like those from Huggingface?

Answer 30

They allow for quick implementation and fine-tuning of state-of-the-art language models without training from scratch

Question 31

What is a latent space in the context of machine learning?

Answer 31

A compact way of describing a dataset, representing the learned statistical structure through a reversible dimension reduction technique

Question 32

How does a variational autoencoder (VAE) differ from a standard autoencoder in terms of latent space representation?

Answer 32

VAEs encode to parameters of a statistical distribution (mean and variance), while standard autoencoders encode to a vector

Question 33

What is the main advantage of using a VAE over a standard autoencoder for generative tasks?

Answer 33

VAEs force all areas of the latent space to be meaningfully decodable, making them more useful for generative exploration

Question 34

What does the 'recurrent' part in Music-VAE refer to?

Answer 34

It uses recurrent neural network components, specifically LSTM (Long Short-Term Memory) units

Question 35

How much training data was used for Music-VAE?

Answer 35

1.5 Million MIDI files from the web

Question 36

What is the dimension of the latent vector in Music-VAE?

Answer 36

256 or 512 dimensions

Question 37

What are the three main components of Music-VAE's architecture?

Answer 37

Encoder (Bidirectional LSTM), Latent Space, and Hierarchical Decoder

Question 38

How does Music-VAE represent musical sequences?

Answer 38

It encodes 2 or 16 bar musical sequences including pitches, durations, and timing, potentially with multiple parts (e.g., bass, melody, drums)

Question 39

What is the 'unrelated fragments problem' in music generation?

Answer 39

When sampling from latent space, generated sequences may not have coherent long-term structure

Question 40

How can one explore the latent space of a trained Music-VAE model?

Answer 40

By sampling random vectors, permuting existing vectors, or making subtle changes to vectors in the latent space

Question 41

What is self-supervised learning in the context of Music-VAE?

Answer 41

The model is trained to reproduce its inputs via encoding and decoding, without requiring separate labeled data

Question 42

Why are VAEs particularly useful for creative applications?

Answer 42

They allow for smooth interpolation between points in the latent space, enabling the generation of new, coherent samples

Question 43

What is the process of 'permuting the sample' in latent space exploration?

Answer 43

Generating a latent vector, saving it, reading it back, and then making alterations to explore variations

Question 44

How does Music-VAE handle multiple instrument tracks?

Answer 44

It can encode and decode sequences with multiple parts, such as bass, melody, and drums

Question 45

What is the advantage of using a hierarchical decoder in Music-VAE?

Answer 45

It ensures better use of the latent representation, allowing for more coherent long-term structure in generated sequences

Question 46

What are the three primary challenges in singing voice synthesis?

Answer 46

1. Timing, 2. Expressiveness, 3. Holding long notes

Question 47

What was the basis for HAL 9000's singing voice in '2001: A Space Odyssey'?

Answer 47

A tube model of the vocal tract, arranged by Joan Miller and Max Mathews

Question 48

What is 'Pink Trombone' in the context of voice synthesis?

Answer 48

A physical model of the vocal tract for speech synthesis

Question 49

What technique was used in the 'Speak and Spell' toy from the 1970s?

Answer 49

LPC (Linear Predictive Coding)

Question 50

What was a significant development in voice synthesis during the 1990s?

Answer 50

Realtime synthesis and the use of HMM (Hidden Markov Model) systems

Question 51

Name three recent deep learning-based singing voice synthesis systems.

Answer 51

1. XiaoiceSing (2020), 2. HiFiSinger (2020), 3. DiffSinger (2021)

Question 52

What are the main components of the DiffSinger system?

Answer 52

Language Model, Speech Model, and Vocoder Model

Question 53

What is the purpose of the Language Model in DiffSinger?

Answer 53

To convert text input into phonetic representations

Question 54

What does the Speech Model in DiffSinger produce?

Answer 54

Mel (spectral) features

Question 55

What is the role of the Vocoder in DiffSinger?

Answer 55

To convert the Mel features into actual sound waves

Question 56

What dataset was used in the LJSpeech model mentioned in the notes?

Answer 56

A public domain speech dataset of 13,100 short audio clips from a single speaker reading non-fiction books

Question 57

What are Mel features in the context of speech synthesis?

Answer 57

Spectral frames or tiny slices of audio that have been transformed, similar to an embedding

Question 58

Name three key terms used in modern speech synthesis systems.

Answer 58

Embedding, Convolutional, Transformer

Question 59

What was a significant development in voice synthesis during the 2000s?

Answer 59

Vocaloid and concatenative spectral synthesis

Question 60

How do deep learning models in the 2010s-20s differ from earlier voice synthesis approaches?

Answer 60

They model sound from the audio level, not using symbolic models

Question 61

What are the main steps in putting together a complete AI music generation system according to the lecture notes?

Answer 61

1. Generate lyrics, 2. Generate music, 3. Extract melody, 4. Sing, 5. Mix the singing with the backing

Question 62

What tool is mentioned for generating lyrics in the AI music system?

Answer 62

GPT-2

Question 63

What Python library is used to extract melody from a MIDI file?

Answer 63

mido

Question 64

What is a challenge in extracting melody from a MIDI file?

Answer 64

There is no standard MIDI channel for melody, unlike percussion which is always channel 10

Question 65

How is timing handled when extracting notes from a MIDI file?

Answer 65

By accumulating the time values of each message and printing the cumulative time with each note

Question 66

What format does the singing synthesis program expect for note input?

Answer 66

A list of notes with repetitions indicating duration, e.g. 'c,g,eee'

Question 67

Name three tools mentioned for mixing the singing with the backing track.

Answer 67

Reaper, Audacity, and librosa

Question 68

Who created Reaper, and what other famous software did they create?

Answer 68

Justin Frankel, who also created Winamp

Question 69

What is Audacity?

Answer 69

A free and open-source audio editor

Question 70

What are the basic steps for mixing audio using librosa?

Answer 70

1. Load the files, 2. Normalize/set levels, 3. Pad the shorter track with zeros, 4. Add the tracks together, 5. Write to disk

Question 71

What file format is mentioned for the final mixed output?

Answer 71

WAV (mix.wav)

Question 72

What popular chord progression is mentioned as an example for music generation?

Answer 72

D, A, Bm, G (referred to as a popular progression used in '4 chords' by Axis of Awesome)

Question 73

How is the quality of the generated melody assessed in the described system?

Answer 73

By listening to it and deciding whether to proceed or keep sampling (note: the lecture mentions this could potentially be automated)

Question 74

What Python script is mentioned for converting MIDI to a note list?

Answer 74

midi_to_note_list.py