Documentation for the Code

Quick Find

• Index
• Module Index
• Search Page

Launcher

Enter point of the program.

App.SilentNotesOverFolder(folderTrain, folder, threshold=0.3, k_fold=5, quantization=24, maxOrder=20, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints=['pitch', 'length'])

Function used in The music of silence. Part II: Cortical Predictions during Silent Musical Intervals (https://www.jneurosci.org/content/41/35/7449) It computes the probabily to have a note in each natural musical silences (using only the duration/rythm dimension).

App.SurpriseOverFolder(folderTrain, folder, k_fold=5, quantization=24, maxOrder=20, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints=['pitch', 'length'], genuine_entropies=False)

Train a model (or load it if already saved) and evaluate it on the passed folder. Computed the surprise signal for each file in the folder

App.Train(folder, quantization=24, maxOrder=20, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints=['pitch, length'])

Train a model with the midi files contained in the passed folder.

App.Train_by_piece(folder, nb_pieces=20, quantization=24, maxOrder=20, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints=['pitch', 'length'], intialization='')

Train iteratively on the passed folder (can be also initialized with western data) and generate the prediction error for each piece along the training. This allows to look at the dynamics of learning during training.

App.checkDataSet(folder)

Function that check if the dataset is corrupted (contains duplicates). Does not delete automatically!

App.comparePitches(list1, list2, k=0.9)

Compare two list of pitches, with a criterion k

App.cross_validation(folder, k_fold=10, maxOrder=20, quantization=24, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints=['pitch', 'length'], genuine_entropies=False)

Cross-validate by training on on k-1 folds of the folder and evaluate on the remaining fold k_fold = -1 means leave-one-out

App.evaluation(folder, k_fold=5, quantization=24, maxOrder=20, time_representation=False, zero_padding=True, long_term_only=False, short_term_only=False, viewPoints='both', genuine_entropies=False)

Main function for the cross-validation

Class IDyOM

class idyom.idyom.idyom(maxOrder=24, viewPoints=['pitch', 'length'], dataTrain=None, dataTrial=None, stm=True, evolutive=False)

This module represent the entire model, this is what you want to interact with if you only want to use the model.

Parameters:

• maxOrder (int) – maximal order of the model
• viewPoints (list of strings) – viewPoint to use, cf. data.getViewPoints()

cleanWeights(order=None)

Delete all trained models and fix an order if given

getDistributionsfromFile(file, threshold, short_term_only=False, long_term_only=False, normalization=True, genuine_entropies=False)

Return likelihood over a score

Parameters:folder – file to compute likelihood on Returns:np.array(length)

getDistributionsfromFolder(folder, threshold, zero_padding=True, time_representation=False, short_term_only=False, long_term_only=False)

Return likelihood over a all dataset

Parameters:

• folder – folder to compute likelihood on
• zero_padding (bool) – return surprise as spikes if True

Returns:

a list of np.array(length)

getEntropy(model, context, short_term_only=False, long_term_only=False)

Return the entropy (genuine, not approximated) given a context

Parameters:

• model – the idyom model to use
• context – the sequence of previous notes

Returns:

entropy as a float

getLikelihood(model, context, nextNote, short_term_only=False, long_term_only=False)

Return likelihood of a not given a context

Parameters:

• model – the idyom model to use
• context – the sequence of previous notes
• nextNote – the note to compute the likelihood of

Returns:

likelihood as a float

getLikelihoodfromFile(file, short_term_only=False, long_term_only=False)

Return likelihood over a score with the genuine entropy computation (not the approx)

Parameters:folder – file to compute likelihood on Returns:np.array(length)

getLikelihoodfromFileSimplifiedEntropies(file, short_term_only=False, long_term_only=False)

Return likelihood over a score (using entropy approximation)

Parameters:folder – file to compute likelihood on Returns:np.array(length)

getSurprisefromFile(file, zero_padding=False, time_representation=False, short_term_only=False, long_term_only=False, genuine_entropies=False)

Return surprise(-log2(p)) over a score

Parameters:

• folder – file to compute surprise on
• zero_padding (bool) – return surprise as spikes if True

Returns:

list of float

getSurprisefromFolder(folder, zero_padding=True, time_representation=False, short_term_only=False, long_term_only=False, genuine_entropies=False)

Return likelihood over a all dataset

Parameters:

• folder – folder to compute likelihood on
• zero_padding (bool) – return surprise as spikes if True

Returns:

a list of np.array(length)

load(path)

Load a trained model

Parameters:path (string) – path to the file

mergeProbas(probas, weights, b=1)

Merging probabilities from different models, for now we use arithmetic mean

Parameters:

• probas (list or numpy array) – probabilities to merge
• weights (list or numpy array) – weights for the mean, should be get from normalized entropy

Returns:

merged probabilities (float)

sample(sequence)

Sample the distribution from a given sequence, works only with pitch and length

Parameters:sequence (list) – sequence of viewpoint data Returns:sample (int)

save(file)

Save a trained model

Parameters:file (string) – path to the file

train(data, preComputeEntropies=False)

Train the models from data

Parameters:data (data object) – data to train from

Class LTM

class idyom.longTermModel.longTermModel(viewPoint, maxOrder=1, STM=False, init=None, evolutive=False)

Module implementing the Long Term Model from IDyOM, this model contains several Markov Chains of different orders weighted by their respective shanon entropy.

Parameters:

• viewPoint (string) – viewPoint to use, cf. data.getViewPoints()
• maxOrder (int) – maximal order of the models
• alphabetSize(optional) (int) – size of the alphabet, number of viewPoints value to take account in

generate(length)

Implement a very easy random walk in order to generate a sequence

Parameters:length (int) – length of the generated sequence (in elements, not beat so it depends on the quantization) Returns:sequence (np.array())

getEntropy(state, genuine_entropies=False)

Return shanon entropy of the distribution of the model from a given state

Parameters:

• state (list or str(list)) – state to compute from
• genuine_entropies (bool) – wether to use the genuine entropies (over the approx) (default False)

Returns:

entropy (float)

getEntropyMax(state)

Return the maximum entropy for an alphabet. This is the case where all element is equiprobable.

Parameters:state (list or str(list)) – state to compute from Returns:maxEntropy (float)

getLikelihood(state, note)

Returns the likelihood of a note given a state

Parameters:

• state (np.array(length)) – a sequence of viewPoint data, cf. data.getData(viewPoint)
• note (int or string) – the interger or name of the note

Returns:

float value of the likelihood

getPrediction(sequence)

Returns the probability distribution from a given state

Parameters:sequence (np.array(length)) – a sequence of viewPoint data, cf. data.getData(viewPoint) Returns:dictionary | dico[z] = P(z|sequence) (float)

getRelativeEntropy(state, genuine_entropies=False)

Return the relative entropy H(m)/Hmax(m). It is used for weighting the merging of models without bein affected by the alphabet size.

Parameters:state (list or str(list)) – state to compute from Returns:entropy (float)

load(path)

Load a trained model

Parameters:path (string) – path to the file

mergeProbas(probas, weights, b=1)

Merging probabilities from different orders, for now we use arithmetic mean

Parameters:

• probas (list or numpy array) – probabilities to merge
• weights (list or numpy array) – weights for the mean, should be get from normalized entropy

Returns:

merged probabilities (float)

sample(state)

Return a element sampled from the model given the sequence S

Parameters:S (list of integers) – sequence to sample from Returns:sampled element (int)

save(file)

Save a trained model

Parameters:file (string) – path to the file

train(data, shortTerm=False, preComputeEntropies=False)

Fill the matrix from data

Parameters:data (list of np.array or list of list of int) – data to train from

Class markovChain

class idyom.markovChain.markovChain(order, depth=0, VERBOSE=False, STM=False, evolutive=False)

Module defining MarkovChain model and usefull functions for the project

Parameters:

• alphabetSize – number of elements in the alphabet
• VERBOSE (bool) – print some strange behoviors, for example if asking for unknwown states

getEntropy(state)

Return shanon entropy of the distribution of the model from a given state

Parameters:state (list or str(list)) – state to compute from Returns:entropy (float)

getEntropyMax(state)

Return the maximum entropy for an alphabet. This is the case where all element is equiprobable.

Parameters:state (list or str(list)) – state to compute from Returns:maxEntropy (float)

getLikelihood(state, note)

Return the likelihood of a note given a state

Parameters:

• state (np.array(order)) – a sequence of viewPoints such as len(state) = order
• note (int) – integer corresponding to the element

Returns:

float value of the likelihood

getMatrix()

Return the transition matrix between states and notes

Returns:transition matrix (np.array())

getPrediction(state)

Return the probability distribution of notes from a given state

Parameters:state (str(np.array(order))) – a sequence of viewPoints such as len(state) = order Returns:dictionary | dico[note] = P(note|state)

getRelativeEntropy(state)

Return the relative entropy H(m)/Hmax(m). It is used for weighting the merging of models without bein affected by the alphabet size.

Parameters:state (list or str(list)) – state to compute from Returns:entropy (float)

load(path)

Load a trained model

Parameters:path (string) – path to the file

sample(S)

Return a element sampled from the model given the sequence S

Parameters:S (list of integers) – sequence to sample from Returns:sampled element (int)

save(file)

Save a trained model

Parameters:file (string) – path to the file

train(dataset, reverse=False, preComputeEntropies=False)

Fill the matrix from data, len(data) should be greater than the order.

Parameters:data (data object or list of int) – pre-processed data to train with

Class Data

class idyom.data.data(quantization=24, viewpoints=None, deleteDuplicates=True)

Class that embed all data processing: parsing midi, representating viewpoints, …

Parameters:

• quantization (integer) – quantization, 16 means 1/16th of beat
• viewpoints (list of string) – Viewpoints to use, by default all are used (see data.availableViewPoints())

addFile(file)

Parse a midi file

Parameters:file (string) – file to parse

addFiles(files, augmentation=True)

Parse a list of midi file

Parameters:files – files to parse

addScore(s)

Parse a midi file and return an internal representation

Parameters:file (string) – file to parse

augmentData()

Augments the data with some techniques like transposition.

augmentRythm(threshold_fast=10, threshold_slow=24)

Augment data by playing the pieces faster or slower

availableViewPoints()

Return the list of available viewPoints

Returns:list of strings

dataToViewpoint(score, viewpoints)

Function returning the viewpoint representation of the data for a given viewpoint. We separate the computations for different viewpoints so it’s easy to add some. If you want to add viewpoints you just have to change this function.

Parameters:

• vector (list of int, or numpy array) – Vector to work with
• viewpoints (list of strings) – list of viewpoints

Returns:

dictionnary

getData(viewpoint)

Return data for a given viewpoint

Parameters:viewpoint (string) – viewpoint (cf data.availableViewPoints()) Returns:np.array((nbOfPieces, lengthMax))

getNote(viewPoint, name, t)

Return data for a given viewpoint, score and index

Parameters:

• viewPoint (string) – viewpoint (cf data.availableViewPoints())
• name (string) – name of the score (by default name of the file)
• t (integer) – index

Returns:

integer corresponding to the note (cf. data.intToNote())

getScore(viewPoint, name)

Return data for a given viewpoint and score

Parameters:

• viewPoint (string) – viewpoint (cf data.availableViewPoints())
• name (string) – name of the score (by default name of the file)

Returns:

np.array(lengthMax)

getSize()

Returns the number of exemples

getSizeofPiece(piece)

Returns the size of a given piece from its index :param piece: index of a piece

Returns:length of the piece (int)

load(path)

Loads a database from a previously saved pickle.

Parameters:path (str) – The path to the folder containing the data.

parse(path, name='database', augment=True)

Construct the database of tuples from an existing midi database.

Parameters:

• path (str) – The path to the folder to load (must contain midi files).
• name (str) – The name to give to the database object, optional.

parseFile(filename, name='database', augment=False)

Construct the database of tuples from an existing midi database.

Parameters:

• path (str) – The path to the folder to load (must contain midi files).
• name (str) – The name to give to the database object, optional.

print()

Prints the names of all items in the database.

save(path='../DataBase/Serialized/')

Saves the database as a pickle.

Parameters:path (str) – The path to the folder in which we save the file, optional.