0 Overview

Project overview (approx 500 words)

Title:

Concatenative Synthesis With Sequence Generation Using Neural Networks

What is it:

• An attempt to construct a ML-based workflow
• modelling sequentiality
• context-aware, interactive granular synthesis

Why is it:

• inspired by recent advances in AI audio synthesis
• to increase the accessibility of AI-based creative tools, especially in context of technical requirements
• attempt to construct something at the overlap of existing techniques

How is it:

• Audio feature extraction and sequence modelling implemented in Python.
  • Methods:
    • Music Information Retrieval
      • Segmentation
      • sliding window analysis
      • extraction of spectral feature
      • MFCCs
    • audio classification
      • KMeans clustering
      • UMAP/DBSCAN
    • time-series modelling and generation:
      • GRU neural network
      • temperature sampling.
  • Frameworks: Tensorflow/Keras, librosa, numpy, scikit-learn, scipy.
• Concatenative synthesis based on the above (example).
  • Methods: granular synthesis, cross-app communication.
  • Software: Max MSP, gen~ DSP, OSC, UDP.