This is IPT~ project page. IPT~ is part of the ERC REACH project.

IPT~ (Instrumental Playing Technique recognition) is a real-time deep learning system designed to identify and classify instrumental playing techniques from live audio input. Implemented as an external object for Max/MSP, it integrates seamlessly into live electronics and interactive music environments, enabling performers and composers to leverage machine listening as a compositional and performative resource. At its core, ipt~ uses compact deep convolutional neural network (CNN) architectures trained on annotated recordings of extended instrumental techniques (multiphonics, harmonics, col legno, flatterzunge, and many others), allowing the system to operate with low latency directly within a performance patch. The resulting recognition stream can drive synthesis, trigger events in score-following systems such as Antescofo, or feed generative agents like Somax2, opening a continuous dialogue between the instrumentalist's physical gesture and the machine's response.

• Install from Github : download last release

• Introduction to IPT~
• Demonstration
• Somax2 and IPT~
• Two more videos are coming..! "How to train your own IPT model" and "Advanced usage with Somax2 and Antescofo".

• GFD, extensive flute IPT dataset. Performers: Kanami Koga & Nicolas Brochec
• EG-IPT, extensive e-guitar IPT dataset. Performer: Marco Fiorini. Sound Engineer: Riccardo Pasini.

• Nicolas Brochec Advancing Human-Computer Interaction in Mixed Music: A Deep Learning Approach to Real-Time Instrumental Playing Technique Recognition, Musicology and performing arts. Tokyo University of the Arts, 2026. English.

• Nicolas Brochec, Jean-Louis Giavitto. Automatic Hybrid Following in Real-Time Mixed Music: A Case Study with Antescofo and ipt˜ for Flute Playing Techniques. International Computer Music Conference (ICMC 2026), May 2026, Hamburg, Germany.

• Marco Fiorini, Nicolas Brochec, Joakim Borg, Riccardo Pasini. Introducing EG-IPT and ipt~: a novel electric guitar dataset and a new Max/MSP object for real-time classification of instrumental playing techniques. New Interfaces for Musical Expression (NIME 2025), Jun 2025, Canberra, Australia.

• Nicolas Brochec, Marco Fiorini, Mikhail Malt, Gérard Assayag. Interactive Music Co-Creation with an Instrumental Technique-Aware System: A Case Study with Flute and Somax2. International Computer Music Conference (ICMC 2025), Jun 2025, Boston (MA), United States.

• Marco Fiorini, Nicolas Brochec. Guiding Co-Creative Musical Agents through Real-Time Flute Instrumental Playing Technique Recognition. Sound and Music Computing Conference (SMC 2024), Jul 2024, Porto, Portugal.

• Nicolas Brochec, Tsubasa Tanaka, Will Howie. Microphone-based Data Augmentation for Automatic Recognition of Instrumental Playing Techniques. International Computer Music Conference (ICMC 2024), Jul 2024, Seoul, South Korea.

• Nicolas Brochec, Tsubasa Tanaka. Toward Real-Time Recognition of Instrumental Playing Techniques for Mixed Music: A Preliminary Analysis. International Computer Music Conference (ICMC 2023), Oct 2023, Shenzhen, China.

IPT~ is born between Tokyo University of the Arts and IRCAM Music Representations Team. This project is supported by the ERC REACH (Raising Co-creativity in Cyber-Human Musicianship), directed by Gérard Assayag.

IPT~ created by Nicolas Brochec (Project Leader, R&D), Marco Fiorini (R&D), Joakim Borg (Development).

Thanks to Kanami Koga and Simone Conforti for their continuous expertise on the flute.

Thanks to Nicolas Souchal for its continuous expertise on the trumpet.

Thanks to Diemo Schwarz for developing the pipo module.

Thanks to George Lewis, Steve Lehman, and Joëlle Léandre for their exploratory usage of IPT~ in concerts.