Research area · 2005–2019
Can we automatically discover how to describe a sound?
Audio analysis by artificial intelligence. Describing the audio signal through formulas — descriptors — requires knowledge of signal processing; the line addresses the automatic discovery of descriptors and the improvement, through search, of already-trained sound classifiers. It begins with Giordano Cabral's PhD, with Sony CSL's EDS, includes HarmIn's harmonic analysis and the sound classifiers for public safety at UFRPE, and a study on the progress of the field itself (2016).
01 · AI-based audio analysis: music information retrieval
Contributions
EDS in chord recognition · 2005–2006
Sony CSL's EDS (Extractor Discovery System) proposes to automatically discover audio descriptors. With François Pachet and Jean-Pierre Briot, Cabral carried out the first part of the validation: Automatic X Traditional Descriptor Extraction: the Case of Chord Recognition (ISMIR 2005, London; zenodo.1415701, hal-01416436) compares classical approaches with the system operated by an inexperienced user, building a guitar chord recognizer. The follow-up is Recognizing Chords with EDS: Part One (CMMR 2005, Pisa; LNCS 3902, Springer, 2006; doi.org/10.1007/11751069_17, hal-01336914). The companion work from the same year is Impact of Distance in Pitch Class Profile Computation (10th SBCM 2005; hal-01416428, PDF in the proceedings). It is the first work of the doctorate presented at an international conference.
Analytical descriptors for harmony and rhythm · 2007
With Luiz Velho, of IMPA, and with Pachet, Pierre Roy and Briot: Analytical Features to Extract Harmonic or Rhythmic Information (11th SBCM 2007; hal-01305977, PDF in the proceedings) and the technical report Some Case Studies in Automatic Descriptor Extraction (Rio de Janeiro, 2007).
HarmIn · 2007
A tool for analyzing harmonic progressions, applied to the work of Tom Jobim from the Cancioneiro Jobim, with the musicologist Robert Willey (University of Louisiana at Lafayette). The progressions become a graph of chord transitions laid out on the circle of fifths, with the thickness of each edge proportional to its frequency in the repertoire; a composer's recurring harmonic paths become visible. Analyzing Harmonic Progressions with HarmIn: the Music of Antonio Carlos Jobim received the best student technical paper award at the 11th SBCM (São Paulo, September 1–3, 2007; symposium page). PDF in the proceedings. In 2026 the tool returned as a portable skill that builds the transition matrix on the circle of fifths for any artist requested.
Audio as game input · 2008–2010
Mapping Sounds into Commands (AES 35th International Conference: Audio for Games, London, 2008) and The Acoustick: Game Command Extraction from Audio Input Stream (SBGames 2010), with Roberto Cássio Júnior.
Audio monitoring for public safety · 2010–2014
At UFRPE, the problem became detecting gunshots, screams, impacts, ambient sound and silence automatically from microphones. This was the subject of the CNPq Technological Development Productivity grant (2011–2014), of the CNPq project "Audio Monitoramento para Segurança Pública" (2010–2013) and of the FACEPE project "Classificação Automática de Áudio para Monitoramento de Espaços Públicos" (call 11/2011, 2012–2014). Out of it came the dissertation by Dalton Francisco de Araújo, Busca como sistema de apoio à melhoria de classificadores automáticos de áudio (UFRPE, 2012, FACEPE grant), and the programs registered as Audio Alerta (2012) and Audio Alerta para segurança privada (2013), with Márcio Dahia, Roberto Cássio Júnior, Ivo Frazão and Raphael Holanda. Audio Alerta later continued as a spin-off of D'Accord.
Evolutionary generation of acoustic features · 2014–2019
The productivity project "Meta-Busca em Classificadores de Áudio e de Dados para Segurança, Entretenimento e Educação" (2014–) addressed generating the acoustic feature instead of choosing it by hand. With José Antonio Alves Menezes and Bruno Gomes, the journal result is Feature Learning with Multi-objective Evolutionary Computation in the generation of Acoustic Features (Inteligencia Artificial, IBERAMIA, vol. 22, no. 64, 2019; doi.org/10.4114/intartif.vol22iss64pp14-35).
Measuring progress in MIREX · 2016
MIREX has been the evaluation arena for music information retrieval since 2005. With Ricardo Scholz and Geber Ramalho, the work compiles six years of results, proposes a configurable index of evolution trend by task, and discusses hypotheses for the stagnation of some of them: Cross Task Study on MIREX Recent Results: an Index for Evolution Measurement and Some Stagnation Hypotheses (ISMIR 2016, New York; zenodo.1416898). It is the only work in the curriculum on the meta-science of the field itself.
Chorus, transcription and podcasts · 2014–2021
Chorus detection based on intensity, with Renato Celso Santos Rodrigues (15th SBCM, 2015; master's thesis co-supervised in 2016); chord transcription post-processing, with Uraquitan Cunha (12th AES Brasil, 2014); predicting podcast popularity from metadata, content and audio, a master's thesis by Luiz Delando Santos Moreira Júnior (2021).
02 · AI-based audio analysis: music information retrieval
Where the work continues
The 2021 Chord Detection Challenge, with Moises.ai, and the course Computação Musical, in which music information retrieval is one of the four syllabus units.
