Introduction
Dr. Evans introduced the Virtual Audio Workstation in 2015, a bold reimagining of music production based on AI and the physical manifestation of music. His submission included a hand-built, unencumbered 3D display, including the first Volumetric Haptic Display (capable of creating haptic voxels in free-floating space). While the industry catches up, PRISM brings a substantial subset of those capabilities to modern DAWs. While PRISM has the optics of a commercial product, it is (at this time) used only by Evans’ production company.
Evans’ stand-alone Virtual Audio Workstation, custom built down to the circuit boards.
On the Record
It was first used commercially in 2013 on the #1-charing Flying Colors album, Live in Europe—and then in full on Live at the Z7 (2015). The original test was a 7-second dropout from a recording of Steve Morse’s guitar solo. PRISM correctly predicted what he would have played, and then created the precise audio that would been recorded. Upon hearing it, the 7-GRAMMY nominee observed that he could not tell what part of his solo had been replaced. Since then, PRISM has been continuously developed, and employed on tracks with dozens of prominent artists.
Three recent releases with PRISM, two of them Gold Records.
Noteworthy Events
PRISM converts audio tracks to MIDI tracks, uniting audio and MIDI into rich-data MIDI tracks. This simplifies many aspects of music production, unites previously-incompatible operations, and increases their power.
PRISM’s modelling of music as a context free grammar is informed by Schenkerian Analysis.
High on Data
The implementation of PRISM adds intelligent, high-level data to these MIDI tracks, imbuing them with semantic meaning. This enables users to edit and mix audio by directly changing the properties they want, instead of trying to first transform audio and MIDI to have these properties (and then alter them). It also facilitates the isolation and independent editing of complementary information pairs, such as sound and performance, and souce signal and ambience.
Evans’ Performance Restoration thesis used the relationship between cognitive musical representations and physical motor programmes as one of the mechanisms to inform performer’s musical intentions.
Natural Intelligence
PRISM also provides a strong, uniform and reliable foundation to deeply integrate artificial intelligence into traditional music production practice. And to provide the foundation for new methodologies. More advanced semantic meaning, and a single event-driven data type, provides explicit feature engineering, allowing the functions of artificial intelligence to be more directly controllable as user-level parameters. It also enables the introduction of new classes of higher-level machine tasks, and systemic use of mature Large Language Model data sets and algorithms.
The first taxonomy of musical performance errors.
Sound Theory
Audio to MIDI conversion requires the synthesis of sound that was never recorded. When two notes are pushed closer together, a new transition between them must be rendered, correctly. When notes are pulled apart, they expose regions of silence. PRISM generates this audio beyond traditional GenAI prediction—it requires authenticity, physical precision, phase-coherence. It also requires that the new information be the result of the note’s performance context—based on the notes surrounding it. In PRISM terminology, the new sound is called Theoretical Audio. And PRISM must render exactly the same audio, every time the preconditions are the same.
Theoretical Audio has been used on dozens of commercial releases with leading musicians.
A Suite in Five Parts
Version 2 of PRISM introduced a suite of five Instrument and Effect plugins that integrate tightly into DAWs. The suite transforms DAWs’ fundamental paradigms by recontextualising their existing functionality. This transformation is implemented while maintaining full compatibility with existing DAW practice—operations, workflows, audio plugins, mixing techniques and other production elements. Additionally, the plugins elevate and enhance many areas of traditional DAW use, solving and easing longstanding problems.
Sceenshots from PRISM v2.2’s five plugins.
Solitary Refinement
PRISM’s Engine plugin converts source acoustic audio tracks to Archetype Audio—a WAV file modelled specifically for DAWs. Regardless of the source material, Archetype Audio is completely isolated by instrument, separated by articulation, devoid of noise, extended in dynamic range, enhanced with additional detail, and removed of all ambience. Evans first deployed it on Flying Colors, Live at the Z7; Wikipedia documented that, “Critics place it as one of the best-sounding live albums ever made.”
A raw hi-hat track with loud enough bleed that foot-chics are inaudible. Archetype Audio renders them as if recorded individually in an anechoic chamber, at approximately double the original dynamic range.
Super Model
PRISM models each instrument, and each note, by re-synthesiing the deconstructed track data. This enables the real-time modification of both physical and artificial parameters, such as shell resonance and sustain time. PRISM models each instrument, and each note, by re-synthesiing the deconstructed track data. This enables the real-time modification of both physical and artificial parameters, such as shell resonance and sustain time.
An alpha version of PRISM’s Structural Editor for Acoustically-Recorded Instruments (SEARI), enabling direct user-level editing of the instruments used on an acoustic recording.
Time Travel Agent
Users can also edit retrospectively, before the recording was made. The microphones can be changed and moved, the instruments substituted or moved to a different room, the cymbal nuts tightened. Likewise, the nature of the performance can be altered—as an improved version of the original, or influenced by the style of a specific musician. The timing of notes is based on their perceptual onset, instead of MIDI’s traditional (and late) reference to note energy.
PRISM enables the microphones used for acoustic recordings to be altered and repositioned after the fact.
Architecture
PRISM’s deep integration requires no changes to the DAW. It creates an internal version of its advanced data structures for its own use, and serves a translated subset to the DAW, granulated according to the host. Therefore, PRISM-enabled projects save with the DAWs native project format, and support interchange formats such as AAF. PRISM’s native data is saved in an independent file that accompanies the project.
The hierarchy of PRISM’s Systems, Components, and Modules.
One for All
The future of PRISM leverages the LLM functionality enabled by PRISM’s event data translation. For example, a user’s performance musical style is encoded, and then applied to other instruments.
PRISM 2.5 generalises the ability for musicians to generate performances in their own styles on instruments they are not technically proficient with.Â
Or, a musician can see how the styles of other artists would sound mapped onto their own performance.
PRISM 2.5 also introduces “Personalities”, a library of performance styles of other musicians.
All for One
Artificial Intelligence can generate music that is indistinguishable from humans. This is a truth not tethered to time, but to Turing Decidability.
The human desire to create, though, is a function of our need to express ourselves. And the more someone—or something—else fulfils that need, the less it fulfils ourselves.
One of PRISM’s first use was to repair musical performance errors—not according to an objective reference, but to performers’ specific musical aims. To restore their original intention—what they would have played, and how they would have performed it. Dr. Evans received a PhD for first demonstrating this possibility, in 2015.
A spectral display of the world’s first Restoration Restoration, taken from Evans’ thesis.
PRISM’s current LLM capacity enables a generalised understanding of our artistic intentions, and to take that next step with us. Because at its heart, PRISM was created for one thing—the one commonality all musicians share throughout their lives—to be better. To become the best version of ourselves.
Evans introduced many elements of PRISM n on Flying Colors’ Live at the Z7.