PRISM
PRISM
Meet PRISM
PRISM evolves the practice of music production with a powerful new methodology for editing music and mixing sound. It reimagines the process from the user’s perspective, and reinvents it from a technical one. Yet it retains compatibility with existing tools and workflows, and preserving granular control. PRISM transitions DAWs to VAWs: Virtual Audio Workstations.
Year: 2020 Chad Wackerman & John Ferraro: Drums / Steve Morse & Blues Saraceno: Guitar / Sterling Ball: Bass
PRISM‘s “audio to MIDI to audio” gives users extraordinary new abilities. But it’s not a feature—it’s one of several underlying technologies used by PRISM to create the real magic. And bridge the gap between audio data and LLMs. Since 2015, PRISM has been used behind-the-scenes with dozens of iconic artists for whom performances are critical—ranging from Alice Cooper and Deep Purple, to Steve Vai and Crosby, Stills & Nash. Ten years later, PRISM is going public.
Meet PRISM
PRISM evolves the practice of music production with a powerful new methodology for editing music and mixing sound. It reimagines the process from the user’s perspective, and reinvents it from a technical one. Yet it retains compatibility with existing tools and workflows, and preserving granular control. PRISM transitions DAWs to VAWs: Virtual Audio Workstations.
PRISM‘s “audio to MIDI to audio” gives users extraordinary new abilities. But it’s not a feature—it’s one of several underlying technologies used by PRISM to create the real magic. And bridge the gap between audio data and LLMs. Since 2015, PRISM has been used behind-the-scenes with dozens of iconic artists for whom performances are critical—ranging from Alice Cooper and Deep Purple, to Steve Vai and Crosby, Stills & Nash. Ten years later, PRISM is going public.
Year: 2020 Chad Wackerman & John Ferraro: Drums / Steve Morse & Blues Saraceno: Guitar / Sterling Ball: Bass
PRISM‘s “audio to MIDI to audio” gives users extraordinary new abilities. But it’s not a feature—it’s one of several underlying technologies used by PRISM to create the real magic. And bridge the gap between audio data and LLMs. Since 2015, PRISM has been used behind-the-scenes with dozens of iconic artists for whom performances are critical—ranging from Alice Cooper and Deep Purple, to Steve Vai and Crosby, Stills & Nash. Ten years later, PRISM is going public.
Meet the Scientist
Evans performed his graduate work at Manchester Metropolitan University and Glasgow University. His first PhD thesis established the methodology of Performance Restoration, a set of technologies and techniques to repair musical errors in recorded music, and restore performers’ original intentions. His second introduced the Virtual Audio Workstation . Based on his research, he has engineered tracks for dozens of iconic artists, including with his own band, Flying Colors.
Meet the Scientist
Bill Evans performed his graduate work at Manchester Metropolitan University and Glasgow University. His first PhD thesis established the methodology of Performance Restoration, a set of technologies and techniques to repair musical errors in recorded music, and restore performers’ original intentions. His second introduced the Virtual Audio Workstation . Based on his research, he has engineered tracks for dozens of iconic artists, including with his own band, Flying Colors.
Foreseen Consequences
A problem manifests when raw audio is edited as MIDI. When notes are pulled apart, they expose Theoretical Audio—the parts of notes cut off by the next ones.
Note: The video’s “Next” button isn’t supported in Vimeo. Depending on server load, try the interactive version.
Year: 2020 Marco Minnemann: Drums / Bass: Lars Slowak / Keys: Frank Us
It is not enough to create believable audio here. Longer and shorter notes are performed differently. And human audio perception is highly-tuned to the contextual interaction of simultaneous instruments. PRISM incorporates these elements into Theoretical Audio—independently of ambience.Â
Sound Evidence
Traditional GenAI creates believable sounds, but we need a higher standard to accurately convey performances. If the audio never existed, though, how do we know if PRISM gets it right?
Year: 2022 Marco Minnemann: Drums
Foreseen Consequences
A problem manifests when raw audio is edited as MIDI. When notes are pulled apart, they expose Theoretical Audio—the parts of notes cut off by the next ones.
It is not enough to create believable audio here. Longer and shorter notes are performed differently. And human audio perception is highly-tuned to the contextual interaction of simultaneous instruments. PRISM incorporates these elements into Theoretical Audio—independently of ambience.Â
Sound Evidence
Traditional GenAI creates believable sounds, but we need a higher standard to accurately convey performances. If the audio never existed, though, how do we know if PRISM gets it right?
Year: 2022 Marco Minnemann: Drums
Sound Evidence
Traditional GenAI creates believable sounds, but we need a higher standard to accurately convey performances. If the audio never existed, though, how do we know if PRISM gets it right?
Year: 2022 Marco Minnemann: Drums
Note: The video’s “Next” button isn’t supported in Vimeo. Depending on server load, try the interactive version.
Year: 2020 Marco Minnemann: Drums / Bass: Lars Slowak / Keys: Frank Us
Sound Evidence
Traditional GenAI creates believable sounds, but we need a higher standard to accurately convey performances. If the audio never existed, though, how do we know if PRISM gets it right?
Year: 2022 Marco Minnemann: Drums
Found in Translation
PRISM translates a single audio recording into many forms. Users and VAWs can request specific representations of a recording, as either rendered data (e.g. audio or MIDI track) or the tools to create one (e.g. virtual instrument or ambience profile).
The PRISM workflow begins with raw audio files from a recorded performance. PRISM is designed to handle any type of audio file (e.g. monophonic mixes or individual tracks) at any audio quality. (In practice, results vary.) The Engine runs as either a stand-alone application, or from inside standard DAW plugin as an offline process.
Deconstructs audio streams into a database of PRISM Audio Atomic Data. These PRISM contain low-level components. Each set represents the data as a different type of building block.
AAD (Audio Atomic Data) is assembled into different types of pre-existing Prototypes. Each Prototype contains algorithms to create mathematical models specific to a performance. The models represent static data, and are tagged by higher-level metadata. Common examples include instruments, articulations, and notes.
Once the models are complete, PRISM analyses them to build semantic representations of the performance. This process is informed by grammatical and semantic (e.g. Schenkerian) analysis, and builds hierarchical structures. This data can then be flattened to create temporal events lists such as MIDI files.
The PRISM Synthesiser renders PRISM models in real-time, as a standard MIDI virtual instrument. It can recreate the original performance, with the original sounds and ambience. It also supports continuous modification of many types of sonic, musical and physical parameters.
The output of the PRISM's virtual instrument may then be rendered to standard audio files within a DAW. They can contain PRISM-specific RIFF metadata, read by the PRISM Effect Processor (i.e., VST, AU, AU) for enhanced DSP.
Better Living Through Alchemy
PRISM solves traditionally hard problems in audio processing, such as isolating instruments from mixes, separating articulations from performances, removing ambience and excising noise. It also makes possible entirely new processes, such as intelligently repairing performance errors, and imbuing existing performances with the personalities of specific artists. It unites temporal audio with event-driven MIDI into a single, editable representation—and isolates previously-immutable pairs of complimentary sonic and musical properties.
3D Haptic Sound-Molding on the Prototype Virtual Audio Workstation from Evans’ VAW PhD Thesis.
PRISM’s abilities, and the VAW, transcend modern DAWs. PRISM’s Alchemy Engine makes them available now. It transforms traditional audio tracks into Archetype format—a WAV file representing the purest and most flexible form of the original. Specially formatted for DAWs, Archetype has characteristics such as no noise or ambience.
Natural & Artificial Flavors
In traditional DAW use, there isn’t necessarily a structure, theme or metaphor for workflows. It can be argued there is simply a conglomeration of tools, menus and options. To improve results and achieve new outcomes, you add more tools. Power and efficacy are correlated with complexity and entropy, and affordances are based on skeumorphisms without real-world counterparts.
While PRISM supports existing DAW processes, its Virtual Audio Workstation methodology builds workflows based on human experience. Users can import existing skills in other domains. One example is informed by the cognitive duality of how we perceive sound modification: changes in the physical world, and artificial (digital) constructs. Each approach brings with it a highly-developed set of abilities and affordances that can then be applied intuitively.
Traditional sound mixing can be characterised as changing how something sounds—after it already happened. It’s like putting a ship back into the bottle. This process is difficult and unintuitive, requiring significant skill and a range of audio tools.
PRISM‘s Transformer lets users change the initial conditions of a recording—so it sounded right when it was recorded. These are physical phenomenon, already deeply embedded in human cognition.
Year: 2024Â Drums: Mike Portnoy
PRISM can also create audio transformations that, while impossible in the real world, provide optimised solutions for sound engineering. In drum mixing, compressors and transient designers may increase a tom’s duration, and decrease its attack.
In this video, PRISM demonstrates superior solutions by stretched recorded toms directly as MIDI notes. And other PRISM features—such as an independent, per-note envelopes—are applied.
Revealing Performances
Due to limitations in recording technology, recording drums has never been direct. As a result, much of a drum performance may not be audibly reproduced. So much is lost that in popular music, drums are thought of as rhythm, not an instrument. PRISM unlocks these performances.
Year: 2020Â Marco Minnemann: Drums
Point of Know Return
PRISM‘s science and technology are the result of creative pull. Its features and functionality arose to serve artistic goals. While its efficacy reflects commercial enterprise, its philosophy reflects creative practice. Dive deeper into PRISM‘s ideas, and explore how its reimagines music production.
For any note created on an instrument, there are an infinite number of possible recordings that we would hear as accurate. PRISM asks the question, “On a given recording, why not replace the recorded notes with more optimal versions?”
PRISM creates an auditory illusion. It preserves our perception of the original sounds, while improving the definition of timbre and clarity of performance.
Year: 2020Â Marco: Minnemann
PRISM can extract the individual articulations in a performance, and assign them individually to an audio track or virtual instrument. Doing this requires PRISM to synthesise the portions of notes that were never recorded. In PRISM terminology, this is called Theoretical Audio.
Year: 2020Â Marco: Minnemann
PRISM advances mixing by introducing direct manipulation of sound. The limitations of temporality (the notion of time) are removed from the mixing process in favour of an event-driven model. With Theoretical Audio, engineers can highlight and make edits that require audio that was never recorded, and performances that never occurred.
Every note, of every drum and cymbal, automatically creates its own fader—for volume, or any other property the engineer wants to assign. Each fader provides the full frequency range of the instrument, even though it may not have been recorded that way (e.g. dynamic microphones). Since every cymbal is on its own track, and all ambience is removable, overhead microphones can be discarded. Articulation-based tracks enable engineering based on performance elements—drum rolls, strikes, foot chics, and ghost notes.
Note-level isolation allows engineers to use the drum and cymbal sounds, tuning and other musical elements that the drummer picked for the song—instead of using sample layers, EQ notches and other techniques to create a new drum sound.
PRISM enables the performance and sounds of a drum track to be edited independently, enabling each to be directly edited without affecting the other—and allowing both to be preserved to a degree generally not previously possible.
The term “mixed-down” implies that information is lost. PRISM is a constructive model—a gradient toward revealing performances, and preserving drummers’ original sounds.
When MIDI references the timing of a note, the associated virtual instrument generally uses the start of the note energy—when there is first measurable volume. But your brains perceive the onset of notes based on other factors which often occur after the initial note energy.
Note timing begins with the note's energy.
Timing begins when we perceive it.
One example of this phenomenon is legato strings and brass sounds. Since few sounds begin as a square wave, we hear the timing of traditional MIDI in DAWs as always being late. As an attempt to remedy this problem, drum and cymbal samples and sounds sometimes have the initial attack portions removed—but that leads to unrealistic drum sounds, particularly with crash cymbals.
PRISM times its events based on perceptual note onset—so the sounds begin playing slightly before the time-stamp in the DAW. Instead, the time-stamp never refers to (and shows, in the piano roll editor), when we perceive the sound to start. As a result, the timing of the PRISM virtual instrument is generally (and noticeably) more accurate than traditional virtual instruments.
Year: 2020 Marco Minnemann: Drums / Bass: Lars Slowak / Keys: Frank Us
Becoming Attractions
PRISM began life as HPAR (Harmonic Phase Analysis and Restoration), based on Evans’ hypothesis that musical performance errors were an objective phenomenon that could be identified and repaired by a mixture of procedural and machine-learning algorithms. And that the analysis that this required could also reused to enhance the audible performance clarity on the whole recording.
Evans successfully demonstrated it in 2015 with the Flying Colors concert video/album, Live at the Z7. Wikipedia conferred, “Critics place it among the best-sounding live albums ever made.” Evans was awarded a PhD for his discovery of Performance Restoration in 2019. From there, informed by the creative pull of his commercial music projects, Evans began applying his VAW (Virtual Audio Workstation) methodology to PRISM.
- Passive Performance Restoration
- Perceptual Clarity
- Noise Removal
- Proto-Isolation
- Point Phase Coherence
- Physical Modelling
- Predictive Audio
- Ambience Removal
- Detail Enhancement
- Articulation Separation
- Dynamic Range Enhancement
- Group Phase Coherence
- Archetype Audio
- Extended Duration
- Detail Enhancement
- Event Parsing
- MIDI Support
- Hybrid Modelling
- Mic Phase Alignment
- Retrospective Editing
- DAW Plugin Support
- Ambience Modes
- Ambience Restoration
- Group Phase Alignment
- Dynamics Modeling
- Component Modeling
- Algorithmic Simulacrums
- Note Mixing
- Personalities
- Ambience Extraction
- Native Binaural Codecs
- Note Onsets
- Secondary Resonance
- Articulation Transitions
- Generative Simulacrums
- Parameterised Encoding
- Guided Perform Restoration
- Generative Composition
- Performance Interpolation
- Actualisation
- Deep Encoding
- Error Normalisation
- Generative Performance
- Repertoire Generalisation
- Active Perform Restoration
PRISM’s near-future stratifies v2’s DAW implementation into a suite of discrete, modular software plugins. Functionally, ongoing research and development generalises PRISM’s existing feature set, leveraging PRISM 2’s LLM support for hierarchical event data, expanding the repertoire of instruments, improving a subset of existing capabilities, and providing greater operational autonomy.
Commercial Generalisation
The set of valid possible input data is increased significantly, making PRISM compatible with a wide variety of programme material. Components of this effort were completed for PRISM v2.
Active Performance Restoration (APR)
PRISM actively monitors recorded tracks, alerting users to potential performance errors. When flagged, users then have the option of fully-automated restoration.
Personalities
Re-renders existing performances of acoustic recordings in the styles of specific musicians. It also adds user-adjustable meta-performance parameters to change stylistic characteristics. Personalities were part of Evans' original Performance Restoration thesis; has applied it to artists such as Crosby, Stills & Nash.
Actualiser
By inheriting functionality from Personalities and APR, the Actualiser improves recorded performances in the end user's specific style—creating a better version of themself that is still authentic to them.
Instance Replacement
This feature enables users to change the instruments used on a recording while maintaining the resonance signatures of the original (i.e., changing the resonant body, not triggering a sample). It is an improvement over the implementation in PRISM v2.
Repertoire Expansion
Expands the unsupervised instrument set from drums to include guitar, piano-derived keyboards and bass.
Generative Interpolation
Predicts the performance that would have occurred continuously between pre-recorded performances. Evans first applied this for the album, Live at the Z7. Seven seconds of a solo by guitar virtuoso Steve Morse was severely corrupted during the live recording, and unintelligible. Evans replaced the seconds with a completely new performance (i.e., not copy and pasted from an existing recording). Morse stated that he was unable to tell which seven seconds he didn't perform.Â
Selling Out
Hear PRISM on tracks with iconic artists, engineers and producers, engineered by Evans. Live songs are from the original audio—no overdubbing. re-recording or sample-layering.
Steve Vai â–ª Steve Morse â–ª Mike Portnoy â–ª Neal Morse â–ª Alice Cooper â–ª Jay Graydon â–ª David Foster â–ª Van Romaine â–ª Albert Lee â–ª Sterling Ball â–ª Joe Bonomassa â–ª Angel Vivaldi â–ª Chad Wackerman â–ª John Ferraro â–ª Marco Minneman â–ª Jim Cox â–ª Steve Lukather â–ª Crosby, Stills & Nash â–ª Peter Frampton â–ª Elvis Cosstello â–ª Peter Collins â–ª Ken Scott â–ª Bob Ezrin â–ª Michael Brauer â–ª Mark Neeham â–ª Howie Weinberg â–ª John Petrucci â–ª Robert Zemekis â–ª Deep Purple
Sound Affects
Stay connected with PRISM, and get an inside look at the sessions from some of music’s most compelling artists. New technologies while they’re in the incubator. And fresh takes on how the science of music production affects us as engineers, producers, artists and humans.
To verify, you’ll first receive an email confirm you signed up. Our privacy policy is here.
Meet PRISM
PRISM evolves the practice of music production with a powerful new methodology for editing music and mixing sound. It reimagines the process from the user’s perspective, and reinvents it from a technical one. Yet it retains compatibility with existing tools and workflows, and preserving granular control. PRISM transitions DAWs to VAWs: Virtual Audio Workstations.
Year: 2020 Chad Wackerman & John Ferraro: Drums / Steve Morse & Blues Saraceno: Guitar / Sterling Ball: Bass
PRISM‘s “audio to MIDI to audio” gives users extraordinary new abilities. But it’s not a feature—it’s one of several underlying technologies used by PRISM to create the real magic. And bridge the gap between audio data and LLMs. Since 2015, PRISM has been used behind-the-scenes with dozens of iconic artists for whom performances are critical—ranging from Alice Cooper and Deep Purple, to Steve Vai and Crosby, Stills & Nash. Ten years later, PRISM is going public.