The art of DJ has gained wide popularity. Today, a DJ is not just someone whose task is to mix tracks; a DJ is now a composer who creates new music. The art of DJing involves live sets where the main task is to mix tracks and arrange music, adding musical fragments and sound effects in real time. The art of DJing is about direct communication with the audience through live music.
Lighting has become an integral part of a DJ's life during sets. It is undeniable that lighting enhances the atmosphere and perception of a musical piece. Lighting can even dramatically alter the perception of the music itself. However, there is a flip side to the coin. If the lighting is not synchronized and does not correspond to the musical piece, it can have a negative effect. Therefore, the key aspect of using lighting in a musical performance is the seamless connection between the music and the lights.
Many lighting equipment manufacturers have started producing lighting controllers for mobile DJs, which allow them to control their lighting fixtures. All these controllers operate on the same principle: you need to write a sequence of commands to control your lighting devices and save this control program in advance. During the set, you can launch the visualization program for your show. To achieve maximum impact from the music visualization, it is necessary to create a separate lighting composition for each track. Audio features allow you to synchronize the lighting program with the music track in one way or another. Additionally, while operating the lighting controller, you can manually switch settings and control additional devices such as smoke machines and strobes.
But this approach has significant drawbacks:
- DJs have to constantly come up with new lighting sequences to control the lighting fixtures because using the same sequence repeatedly can annoy the audience, even when playing the same musical track.
- A unique lighting control sequence written for a specific musical track may not be suitable for another musical piece.
- Such lighting controllers are highly inefficient for live music. DJs are creative individuals, and during a performance, they need to focus on the music. The modern approach with such controllers distracts DJs from their main task. That's why in popular DJ sets today, lighting effects are often absent.
- Lack of a real connection between the created lighting patterns and the music flow. Synchronization mechanisms are imperfect and often simply don't work.
Manufacturers of modern lighting controllers for mobile DJs mainly focus on creating compact, modern designs in a new style. These controllers typically have illuminated buttons, encoders, and a graphical display, but there are no revolutionary innovations in these consoles. The only desire of the manufacturer is to present existing visualization principles differently and package them in a new way.
I believe that DJs need lighting equipment with intelligent capabilities that can independently create music visualization in real-time. Such equipment should allow DJs to fully concentrate on the music. The key feature of such a system should be a visible connection between the visual elements and the musical composition. The ideal scenario would be when the music itself generates the light show.
I have spoken to so-called lighting technicians whose task is to create light shows for concerts, and they assure me that creating such devices is impossible and that it is a creative process beyond the reach of electronics. Talented lighting technicians are artists in their field and create stunning light shows, but even in those cases, the shows are often one-time events. Furthermore, I have never seen a "good" synchronization of light and music, which is sometimes disappointing. Most often, such light shows consist of a combination of various lighting effects, running lights, strobes, all mixed together in the hope that somewhere, somehow, it will align with the music and look good.
For a DJ who uses live music in their work, using a pre-programmed composition approach is impractical, and hiring a lighting engineer is simply unrealistic. That's why we often see that the lighting used in DJ performances has little to do with the music.
Humanity has long noticed that music has the ability to evoke visual imagery in the human mind, and that's why light shows have become an integral part of any music concert or performance.
I have dedicated all my free time to a hobby that involves developing real-time music visualization systems. Initially, I built many devices based on existing schematics and created output optical devices within my financial means. But the result of these efforts always led to the same outcome: the newly constructed device would initially bring excitement with its variety of effects, but after a few days, it would no longer bring joy, and within a few weeks, interest in it would simply fade away. This effect is known as visual fatigue, which is due to the human brain visually memorizing a lot of information and requiring constant refreshing.
From my own experience, I concluded that the existing approach to building automatic devices that can visualize music in real time does not correspond to reality. New rules, theories, and technological principles are needed to offer a fresh perspective on music visualization. According to these principles, it would be possible to create devices capable of automatically visualizing music in real time without human intervention. Moreover, I want to emphasize that such devices should have algorithms that allow for the creation of visual arrangements for melodies, which would generate new visualizations each time the same musical composition is played, eliminating the effect of visual fatigue.
I am not a musician; I simply love listening to music, but I am a fan of music visualization. That's why I embarked on a path where I defined the fundamental principles that music visualization represents to me. I named these principles "Real-Time Music Visualization Technology" or, in short, RTMV-technology.
Basic principles
- In any musical composition, there are two components: melody and rhythm.
- Melody is a smooth element that resembles a wave, threading through the entire musical piece.
- Rhythm, or pattern, can include a solo instrument, the singer's voice, and of course, the rhythm itself. It is the dynamic component of the musical composition.
- The visualization algorithm should execute a "light arrangement" of the musical piece. What does this mean? It should create a new visualization each time the same musical composition is played. The visualization algorithm should be somewhat intelligent.
- The Output Optical Device (OOD) can be a screen or a set of various light sources positioned in space, both in relation to each other and the audience.
- The principle of symmetry is the principle of beauty. I came to this realization when creating output optical devices or determining the placement of lighting fixtures on stage. If the generated light patterns have symmetry, it is always perceived as beautiful. If you look at all the light installations, you will see that the principle of symmetry is present everywhere.
- The algorithm of the music visualization system should be fully autonomous. The algorithms should independently determine the style of the musical composition and create control sequences to manage the lighting fixtures accordingly.
- The frequency analysis mechanism should distinguish one note from another.
- The main requirement of RTMV-technology is the principle of creating light patterns. Light patterns should have a visible connection to the musical composition, which means that the visualization should be linked and synchronized with the music. In other words, the music itself should create the visualization. Each light pattern created by RTMV-technology should be derived from the melody and rhythm of the musical composition.
Based on these principles, I began developing algorithms and designing controllers. When designing such devices, the format of the digital data required for music visualization is determined by the light sources used in the light show. The most accessible light sources for me were LEDs. With the advent of LEDs with built-in drivers, they became the primary means of visualization. This does not mean that RTMV-technology cannot be used with modern stage devices. It means that it requires a description of additional control functions for a specific lighting fixture or group, and nothing more.
Initially, for music visualization, I intended to use a 2D screen, and the development of RTMV-technology was focused on this type of screen. When the development process reached a certain level, it was necessary to create a working prototype to showcase the idea for objective feedback. Obtaining feedback is an important step as it allows you to look at the results of your work in a new way and go beyond your subjective understanding of reality. However, it turned out that creating a prototype for the 2D version required significant financial resources that I could not allocate from my income. Without demonstrating a working device, finding an interested sponsor or lighting equipment manufacturer seemed impossible.
I decided to deviate from the chosen direction and create a series of music visualization devices, similar to sound level indicators (VU meters), in order to earn the necessary startup capital. However, none of the solutions I created satisfied me. I tried adding more lighting effects, but it always ended up with the same nagging issue of visual fatigue. In other words, after a certain period of time, the device would become boring, uninteresting, and unappealing. This contradicted the goal of my work. That's when I decided to apply a simplified version of RTMV-technology for a linear-type display. A linear display involves arranging uniform light sources in a straight line, and this variant became known as 1D visualization. The initial results of testing pleased me greatly. You see, at the beginning of developing RTMV-technology, I rejected the concept of 1D visualization, believing that it would not be able to create sufficiently effective light patterns. However, as the experiment showed, it is possible and can be achieved at a sufficiently high level of efficiency.
This project was named CLUBBEST. Prototypes of two types of visualizers, models M68 and M100, were developed and manufactured. They differ in the type and quantity of LEDs used, as well as the approach to the data visualization mechanism. The primary purpose of these models was for personal, home use. Personally, I intended these projects as decorative devices for listening to music on one's PC. However, the feedback showed that this visualization approach also interested music professionals. To popularize the concept, I started recording videos demonstrating the devices' real-time operation, aiming to gather more information about the perception of this new form of art. And I began receiving positive feedback. The most inspiring feedback for me was when a subscriber of my YouTube channel wrote, "I felt like I saw the music." Yes, that's the truth - I am developing this technology so that people can see music.
Take note of how well-known lighting equipment manufacturers approach showcasing new technologies or visualization systems. They advertise and create promotional videos in an interesting way. In their videos, they provide extensive and sometimes even tedious descriptions of the new device, its unique features and capabilities, the number of new connectors, beautiful buttons, convenient encoders, and now, even OLED displays. They talk about how many lighting control sequences can be programmed into this new device, without mentioning the additional hours of work required. And most of the time, there are no real video demonstrations of the device's operation with lighting fixtures, or at best, a maximum of 10 seconds of demonstration. Why is that? It's because there's simply nothing worth watching beyond 10 seconds, as not much novelty has been developed for such devices. And for some reason, everyone remains silent about the fact that using this approach to visualization leads to the well-known effect of visual fatigue.
I believe that for a DJ, especially a mobile DJ, it is important to know how convenient, efficient, and user-friendly the newly acquired equipment will be. They need to consider how the use of this equipment will simplify their lives with lighting fixtures and not distract them from their primary focus on music. Moreover, this new device should aid them in their musical creativity. A DJ, after creating a musical composition, should be able to immediately see its visualization in real time. Unfortunately, the modern technologies offered by contemporary brands do not allow them to do so.
To evaluate the performance of the new visualization technology and demonstrate its objectivity, I create promotional videos by recording the entire musical composition. Please note that I don't edit the videos to include only specific moments. I aim to provide information about the real-time operation of the music visualization mechanism, showing the visualization results throughout the entire musical composition and highlighting the connection between light and music. Even in live broadcasts, I try to show a 5-10 minute segment of video to assess how effective a DJ's performance would be if accompanied by visualization. I record visualizations with different musical styles to demonstrate how the technology handles different types of music. Music is meant to be heard, while visualization is meant to be seen. The visualization should be of such quality that it gives the impression that the sound is coming not only from the speakers but also from the light itself.
CLUBBEST is built on the principle of "Plug and Play." You only need to feed the audio signal, and RTMV-technology takes care of everything else. CLUBBEST does not have a single adjustment knob. It automatically identifies the musical style and generates visual effects synchronized with the music on the available output optical device. In the CLUBBEST project, everything is simplified, with the light sources having consistent characteristics and a fixed quantity. Therefore, the algorithm is simplified and designed specifically for this configuration of the output optical device.
It's important to note that CLUBBEST is a highly simplified version of RTMV-technology, but the prototypes already developed can serve as a foundation for creating a range of serial products for mobile DJs and individual consumers to enhance their music listening experiences.
Furthermore, these systems that utilize real-time music visualization have a wide range of applications in clubs, cafes, bars, and restaurants where music is played and the venue owner wants to add dynamic lighting to attract visitors' attention.
Approaching the use of RTMV-technology for controlling stage lighting fixtures, the resulting lighting console would have one main feature that distinguishes it from modern consoles: it won't have a tedious programming mechanism for lighting control sequences. RTMV-technology is about music itself creating the light visualization.
The initial configuration of such a lighting console may involve visually describing the lighting fixtures to be used and their spatial arrangement. Currently, RTMV-technology allows for 1D and 2D arrangements of lighting fixtures in space. Theoretically, a 3D variant is also possible, but I have not yet defined those requirements for myself.
For a user (DJ), how would the connection of lighting fixtures look? You simply need to select the type of symmetry used in the visualization, specify the type (brand, model) of the light sources, and indicate their spatial positions according to certain rules. That's all for the setup. For more advanced users, there will be additional options for customization, such as limiting the rotation angle or maximum brightness of the light source, and similar parameters.
The preliminary configuration for RTMV-technology is necessary for the system to understand which control parameters need to be generated for managing the connected lighting fixtures. After that, the user simply connects to the audio source, and the light show begins.
Modern music visualization technology involves the process of pre-creating and programming a light composition for music, taking into account the lighting fixtures used. I believe this is a separate direction intended for lighting artists and represents a distinct form of art with its right to exist and develop.
However, it's time for a new form of art - the art of algorithms and electronics. This kind of music visualization art envisions a scenario where composers, while creating their musical compositions, also determine the accompanying light show and the visual representation of their musical works. But to achieve that, they need entirely different music visualization tools.
Now is the time to develop and create so-called intelligent real-time music visualization tools. Here's one opinion that recognizes my point, providing further confirmation that the time has come:
After watching the previous video, click here to see how RTMV-technology works, and you will fully agree with me - The time has come!
Catcatcat Electronics, Dnipro, Ukraine, 2023