The Story of BLOCKS

I wrote “The Story of WATCHOUT” a couple of years ago now, shortly after leaving Dataton, with my gaze firmly in the rear view mirror. WATCHOUT was clearly a mature product, well on its way into adulthood (it recently turned 18). At that phase of a product’s life-cycle, it’s more about incremental improvements, marketing and management.

This is somewhat similar to the stewardship of the iPhone over the last couple of years, albeit on a different scale. The iPhone is still a fine product (I own one, and like it). It will likely serve its users – and its manufacturer – well for years to come. But there isn’t really that much more that can be done to improve on it in significant ways.

So one of my reasons for leaving Dataton was to allow me to build new products, starting at the ground floor. At Dataton I felt somewhat trapped by our very success. Should I have stayed, I would likely have been forever chained to my baby, since it’s such a core piece of the business there. The only way to truly break free was to start anew somewhere else. So I did.

Now at PIXILAB, I’ve had the great pleasure of being part of the creation of two new products; MOMENTS and BLOCKS. While they share many core ideas and technologies, they’re quite different from a user’s perspective, as well as how they’re sold and deployed.

This article is about how BLOCKS came to be, and what it now has become. I’ll get back to MOMENTS in a separate article.

The Power of Vision

Neuroscientists talk about the dominance of our sense of vision. They say that roughly half of our mental processing power is primarily used for vision or visual recall.

I’ve spent most of my professional life on various ways to use that insight. When I started working at Dataton, multi-projector slide-shows were still in their heydays. Slides eventually gave way to video playback devices, such as Betacam decks and laserdisc players, controlled in various convoluted ways by TRAX and SMARTPAX to create impressive shows. Then those disappeared too, made obsolete by the increasingly powerful computer. I came up with WATCHOUT, essentially doing the same old multi-projector thing all over again, but without the slides.

Then desktops and laptops gave way to phones, tablets, streaming devices and smart TVs. Displays are now everywhere. From the tiny ones we carry in our pockets to huge LED walls and projections. Some are interactive, some very impressive and others just informative. One thing that dawned on me during my last few years at Dataton was how few of those actually required something like WATCHOUT.

The Need for Control

When you want to use multiple image sources to create a presentation, you need a system to control and orchestrate it all. It doesn’t matter if those image sources are slide projectors, laserdisc players, computers or smart displays – the basic control system needs are pretty much the same. Furthermore, you often want to control more than just the visuals. Lighting, doors, audio, motors and general power management are just a few examples.

Control systems typically consist of four main parts:

User interfaces, allowing operators and visitors to control and interact with the system.
Programmable “logic”, for scheduling, decision making and generally calling the shots.
Displays, relays, sensors, dimmers, loudspeakers and other input/output devices.
Some kind of network fabric, linking all the pieces together.

Incidentally, my formal training was in industrial automation systems – something I’ve been able to put to good use on a number of occasions throughout my career.

Early Attempts

Some of my earlier control system attempts include the MICTOUCH programable control panel. The panel itself handled 1 and 2 in the list above. It had its own multi-tasking operating system and a quite powerful programming language called MAGIC. However, the I/O capabilities at its disposal were rather weak, and the “network” used to interconnect it all was terribly slow.

The next attempt was TOUCHLINK, which was part of the TRAX/SMARTPAX product generation. This was quite a step up in terms of user interface capabilities. It had a small color touch screen, not unlike a modern tablet or phone, but about 10 years before those appeared. It used TRAX, our Mac-based show control software, for programming.

But again it was held back by inadequate hardware, both in terms of its own capabilities and by the I/O system that accompanied it. It was quickly eclipsed by systems such as AMX and Crestron, having much richer eco-systems. Furthermore, the TRAX software wasn’t particularly good at more complex control tasks, often calling for conditions and logic that was hard to express in a timeline-based system.

A Fresh Start

After leaving Dataton in the summer of 2015, I took some time off to think about all this, and what it meant moving forward. Just as when I came up with WATCHOUT, the landscape had changed again:

Mobile devices, such as phones and tablets, are ubiquitous.
There’s a lot of capable yet low-cost “players”, as well as smart displays that don’t even need a player.
Pretty much all of these run modern web browsers supporting interactivity, graphics, video, audio and even 3D.
Fast networks are commonplace, allowing even the most demanding content to be streamed in real time.
Projectors, displays, sensors and other I/O subsystems all have converged on the Ethernet wiring standard and communication protocols, such as TCP/IP.
Server technology is mature and efficient. It can easily provide all the media streaming, interactivity and programmability needed.

So I started working on various bits and pieces along those lines, eventually coming up with what’s now PIXILAB BLOCKS.

Display and Content Management

Visual communication remains front and center for me. BLOCKS is a multi-display control and management system. It talks to the players driving the displays, or directly to smart displays. It can also control other aspects, such as power and source switching for showing live feeds. Since content is streamed to the displays in real time over the network, you gain a great deal of flexibility while also being able to use simpler (and cheaper) playback devices.

Standardizing on HTML5 browser technology as the baseline for interactivity and playback means we can target a wide variety of displays and devices, including phones, iPads and Android tablets, Brightsign players, ChromeOS devices, Intel NUC, and smart displays from many vendors.

On the content side, things got simpler too. As I focused on HTML5 for display and playback, content formats also became standardized; PNG and JPEG for images, web fonts and SVG for text and vector graphics, H.264 and H.265 for video, including 4k and streaming sources.

The fact that all content is streamed from the BLOCKS server also makes it fast and easy to update, re-purpose and re-schedule content. All you need is a web browser to update the server by dragging a new image or video into BLOCKS.

The Mobile Revolution

The single most significant event over the last decade in terms of display technology and interactivity is undoubtedly the arrival of the modern cell phone, along with iPads and Android tablets. Powerful media playback capabilities, paired with their personal and truly mobile nature, make them excellent vehicles for interaction on-the-go.

BLOCKS embraces this in two ways:

Using visitors’ mobile devices (or on-site loaner units) to provide “mobile guides”, integrating audio, additional text or links and – if desired – even video into the visitor’s experience. As the phone is a personal device, it can provide all this in the visitor’s own language, and in ways that adapt to the visitor’s location and movement.
Allowing human guides and other staff members to control displays, lights, sound playback and other room functions from their own portable devices – essentially turning those into very powerful and programmable remote controls.

MOMENTS (our other product) takes this interactivity even further by engaging the audience with game-like and social-media-style interactivity, including live posting of text and images, lottery, quizzes, voting and similar. This connects people to what’s happening on stage and to other displays, thereby turning them from passive spectators into active participants.

What all of those devices have in common is web technologies (HTML5, CSS3, JavaScript). This makes them a perfect fit for both BLOCKS and MOMENTS, as those products are entirely based on this standards-based technology. It is available everywhere, highly battle-tested and rock solid.

Networks Everywhere

Of course, none of this could have happened without reliable and ever-present networks. These come in three flavors, with different pros and cons.

Traditional, wired Ethernet. Very cost effective for installations. Super-reliable high bandwidth (up to 10 Gigabits per second). Can also supply power to many devices using Power-over-Ethernet (PoE). This is the preferred way to connect displays and other I/O devices in a BLOCKS system.
Local-area Wifi. Also very cost effective and quite reliable even for larger applications when using enterprise-grade Wifi systems from companies such as Ubiquiti or Cisco. Paired with a local BLOCKS server, this provides predictable media streaming and interaction, making it the perfect fit for mobile guides and visitor interaction.
Cellular connectivity (3G, 4G, LTE) is the foundation for MOMENTS, since it requires no additional infrastructure. (MOMENTS lives in the “cloud”, hence directly accessible from any mobile device anywhere in the world.) BLOCKS can also be accessed in this way, for use in city tours and similar wide-area applications.

The Intranet of Things

Yes, I know it’s usually referred to as the “Internet of Things” (IoT). But from my vantage point, connecting things using a local (preferably wired) network is often more relevant. Much of what you want to control or interact with in a system has no business passing through the Internet. Indeed, in most cases, the protocols used to interact with these devices predate the current slew of IoT gadgets – sometimes by decades. So both from a security and reliability point of view, approaching this as an “intranet” thing (accessible only on a local, secure, network) often makes more sense.

Displays and players need solid and predictable bandwidth for reliable video playback. Many displays also have additional control capabilities for turning power on/off, switching to alternative video inputs, etc.
Lighting fixtures used in professional applications have their own standards, which have been around for a very long time. For architectural lighting, DALI is a widely used control protocol, while for more performance-oriented applications, DMX-512 is still the lingua franca.
Switches and sensors are best entrusted to I/O equipment used in the manufacturing industry. These clip onto DIN rails, for use in fixed installations, and typically use the MODBUS protocol.

BLOCKS comes with built-in support for these standards. Even old stalwarts such as RS-232 and RS-485 can be connected using an Ethernet-to-Serial adapter

Multi-tiered Programmability

Sometimes, all you you want is a button or slider to control a light or open a door. BLOCKS caters to this simple form of programming by directly linking a button or slider on a panel to a function of a device. (This form of direct linking was also available in TRAX, in case you’ve been around long enough to remember.)

However, direct linking of a button/slider to a device function is really only applicable in the simplest of cases. For anything more complex, you need more expressive capabilities:

To perform multiple actions together.
In cases where actions should only be taken if a condition is met.
To repeat a sequence of commands.
To incorporate variables, possibly representing some system state.

In such cases, the task-based programming built into BLOCKS often fits the bill. This is an easy-to-use, drag-and-drop based, programming language that borrows its expressive syntax from JavaScript – the world’s most popular programming language.

Taking a cue from MICTOUCH mentioned at the beginning of this article, with its MAGIC programming language, tasks in BLOCKS can run concurrently. This is often required in real-world applications, where different parts of the overall system run independently and in parallel.

Advanced Scripting

Task-based programming is fine up to a point, and quite a lot can be done well by that method. However, when the requirements move beyond a certain point, the drag-and-drop style of programming tends to hit a ceiling. There’s a reason why almost all serious programming is done with “code”, rather than wiring up boxes. The “boxes” metaphor simply breaks down once you move past a few dozen boxes.

Here you must instead enter into the realm of “real” programming, using a “real” programming language. Even more importanyly, this is also the point where you bring in “real” programmers, since they know how to deal with large, complex systems in a well mannered way – avoiding the “spaghetti syndrome” that otherwise tend to hamper such efforts.

BLOCKS is somewhat unique in that it provides a third level of programmability, beyond direct linking of buttons and the dragging-and-dropping of boxes. This kind of programming – sometimes referred to as “advanced scripting” – uses a formal programming language named TypeScript. TypeScript is a superset of JavaScript, which is arguably the world’s most common programming language.

Using a programming language with such a well established pedigree means that hordes of programmers are familiar with it. That’s a great advantage when looking for talent, compared to other solutions such as AMX, Crestron or MEDIALON, which all have their own proprietary programming languages.

Another great advantage of “going with the flow” here is the excellent tooling support in the forms of code editors, IDEs, syntax checkers and compilers. All of this is available for free, from multiple sources. Much of this infrastructure is also “open source”, meaning that there’s a lot of free code libraries, tutorials and documentation out there that can be leveraged.

But the most important aspect of all this is probably that it works in ways programmers are familiar with. When building large systems, programmers often use features such as interfaces, classes, instantiation, encapsulation and modules. Common patterns such as events and promises deal with the asynchronous and “multi-tasking” nature of control systems. Such concepts are already familiar to professional programmers, steeped in modern web frameworks such as angular, react or node.js.

The three-tiered programmability provided by direct linking, drag-and-drop tasks and advanced scripting deals with the high-level “logic” of a system; the functionality explicitly called for by the client application. However, there’s another aspect of most control systems that is of equal importance, albeit being at the other end of the food-chain; the devices being controlled.

Controlling Devices

While BLOCKS comes with a range of device control capabilities out of the box, some of which have been mentioned earlier in this article, there’s the small problem of manufacturer-specific (and sometimes even model-specific) control protocols. On the hardware and networking side of things, most devices these days support the Ethernet and IP standards. Those that don’t (such as RS-232/485, DMX-512 and DALI) can generally be connected using a simple Ethernet-to-whatever adapter.

However, when it comes to the actual languages spoken by devices, they’re all over the map. I was recently involved with a medium-sized control system where the integrator specified only Christie displays and projectors. Great, I thought, assuming they would all speak the same language. Wrong I was, as Yoda would say. The displays were two different models, each with its own language. The projector again spoke a different language. And those weren’t just “dialects” – they were totally different.

Déjà vu! I’ve been here before. With the TRAX/SMARTPAX generation of products, I also had to deal with this situation. But at that time, the hardware was so underpowered that we had to write and support most device drivers as part of the product. This turned out to be quite unmanageable, considering the sheer number and variety of devices and control requirements at hand.

So this time I’ve hopefully hit the sweet spot with BLOCKS in this area. Here are the broad strokes:

Drivers are written in a proper and appropriate high level language (more on this below).
A flexible and documented plug-in architecture allows drivers to be written, installed and updated without interfering with normal system operation.
Leveraging an open-source model allows anyone with adequate programming skills to write and extend these device drivers.

This last point is likely the most important one. Due to the number of devices and control languages out there, and the speed at which new ones are introduced, it’s just not possible for a single company to handle them all adequately. The only solution is a community-driven “open source” approach.

The language used to write drivers is again TypeScript, the same as used for the system-level “advanced scripting” discussed above. This language, just as its JavaScript sibling, excels at dealing with the asynchronous nature of communication protocols. It also has built-in support for the string manipulation, parsing and bit-fiddling often called for, as well as the logic required to deal with device language idiosyncrasies.

By following a specific coding pattern, device functions can be exposed as readable and settable “properties”. This ties very nicely into all three tiers of the system-level programming discussed above, making such custom-designed device drivers first class citizens of BLOCKS. For instance, when writing a driver for a projector, you don’t want to expose the command strings sent to the projector at the system level. Instead, you want to expose higher-level properties that make sense at the system level, such as “power on/off” or “select input hdmi1/hdmi2/dvi/vga”, etc. Those are properties supported by all projectors, regardless of the specific language used to talk to a particular model.

Wrapping up

So there you have it. Hopefully a useful “bag of tricks” that will enable you to create amazing new things. I hope you’ll find it as fun to use as it was to build. Now, our main challenge moving forward is how to explain to you all what it really is. With WATCHOUT, that part was rather simple – it was “digital multi-image”. BLOCKS is more of a “Superman” thing:

Is it a multi-display and content management system (is it a bird)?
Is it an interactive mobile/audio guide system (is it an airplane)?
Is it a control system (is it a rocket)?

Yes! It’s PIXILAB BLOCKS!

Mike