Live visual development with TinkerForge

TinkerForge is an excellent and easy to use open-source hardware solution for physical computing. Praxis LIVE is an open-source visual development environment, particularly (not exclusively) designed for working with audio & video, with the ability to run projects from the command line or as standalone (NetBeans runtime) applications. So, what happens when we bring these two worlds together?

This is best watched in full-screen HD!

Experimental support for a few TinkerForge components has been in Praxis LIVE for some time.  However, in the weeks leading up to the last release, these were tidied up and expanded.  There is now a concerted effort to complete these bindings, and promote Praxis LIVE as a (maybe the?) visual development environment for TinkerForge. Praxis LIVE is great for working with other media – use an IR sensor to control an OpenGL filter, shout to switch on a light, or use an OSC controller on your Android phone to control a servo.  It’s also fun for just working with TinkerForge by itself.

Tinkering with Praxis LIVE

Praxis LIVE mixes a familiar visual dataflow model with a lock-free (potentially distributed) architecture loosely based on the actor model. It is designed around the central concept that everything is editable live, providing instant feedback without interrupting your creative flow, while the architecture shields you from issues relating to concurrent processing (eg. TinkerForge, audio, video, OSC, resource loading, etc.).

Individual Praxis LIVE components have a superficial similarity to JavaBeans, except that all communication happens asynchronously – the visual editor based on the NetBeans platform actually reuses and adapts various features that were originally designed for working with JavaBeans to work with this model. In addition, Praxis LIVE is built with the realisation that no component-based architecture can meet all needs, and allows for custom coded components using a simple Java API – keeping with the edit everything live philosophy these are compiled in-place allowing you to build and adapt (and fix!) the code as you work.

The bindings in Praxis LIVE are different to most, if not all, of the existing ways of working with TinkerForge in that they are both live and stateful – essential to being able to build up a patch on-the-fly, then save it and run it as you want. Each TinkerForge brick or bricklet type is mirrored by a Praxis LIVE component. These components act as proxies, mirroring the state of the underlying brick(let). Only one component can be connected to any particular brick(let) at a time. The components also encapsulate the principles within the Rugged Approach page on the TinkerForge website – eg. remove the USB cable to add another bricklet, and when you reconnect your patch will continue to run as it was before.

Quick start

This post is aimed as an overview of Praxis LIVE and TinkerForge, and how they interact, not necessarily a full tutorial, so this will be a very quick overview in getting started.  You’re recommended to also watch the video above, read the Praxis LIVE manual, etc.

1. Make sure you’ve got some TinkerForge bricks and bricklets connected to your computer (and working with the Brick Viewer).
2. Within Praxis LIVE, create a new project (toolbar button or popup menu in Projects window).
3. Once the project is created, use the popup menu on the project to create a new TinkerForge patch. The new file’s patch window will open automatically, and you will notice the TinkerForge root appear in the Hub Manager.
4. Start the TinkerForge root using the play icon in the newly opened window’s toolbar. NB. if you’re not using the default USB setup, you will have to open the root configuration window (using the button next to play) and set the correct IP address and/or port.
5. Using the palette, find the component that corresponds to your brick / bricklet and drag it into the open patch window.
6. Double-click the component to open the editor. Set the UID of the component (if you know it) or just use <auto> (see below). Check that the connected control is showing true. You can now interact with your component.

You can also explore the examples created for the video. Make sure to restart the hub (using the red circular arrow) between examples – you will be given the opportunity to save any changes you make.

Some useful additions (and a few things missing)

The component bindings for TinkerForge within Praxis LIVE have some extensions to the underlying API in order to ease working within the wider Praxis LIVE ecosystem. For the same reason, a few things have been deliberately (for now at least) left out.

• UID: <auto> All TinkerForge brick(lets) have a unique identifier. The various language bindings require this to be specified, however this is awkward when trying to create simple patches or shareable examples. As an alternative, the Praxis LIVE bindings support the value of <auto> being given to the UID property. This will link to the first available brick(let) of the correct type. Multiple components of the same type will link in enumeration order, which assumes stability of enumeration – large, complex patches are still recommended to specify the UID of brick(let)s.
• Normalization. Many of the TinkerForge brick(let)s provide values in usable, real-world units. However, for some such as the Rotary Poti or Joystick these values are somewhat arbitrary – an optional normalize property on these components changes the output to be between 0 and 1, useful for connecting to many other Praxis LIVE components. Alternatively, values can be scaled to any other range using a core:math:scale component. It should also be noted that the Servo brick currently only accepts input in a normalized range – input in degrees could be accomplished using core:math:scale again if necessary.
• Button mode. Various TinkerForge bricklets have buttons on them (LCD, Joystick, etc.) The button mode property switches between sending the value of the button as a boolean (true/false) value or as a signal when the button is pressed. The difference between these can be seen in the first two examples in the video, where value is used to control the backlight and signal is used to trigger audio.
• Invert Input. There is a property on the IO16 binding to invert the incoming input value by default. This makes it easier to have buttons triggering things, etc.
• Threshold. There is currently no support for threshold listeners on components. A similar result can be obtained by using a core:math:threshold component on the output.
• Servo animation. Support for animating servos is not currently included – use a core:timing:animator component on the input to achieve similar results.

Core components

Praxis LIVE provides a range of useful components that can be used in patches when working with TinkerForge. This is just a few that are particularly useful and used in the example patches in the video.

• Variable (core:variable). Stores a value and emits it when triggered. This is used in the second example for storing the various strings (HEL, LO, WORLD), as well as the empty clearing string, passed to the LCD.
• Array Iterator (core:array:iterator). Cycles through a set of values when triggered. Used in the third example to make a flashing LED using the values 1 0 (this could also be true false – numbers are automatically converted).
• Send (core:routing:send). Sends a message from a component in one root (patch) to one in another. Used in the second example to send the button signal to the audio sample players, and in the fourth example to send the audio levels into the TinkerForge patch.
• Delay (core:timing:delay). Delays a signal by x seconds. Used in the second example to clear the screen a short time after the button is pressed. NB. values are not queued – input will clear the current pending value.
• Timer (core:timing:timer). Emits a signal every x seconds. Can trigger various things. Used in the third example to drive the array iterator to flash the LED.

Live coding

As mentioned earlier, one of the key aspects of Praxis LIVE is the ability to use a core:code:custom component to run fragments of code where an included component does not satisfy your needs. The code within these components is compiled on-the-fly using the Janino embedded compiler – there is no extra overhead over any of the in-built components. These custom coded components can also be shared and reused within subgraph (.pxg) files, and a repository of these is in the works.

The fourth example in the video shows a use for a simple custom coded component in converting an input value from 0..1 into a string value for display as a level meter on the LCD display. The code for this component is here –

import net.neilcsmith.praxis.core.types.*;

String BASE = "------======****####";
int length;

public void setup() {

}

public void update() {

  if (d(1) > d(9)) {
    p(9).set(d(1)).to(0).in(1).easeOut();
  }
  int len = (int) (d(9) * 20);
  if (length != len) {
      length = len;
      send(1,PString.valueOf(BASE.substring(0,len)));
  }
}

This moderately complex example converts the incoming value into a proportion of the BASE string. It also makes use of animatable parameters so as to immediately reflect peak values but smooth return to 0, making for a better volume meter. This code will be even shorter when the rather obvious missing method to send a String is added!

Further details of the custom code API are on the Praxis LIVE website.

So, what’s next?

The TinkerForge bindings are not yet complete. Because of the stateful nature of the bindings, I’m wary of releasing bindings without access to the brick(let)s, or at least willing testers. A nice box of extra bricklets arrived yesterday though, so expect lots more bindings soon. The aim is to be mostly complete in the next month.

The TinkerForge integration has always got a lot of interest when I’ve been demoing Praxis LIVE to people, and the possibility of building useful tools visually without having to edit code has definitely sparked some interest from people interested in education. However, it also reminds me of an extremely souped-up version of an editor I used for a Programmable Logic Controller about 10 years ago – I wonder if a customised Praxis LIVE / TinkerForge combination might have wider uses?

Other aspects of Praxis LIVE will also see more active development in the near future. I’ve got a couple of quiet months, which is good for development if not for my bank balance. Sponsoring features is welcome you know – get in touch! 🙂