DesireData

Mathieu Bou hard <matjuartengine. a> (Montral),

Chun Lee < hungoto10.org> (London)
O tober 21th, 2005 (nal version)

Abstra t
DesireData is a derivative of PureData with many goals, the rst of
whi h is to implement a true lient-server separation of the ore vs the
GUI. It is part of the devel_0_39 bran h of PureData, available as a
ompile-time option. This paper serves as an introdu tion to DesireData
and its design prin iples. It is primarily targeted at people who are familiar with the urrent development status of PureData; however, we hope
to attra t keen developers through the new features of DesireData.

Contents
1

Introdu tion

1.1

Prin iples
2.1

DesireData and PureData . . . . . . . . . . . . . . . . . . . . . .

On eAndOnlyOn e or the DRY Prin iple

. . . . . . . . . . . . .

Patterns (of de oupling)

3.1

3.2

Obje tive T l . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1

Classes

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.1.2

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.1.3

Instan e Variables

. . . . . . . . . . . . . . . . . . . . . .

3.1.4

Further Developments . . . . . . . . . . . . . . . . . . . .

3.1.5

Pra ti al Class Hierar hy

Client-Server

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.3

Model-View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.4

Observer-Observable . . . . . . . . . . . . . . . . . . . . . . . . .

3.5

Remote Pro edure Calls (RPC or RMI)

. . . . . . . . . . . . . .

3.6

User Interfa e Metaprogramming . . . . . . . . . . . . . . . . . .

Current status

Appendix A: Obje t Pointer Diagram

Introdu tion
1

DesireData, the name being derived from latin desiderata , is a proje t whose
main goal is to enhan e the everyday experien e of the PureData users and
developers. The most riti al hanges are those that happen in the internals of
PureData, be ause they enable developers to extend PureData with less eort
and even enable them to do things whi h otherwise would have been nearly
impossible. Some of those features are: an Obje t-Orientation extension to T l,
Model-View separation, Client-Server separation, an abstra ted RPC me hanism, et . DesireData is implemented as a ompile-time option to PureData's
devel_0_39 be ause it represents a almost omplete rewrite of the user interfa e odebase, and would ae t developers using the onventional PureData UI.
As well, it will mean that DesireData will benet dire tly from development
progress on that bran h. This is dierent from the original proje t, then alled
ImpureData, whi h Mathieu Bou hard developed between February and May
2004, in that a fully separate bran h had been reated (impd_0_37), whi h in
retrospe t was a real bad idea.
What does this mean to the users? I in lude developers here: it's user as a
role. Well, a more user-friendly front-end, with aids su h as lass browsers, lass
name ompletion, argument ompletion, tooltips on arguments/inlets/outlets,
ongurable look and feel, olourised onsole with hyperlinked error messages,
status bar, button bar, T l evaluator, zoomable anvas, and so on.

Some of

these features are implemented, some have been in impd_0_37 but need to be
reintegrated in the new ode, and some are pure vapourware (but have inuen ed
the design de isions of either impd_0_37 or DesireData).
This paper aims to present the design and related issues in the implementation of DesireData.

1.1

DesireData and PureData

The relationship between DesireData and PureData may not be obvious at

rst sight. It is therefore worth taking a few moments examining how the two
proje ts relate to ea h other. PureData, as ommonly known, is the proje t of

Miller Pu kette . Around version 0.35, a team of PureData developers started

3 so that they ould ollaborate together on extending Pure4

Data in a way that externals an't extend PureData. Also the CVS serves

a CVS repository

the purpose of entralising the externals so that they are easier to distribute.
Around version 0.38, Miller started using CVS dire tly. Within the PureData
CVS, there are two bran hes that are urrently being developed. The MAIN
bran h is mostly Miller's eort whereas the devel bran h is mostly the produ t
of team eort and is often onsidered being more exoti . To be pre ise, the

1 it is also a word in English. Merriam-Webster Online denes

desired as essential

desideratum

as something

2 http:// r a.u sd.edu/~msp/

3 http://sour eforge.net/proje ts/pure-data whi h was originally hosted on iem.at
4 also known as: external lasses, plugins, shared libraries

devel bran h is a tually one new bran h per release of PureData: devel_0_37,
devel_0_38, devel_0_39. Miller does not work on the devel bran h, whi h is
ontrary to what happens in most other proje ts, where all new features rst
start in a devel bran h, no matter who odes them, and a new bran h is reated
for ea h stable release, updated only with bugxes.

Prin iples

2.1

On eAndOnlyOn e or the DRY Prin iple

On eAndOnlyOn e (OAOO) a term that is used in the Smalltalk and Extreme

Programming ommunities. DRY (DontRepeatYourself ) is a term that is used
in the Ruby and Pragmati Programmers ommunities. They're essentially the
same: avoid ode dupli ation. Use bottom-up thinking to unify parts that are
the same. Modularise program parts in a way that separate the spe i things
from the generi things, so that the generi things may be merged together,
and so that the spe i things read more learly. A related on ept would be
Data-Driven Programming, whi h is also quite do umented.
This is the prin iple that motivates the redu tion of the two radiobutton
lasses, horizontal and verti al, to only one, for example. But that prin iple has
a mu h bigger impa t. The ode spe i to radiobuttons is 53 kB in regular Pd.
In ImpureData it's down to 9 kB. In DesireData it's be oming even smaller.
Be ause there are a variety of te hniques that an be justied by OAOO, there
are that many te hniques that an be used to apply the OAOO prin iple.
Some of OAOO's greatest strengths:
1. Merges bugs together so that there are less of them to x.

The best

debugging te hnique in the universe is to write less ode.

2. Merges features together so that there are less of them to update.

By

entralising behaviours relative to families of features you may more easily

implement a new feature.

Think of oding a new feature that has to

intera t with 10 existing features.

Now imagine oding a new feature

intera ting with only one feature whi h is the ommon part between the
10 features I just mentioned. That's the point.

Patterns (of de oupling)

3.1

Obje tive T l

A major tool for organising ode better (and thus enabling better OAOO) is
Obje t-Oriented Programming (OOP). An in arnation of it, alled Obje tive

T l, has been designed espe ially for DesireData (and was merely a dream when
the proje t was alled ImpureData). The user interfa e of PureData is made
using a popular s ripting language alled T l/Tk (as well as way too mu h C
ode).

T l/Tk is not Obje t-Oriented.

This auses some limitations in the

way that pro edures may be organised together. Well, almost so: in a remarkably short pie e of T l ode (no C required), T l an be extended with those
additional semanti s. Yes, all of Obje tive T l is written in T l.
Nota Bene: the name Obje tive T l was hosen with haste and navet, as

there are already a few shovelfuls of OOP systems for T l, and already at least
two are alled Obje tive T l. It doesn't matter mu h. We'll pi k another name
later)

3.1.1

Classes

pro lass_new {self {super {thing}}}5. For

textbox lass, we do lass_new textbox {view}.

A lass is reated by alling

example, to initialize the

This rst de lares a lass named textbox and also de ides its inheritan e.
There is a root lass alled thing.

A lass may inherit from several lasses

at on e, so the se ond argument to lass_new is a list of super lass(es). Then

pro _new also denes onstru tors CLASSNAME_new and

CLASSNAME_new_as whi h

are used to onstru t instan es of the lass using a generated ID or a spe ied
ID, respe tively. The IDs of obje ts are used as strings; we usually make them
hex odes.

3.1.2

Methods

The syntax Obje tive T l uses for alling methods (sending messages to obje ts)
may be des ribed as the subje t-verb- omplements syntax, also known as the

re eiver-sele tor-arguments syntax.

It is essentially the same syntax that is

used in almost all OOP systems.

pro def { lass sele tor arglist body}6 and of

the redenition of pro unknown provides the me hanism to all methods on
any instan e of any lass. First, pro def denes a pro ${ lass}_$sele tor
to be de lared. For example, alling def obje tbox draw will dene a pro
obje tbox_draw. Se ond, pro s unknown and lookup_method together nd
The ombination of

appli able methods for ea h message, so that you may now all a method of any
instan e of a lass through the following syntax: OBJECT_ID ACTION ARG1
ARG2 ...

In su h syntax, OBJECT_ID is alled the re eiver of the message

825ea50 init 450 500 +0+0 1

825ea50 with the arguments 450 500 +0+0 1 and the method
alled is anvas_init with 825ea50 inserted at the beginning of the argument
list. In this ase, 825ea50 must have been the ID of an obje t of lass anvas
or any sub lass of it whi h does not redene init. It is also possible to bypass
the lookup by alling anvas_init, but there's rarely a reason to do that.

and the ACTION is the sele tor. For example,

initializes obje t

5 In T l syntax, {} is a list delimiter, pro s are dened by alling the pro pro , whose
2nd argument is the list of arguments, in whi h ea h argument may be represented by a list,
in whi h the rst element is the name of the argument and the se ond is the default value.
See? it's

that

easy. ;-) Now just distinguish between dening a pro and alling a pro that's

dened like that.

6 pro def is intended to mimi pro pro ex ept an extra 1st arg has been added so the

argument-list is now the 3rd arg.

Methods sharing the same name in the same hierar hy may be hained using

pro super {args}, whi h nds the method in the hain it is alled from, and
gures out the next method in the hain to be alled. This allows sub lasses to
wrap methods dened in the super lass. The method lookup algorithm traverses
the re eiver's lass' an estors, depth-rst, pre-order, with ordered hildren (ea h
node has a list of parents), and dupli ates are (supposed to be) removed, keeping only the last instan e.

This is basi ally the same system alled mix-in

inheritan e that is used in Lisp, Ruby, and the diale ts of Smalltalk that have
multiple inheritan e.

3.1.3

Instan e Variables

In Obje tive T l, the  sigil is the prex whi h denotes instan e variables.
This makes typing variable names mu h qui ker ompared to what it was in
ImpureData. For example, instead of typing _($self:x), all we need to do now
is just x. Furthermore, this makes it easy to adapt the existing ode to work
with the new

di t feature

of T l 8.5, or hange it so that ea h obje t is stored

as a list instead. All that is required is to update the arguments of regsub that
handles the prex substitution (in

3.1.4

pro def).

Further Developments

I know that there is a language alled in rT l whi h is some sort of T l/C++

hybrid, but given that T l is lose enough to LISP, and that CommonLISP has
its obje t system implemented as a library, why shouldn't T l have the same?
Anyhow, in rT l, despite its age, is not popular enough to warrant its use. It's
usually not bundled with T l and usually not even on the same CD as the T l
pa kage in a typi al OS distribution, if present at all.

The benets of being

able to update one's own obje t system allows one to grow the system as needs
grow. The world does not end at C++. It doesn't even end at Ruby.

If we

have a use for them, we ould implement more exoti OOP features su h as:
lass-obje ts, meta- lasses, ree tion, delegators, around-methods, ontra ts,
point uts/advi es, multiple-dispat h, et .
Update: there are OOP systems for T l that support those features. We

will onsider one to be bundled with PureData, with a free enough li ense  or
maybe we an sti k with what we've got, to satisfy Mathieu's language design
addi tions.

3.1.5

Pra ti al Class Hierar hy

Currently, the lient-side lass hierar hy looks approximately like this:

thing
> view
|>
|>
|>
>

< observable
wire
canvas
scalar
box
|> textbox
|
|> objectbox
|
|> atombox
|
|> messagebox
|
> comment
|> numbox
|> radio
|> slider
|> bang
|> toggle
|> vu
|> dropper
> cnv

Whi h is almost the same as it is in the server-side GUI obje ts.

In this

diagram, a single-inheritan e hierar hy is assumed so that the hierar hy may be

drawn as a tree, and then the additional inheritan es are grafted on the tree.
Currently the only se ondary super lass in the hierar hy is

observable.

The hierar hy will of ourse expand as needs and opportunities arise.

3.2

Client-Server

In the urrent PureData, the ode that handles GUI elements mostly lies in the
server. For example, the lient sends ba k the mouse positions and then the
server gures out whi h obje ts needs to be sele ted and updated. As a result,
the ommuni ation between the server and the lient is very ongested and the
server is doing most of the work that the lient ould/should do.
Desiredata therefore de ouples su h a Client-Server relationship by implementing a Model-View design, removing the rendering ode from the server and
rewriting it as part of the lient.

3.3

Model-View

The Model-View design essentially refers to the separation of GUI obje ts into
a non-GUI (Model) part and a GUI part (View). The Model lives in the server
while the View lives in the lient. This Model-View implementation is similar
to the standard Model-View-Controller[3 design but with two main dieren es.
First, the View and Controller in DesireData refers to a single entity. Se ond, a
opy of Model is stored in the View. This is be ause that Model and View may
be far apart so that it's more e ient to a he the Model's data in the lient.
This ould have also been handled by opying ea h Model obje t on the lient
as separate from the View, but for today's needs, the benets of that separation
are not obvious.

3.4

Observer-Observable

To handle the ommuni ation between Model (on the Server) and View (on the
Client), an Observer-Observable[4 pattern is implemented, as in most implementations of MVC. Several instan es of the pattern are hained to ea h other
for several reasons.
First, ea h obje t is observed by its owner, so that noti es are aggregated
towards the root of a tree and then managed together using an ordered-set
(priority queue) that prioritizes updates, removes dupli ate updates, and delays
too frequent updates.
Se ond, there are two su h trees (per pat her window): the tree of models
organises the sending of updates to the views on the lient; the tree of views
organises the sending of updates to the Tk anvas. (We have not veried whether
any other systems do Observers the way we do; in fa t Mathieu has just hanged
the design of DesireData's Observers and at the time of this writing they're not
really implemented yet).
In the server, the updates get s heduled using one
pat her window.

t_ lo k

In the lient, it is instead done using

allba k per

pro after.

Both

me hanisms are fun tionally equivalent, it's just that one is a server-side feature
of Pd, and the latter is a feature of T l.

3.5

Remote Pro edure Calls (RPC or RMI)

Conventionally, the ommuni ation between Pd server and lient is via

onstru ting a tual T l ode for the lient. For example,

strings su h as  pdtk_ anvas_new

sys_vgui()

sys_vgui()
generates

.x82946 0 450 300 +0+0 1 that are evaluated as T l ode on the lient side, alling pro pdtk_ anvas_new. DesireData
repla es

sys_vgui with sys_mgui, whi h is alled like this: sys_mgui(OBJECT_ID,

sys_mgui(me, do_stu ,

sele tor, argtypes, arg1, arg2, arg3...); for example

s, x1, y1, name), where s means two oats and a string. This way, the
aller is no longer onstru ting T l ode strings and does not have to know T l
syntax.

As a result, it is possible to eliminate some bugs easily, su h as the

many that urrently plague the use of bra es, ba kslashes and even spa es (!)
within strings sent through

sys_vgui().

It is even possible to swit h to another

proto ol (e.g. OSC) rather easily.

pd_upload()

further abstra ts server-to- lient ommuni ations by only

making data updates and letting the lient gure out whi h methods have to
be alled and when. This is the part that sits between both Observer noti ation trees mentioned above. It opies the server-side obje t to the lient-side
using

t_savefn,

the very same interfa e that is use to save in the

*.pd

le

format. A faster upload me hanism may send only dieren es from previously
sent data by notifying only about attributes being modied instead of about the
whole obje t. The noti es are still sent towards a pat her window Model whi h
now also manages dirty-sets of attribute-keys within ea h entry in its priority
queue; noti es relative to a same Model obje t get oales ed by omputing the
set-unions of their dirty-sets; and when the time of

pd_upload()

omes, only

the new data is sent, whi h ould be X11-style (send a bit-set of keys, then all
the values) or keyword-argument style (alternate between keys and values).

3.6

User Interfa e Metaprogramming

Creating dialog boxes, element by element, is tiresome. It would be mu h better

if dialog boxes ould be reated from a des ription of a data stru ture. Well,
DesireData already implements that and uses it in GUI obje t property dialogs.

Current status

DesireData is urrently in its early stage of development and is not fun tional.
Obje tive T l has already been implemented along with parts of the rendering ode in the lient, while the Observer-Observable pattern is urrently being
implemented.

An alpha release ould be expe ted in the next 6 months. In-

stru tions to obtain and run DesireData are listed below.

To obtain the sour e les from PureData CVS:

export CVSROOT=:pserver:anonymous vs.sour eforge.net:/ vsroot/pure-data

vs login
vs -z3 he kout -r devel_0_39 pd
To ompile with DesireData enabled:

s ons desire=yes debug=yes wall=yes install

wish desire.tk
Note: the realtime option should be disabled by using  -nrt in the .pdr le.
To disable the GDB debugger, start DesireData by  wish desire.tk -nogdb.
To run DesireData, from the pd/sr dire tory:

Referen es
[1 T l by John Ousterhout, 1987; T l8, 1997.
[2 Tk, a GUI toolkit for T l.
[3 http://en.wikipedia.org/wiki/MVC
http:// 2. om/ gi/wiki/wiki?ModelViewController
[4 http:// 2. om/ gi/wiki/?ObserverPattern
http:// 2. om/ gi/wiki/wiki?PublishSubs ribeModel
http://en.wikipedia.org/wiki/Observer_pattern

Appendix A: Obje t Pointer Diagram