To prepare the work session between Tomas Turine and Lisa Nelson, I prepared a OSC serialisation of all the objects available in the 3D world, including mouse and cameras, and built an interface in pd to visualise and use the data.
Tagged: Pure Data Toggle Comment Threads | Keyboard Shortcuts
Based on the patches started yesterday evening, the example.pd shows how PD can be used with its user interface to interact RT with the engine. Once the work is done, by commenting a #define in the main class, you switch to release mode, without changing anything to the patch!
Parts of the video:
- edition: puredata used with ogre3d
- release: libpd loads the pd patch inside of ogre3d
As explained in this post, the edition is based on an OSC communication between the two apps, it works with the basic puredata.
There is 2 VERY important objects in this patch:
- pobject_in: it can parse osc messages as well as internal messages (when used in release mode); it also create 4 internal senders pobject_in_trans, pobject_in_rot, pobject_in_scale & pobject_in_all; no need to be plugged to the pobject_in box, you can receive the messages anywhere in your patches;
- pobject_out: it can pack osc messages for the engine as well as internal messages (when used in release mode); it also create 4 internal receivers pobject_out_trans, pobject_out_rot, pobject_out_scale & pobject_out_all; no need to be plugged to the pobject_out box, you can send messages from anywhere in your patches;
On the C++ side, the PdEngine has to be set in edition or release mode to works properly with pd. The communication with pd happens through the same methods. The idea here is to make the two modes transparent to the user, even if the c++ guy has to know what he’s doing :).
Keep in mind it’s a demo and there is a lot of optimisation to do on the C++ level, but it gives a good idea of the integration of the 2 libraries.
And, yeah, i’ll have to check how to record the sound with ffmpeg on linux…
3D – puredata communication
Starting to work on the first external to enable an easy integration of puredata in the polymorph engine.
The idea is simple: you place a pobject_in object in your patch to receive position (translation), rotation and scale of the 3D objects.
In edit mode, meaning you have pd with ui and the game running aside, info will go through OSC, allowing a RT modification of the patch.
In release mode, the patch will be loaded by the application thanks to libpd.
The pobject_in will work seamlessly in the 2 modes: in edition, you just add an udpreceive object and link it to pobject_in. Once edition is done, if you want to be clean, you just remove the udpreceive. In release, the info will come through a special receive object, polymorph_pobject_in_RT, already included in the pobject_in.
LibPD being integrated since several month it shouldn’t be too hard to adapt the required methods on the C++ side. I’ll work on this tomorrow, I hope to have a cool demo for sunday!
Triggering sounds when there is a collision between 2 objects is now feasible with the code. A message is sent to pd with the name of the objet and a “contact” flag (left of the image).
It has been a hell of a fight to get extract “useable” collisions information out the physical engine. There are 3 or 4 ways to listen to them and for each collision you get a LOT of data:
- number of points
- localisation of these points on each objet, in local and world cooridnates
- impulse, frictions, etc
And, obviously, all this happens for all objects currently colliding…
The first version of the code will runs on friday evening. It is certainly not the final version. A more serious approach is on its way, but it was too complex (impossible) to finish for tonight.
And, the sound design is close from 0 🙂
Stay tune for more!
Today, visually, it is ULTRA minimal.
The two dots are 2 sound objects. The left one is muted and is related to pobject_3_object messages in the pd patch. The right one is on and is related to pobject_6_object messages in the pd patch.
In human language, this means that the 3d objects are directly connected to the sound engine. For instance, when the right dot moves, it sends a message named “moved” to the sound. In this case, the message triggers a short sound.
Soon with more!
A bit of architecture
Concerns sound sources located in the 3d environment.
On the C++ side, sound::system manages all the sound sources and knowns about the location and orientation of the cam. It also contains the general configuration of the mix (gain, number of tracks, etc.).
At each frame, the sound::system renders a serie of relations, representing the position of sources in the camera local coordinates (where are the sound sources in the user point of view, to be a bit simplier).
The sound::system being aware of the number of tracks in the sound engine, it is able to manage efficiently wich source to send to witch track. For instance, if the number of active sound sources is higher than the number of tracks, sound::system will order the closer or the most important sources to send to the sound engine.
On the #puredata side, the job is simplified. All tracks are similar and reacts to messages sent by sound::system.
Each sound source knowns about the sound file to load. This info is sent onnce when the source becomes active : activated status.
Sound::system is able to handle 2 types of communication.
- In edition, it sends osc messages to puredata, the program with graphical interface. This allows musician to work live on the sound.
- In production, it sends the messages to libpd internally.
This 2 modes implies using custom objects in pd that are able to use both modes.
Thanks to @yacine_sebti for his expertise in pd.
Currently in development in #peel.
Puredata addon for Ogre3d.
The first version of the OgrePD addon is ready in bitbucket. It uses SDL and pulseaudio to send the sound to the sound card. A huge advantage of pulseaudio, compared to the standard alsa in pd, is that the sound card is not kidnapped by the application.
An example will follow very soon.
Open 2 big worksites: #bullet and #pd.
Both of them are buggy, but for different reasons:
- bullet: the addon dates from version 1.9, and a lot of things have changed in the way ogre is managing resources. A cleanup is started, but there are important classes declarations that needs to be reviewed.
- libpd: the library is a marvellous to install, and the example-libpd worked after several minutes. The main issue here is to setup rtAudio in the proper way. The code provided in the library cpp example is not compiling. Require a closer look.