The equation I spent the entire afternoon working on takes a distance between 2 points seen by the Wiimote and uses triangulation to calculate the distance of the sensor bar from the Wiimote. Unfortunatley, because the distance between the points is measured in pixels and the distance between the sensor bar and the Wiimote is measured in ‘real’ units (i.e. cenitmetres or inches) there is a conversion or ’scale’ factor to be taken into account.

…and after all these scribbles, the final equation is as follows:

Where:
d = distance between Wiimote and sensor bar
s = distance between the 2 points as seen by the Wiimote (pixels)
Horiz FOV = Horizontal field-of-view of the Wiimote, roughly equal to 41° (to be measured)
Vert FOV = Vertical field-of-view of the Wiimote, roughly equal to 31° (to be measured)

The problem, as I mentioned is that the scale of the model has not been set, and d may need to be multiplied by a certain number to make the numbers work, but nevertheless the relationship between d and s should work out to be correct.

Yet more maths to follow!

Below is a crude(!) video taken on my old-as iPhone 3G, showing the tracking as I move the Wiimote around:

The next step in my development is to use triangulation to determine the distance between the Wiimote and the LEDs, in such a way that I can change the configuration depending on the fixed distance between the LEDs.

If i were just moving the Wiimote along a single axis towards the LEDs, then the calculations would be very simple, but nothing in my life is that simple! Not only is the positional and spacial relationship between the 2 objects 3-dimensional, but the rotation (pitch, roll and yaw) of the LEDs (or in this case, the Wiimote) is also 3-dimensional and it is here that lies the problem.

If the 4 light sources are close together, there could be 2 possible senarios:

• The LEDs are far away
• The LED array has been tilted so that the angle of incidence is very small

A successful and stable solution for this has not yet been created. Until now…maybe!

Because the IR camera on the front of the Wiimote is of such a high quality (1024 x 768px!) with a narrow viewing angle (~45°) it suffers foreshortening much more pronounced and acurately reproduced than that of a normal webcam. This means that if the sensor is far away from the source, the distance between the points will be small, but consistant between them. If the sensor is tilted, the distance between the 2 leftmost points will differ from that of the 2 rightmost. There is a value returned for the size of each of the light points, but this is inacurate and unstable due to the narrowviewing angle of the LEDs. I am currently looking into way to increase the viewing angle of the IR LEDs, which will help the accuracy and reliability of the values returned.

Finally, credit where credit is due. Thank you to Alex Harker for the updated ajh.wiiremote Max5 patch: http://cycling74.com/forums/topic.php?id=22500

As it turns out, I had forgotten a lot of what I had been taught in GCSE Maths, but I eventually got going and used my powers of substitution and diagram-drawing:

To see if the final equations worked, I created a Max patch to check the variable inputs for a 2-dimensional system. The full system will be the same, but with values for the y-axis as well.

Amazingly, the patch returns values that are believeable, but I will only know if it truly works when I get my homebrew Wii Sensorbar made up. I already have on my desk a Wiimote and I have downloaded the Max5 patch to get the data from it:

So expect to see some tech-tests uploaded soon!

My original idea was to create a tactile-less instrument (similar to the one shown below) that required little or no musical experience. My new idea is similar, but with a new and different application of hardware.

My original design required the use of ping, or ultrasonic sensors that used echolocation (making very high-pitched clicking sounds and listening for the echo) to determine the distance to a surface (similar to bats and dolphins) or lasers that used a similar method, but with light instead of sound.

I discovered that there were a few problems with the ping sensors, the most worrying being that because I needed 2, and unless I could wire in a switch so that alternated the source and receiver or created 2 different wavelengths, the 2 pings would interfere with each other. The ping sensors for the Arduino cost around £35-40 each and being a final year student, I am very very very low on cash and I decided to look for a cheaper alternative. The biggest problem however, was that if I choose to use my own sensors (which I have sitting on my desk now) I had no idea how to create the circuit to make them work.

So began my search into alternatives. Whilst experimenting with getting values from LDRs into Max5 via an Arduino, I discovered that covering up an LDR was very inaccurate in that my hand covered such a large area that the poor little LDRs resistance would read close to infinity whenever I went near it. However, when I was playing around with one, I tilted it down onto the desk and the reading was a lot more consistent. I suddenly realised that if I were to attach one to the end of each finger, I could use them to trigger notes and chords by moving my fingers up and down. On my white desk, the LDR was giving back values until it was only a few millimetres off the surface of the desk.

However, as with the hand-covering, the data retrieved from the LDR varied quite considerably. I came to realise that the only true and reliable way of using LDRs was as a switch, giving a value of 0v or >0v. But this property completely destroyed my analogue-based distance system, so once again, I had to put pen to paper and have a re-think.

I have 5 digits on each hand, making 10 in total. If each of them could give a value of 0 or 1, using binary notation I could have a potential 1,023 (2¹⁰ – 1) combinations of fingers, ignoring the combination 0000000000 (all fingers up). That is far more than I originally had planned and maybe too many. If I split my hands up however and treat them as 2 separate entities, I could have 62 (2(2⁵ – 1)) combinations, which would be more than enough, seeing as there are only 8 notes in a scale. Splitting the 2 hands up, I would have chords on the left and melody on the right, with 31 combinations of each.

Above is the map I drew to illustrate the combinations. The actual number of combinations in this configuration is 40 ((2³ – 1) + (2² – 1) + 2(2⁴ – 1)). The number I am using is less because I am restricting the notes playable by the 4 fingers of the right hand to 12, rather than the maximum of 15, purely because there are 12 notes in the chromatic scale and the thumb can then act as an octaver, adding 12 semi-tones to the current note. This means I can have 2 full octaves in the right hand and all 7 major triads, each with 3 optional variations on the left hand.

The other brilliant thing about this idea is its flexibility. In the future, I may look into the possibility of allowing the user to switch between playing notes and percussion. There is also the possibility of using more sensors for a wider range of possible sounds, instruments and dynamism in the music. Of course, the current design acts as a MIDI controller, so you can just alter the voice on the system to whatever you want, but you must remember that the mapping of the fingers would need to be made more intuitive for the instrument chosen.

The LDRs will be attached to a glove to allow the wearer to play the instrument anywhere, within reason. The position of the fingers in relation to one another is not important as it is with any tactile instrument. Given the right material, the wearer may even be able to play by tapping their fingers against their clothes.

Of course, my current working patch in Max5 was useless, so I set out to create a new one, stealing bits and pieces from the one that previously took me several weeks to carefully prepare. A work in progress:

Of course, the unlocked patch is as nasty-looking as ever:

There was of course, the problem I was dreading – I had to make 2 Arduinos talk to Max5 separately. This caused me a few headaches, but I overcame it. Lucky I bought 2, eh? :D

The only problem that I can foresee is that not many people can think in binary very quickly. I am having a lot of trouble trying to grasp it because I can only think either musically or binarily. Mixing the 2 is not easy for me! So as a fix, I am building LEDs into the fingers of the gloves and if I take the idea further, I will create a system that can flash the lights of the fingers you have to tap to play a particular song. I could also build in a function to allow the recording of your taps and play them back to you.

There are a few points that I believe make this instrument better than the one I had originally planned. With analogue inputs, it was hard to find a point at which to trigger the playing of the chord or note. There are of course, no such problems with digital; it’s either on or it’s not! Although not as technically outstanding, the new instrument should be more reliable, easier to reconfigure and more accurate.

This idea has already been produced (such as here and here), but every existing application only allows the user to play a single not per finger. My design allows a chord to be played with a single finger, making for many more possibilities.

Gloves off eBay – Check!
Arduino x2 – Check!
LDRs, LEDs and wire – Check!
Time – ARGH!

I wanted to create sounds that would make the movie clip look as though it was as close to the original as possible. This included adding in gasps and dialects from other Anime films and series’, machinery sounds, wind, rumbles and electrical sparks. Looking through the clip, it was a fairly dramatic and tense moment in the film, with characters feeling fear, jumping over fissures in the ground, climbing a surface to get to a friend in trouble and grasping their chest as if in pain.

For this, I needed a tense piece of music to play in the background. I didn’t want a rock, rap or R n’ B tune, but instead something more orchestral. I had a look around and started listening to several film soundtracks:

• Monsters Inc
• Transporter
• Blade
• Tomb Raider
• Kill Bill

After some deliberation, I decided on 2 songs, both very different in feel. I decided on:

• “Mission” from the Transporter OST - very loud, dramatic and hugely orchestral piece written in a minor key that fits the scene perfectly with huge amounts of tension
• “The Grand Duel” (Parte Prima) from the Kill Bill OST – a slow, orchestral piece with a thick string melody and a slow string accompaniment written in a major key which purposely juxtaposes the clip

From the word go, I ran with the first tune, which almost ’scares’ the viewer when put alongside the clip.

One of the main parts of the clip is someone climbing up a sphere-like object to reach a friend who appears to be on fire, or is aflame with energy. After watching Anime shows like Dragonball Z, in which the characters are glowing with energy during battles, I realised that the sound effect is essentially wind with a throbbing (or flanging) sine wave overlayed. I managed to find a sample of a strong wind on the Internet and it worked perfectly when placed over the clip.

I soon noticed that it sounded wrong when the wind was playing through the shot of the person climbing up the sphere to the man on fire, followed by a view from above the head of the man on fire, so I altered the frequency, balance and volume of the wind when the camera was further away from the source. Similarly with the sound of the earthquake/rumble at the start, I adjusted the volume slightly to properly match the camera position and angle. However, I knew from experience that the lower the frequency of the sound, the less directional it becomes. Therefore, the rumble only needed slight adjustment whereas speech and higher-pitched sounds needed to be tweaked more.

For the gasp of the old man, I had to record myself, and since I had no recording equipment on me, I had to record a Skype call to myself. I did the same with the gasps and the struggling. The machine-like clunk was so hard to get, I ended up with a sound nothing like I wanted. I was after an artillery sound, but a deep, bassy one. I found no such sound and was unable to replicate it at home, so the sound I ended up with was far too tinny.

In short I am quite happy with the final movie, although it could have been more creative. So as I say goodbye, I shall leave you with my take on a 56-second clip from the Anime film, Metropolis. Goodbye.