Dale, Colin, Farhan, Raymond – 4

We decided to build a Glove that controls the pitch of a song as it is being played. It was a real time interactive control on the music which allowed the user to create dubstep like effects on the music stream. The final system implemented the desired control well and mapped the up and down gestures to the pitch of the music. One improvement could be a more continuous mapping to the pitch, instead of the up and down motion corresponding to step shifts in the pitch, we could have a more fine control using the movements of the hand. We would also like to add gestures for controlling the wobble effect on the music (a left right twist of the wrist or such).

Instrument 1 – Scream Box (aka The Egg from Harry Potter):

The first instrument we designed was a box that starts making high pitched noise when you open it. As you try to cover the top with your hands, the pitch of the sounds goes down and to shut it off you close the lid of the box. The aim was to make a system that is a standalone musical object that has a very real-world mapping to it – you try to cover  as much of the box to make it quieter. The device uses a photo-sensor to measure how covered the top of the box is.

Instrument 2 – Touch Sensing Piano:

The aim was to recreate a piano like instrument with capacitive sensing aluminium foils as the keys. The mapping to create sound is natural, and the use of capacitive sensing makes the interaction feel natural and unobstructed.

Instrument 3 – Pitch Controlling Glove:

This instrument tries to map physical gestures to pitch control on music. The device uses an accelerometer to recognize motions of the hand and maps them to controlling the up of down pitch of a song. This can be used to recreate the “dubstep” effect on songs.

Final Instrument: Pitch Glove

We decided to refine the pitch glove because it afforded a very novel kind of interaction with the music and allowed control over a parameter of the music with gestures. The refining process was mainly dealing with mapping the accelerometer data to correct gestures and making the piping to the pitch control software work.

Parts List:

1. Arduino Uno
2. Processing Software

Instructions:

1. Download the code for the processing of the music.
2. Attach the Arduino either to your wrist using a strap, or hold it in your hand.
3. Enjoy the music move with your gestures.

Code:
Arduino Controller

const int groundpin = A2; // analog input pin 2
const int powerpin = A0; // analog input pin 0
const int xpin = A5; // x-axis of the accelerometer
const int ypin = A4; // y-axis
const int zpin= A3; // z-axis
void setup()
{
//initialize the serial communications:
Serial.begin(9600);
// Provide ground and power by using the analog inputs as normal
//digital pins. This makes it possible to directly connect the
//breakout board to the Arduino. If you use the normal 5V and
//GND pins on the Arduino, you can remove these lines.
pinMode(groundpin, OUTPUT);
pinMode(powerpin, OUTPUT);
digitalWrite(groundpin, LOW);
digitalWrite(powerpin, HIGH);
}
void loop()
{

if(analogRead(zpin) > 440)
{
  Serial.print("h");
}
else if(analogRead(zpin) < 280)
{
  Serial.print("l");
}
else
{
  Serial.print("n");
}

}

Music Controller

// Parts of this code were based on Sampling_03.pde from Sonifying Processing tutorial

import beads.*;
import processing.serial.*;
AudioContext ac;
SamplePlayer sp1;
Gain g;
Glide gainValue;
Glide rateValue;
Glide pitchValue;
Serial myPort;
float rate;
float pitch;
char serialVal;
int bufferSize = 0;

void setup()
{
  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);

  size(512, 512);
  background(0); // set the background to black
  stroke(100);
  line(width/2, 0, width/2, height);
  line(3*width/4, 0, 3*width/4, height);
  line(0, height/2, width, height/2);
  text("", 100, 120);

  ac = new AudioContext();
  try {  
    sp1 = new SamplePlayer(ac, new Sample(sketchPath("") + "music.mp3"));
   }
  catch(Exception e)
  {
    println("Exception while attempting to load sample!");
    e.printStackTrace();
    exit();
  }

  sp1.setKillOnEnd(false); // we want to play the sample multiple times

  pitchValue = new Glide(ac, 1, 30);
  sp1.setPitch(pitchValue);

  rateValue = new Glide(ac, 1, 30);
  sp1.setRate(rateValue);

  gainValue = new Glide(ac, 1, 30);
  g = new Gain(ac, 1, gainValue);
  g.addInput(sp1);
  ac.out.addInput(g);

  ac.start();  
}

void draw() {

  //read value from serial port.
  if ( myPort.available() > 0) {  // If data is available,
    serialVal = myPort.readChar();         // read it and store it in val
    bufferSize++;
  }

 if(bufferSize > 10)
 {
   myPort.clear();
   bufferSize = 0;
 }
 float dRateValue;
 if(serialVal == 'h')
 {
   dRateValue = 0.01;
 }
 else if(serialVal == 'l')
 {
   dRateValue = -0.01;
 }
 else
 {
   dRateValue = 0;
 }

  println(serialVal);
  // calculate the mouse position on screen as displacement from center, with a threshold
  // TODO: feed in the accelerometer values here
  //float dRateValue = ((float)mouseX - width/2.0);
  //float dPitchValue = ((float)mouseY - height/2.0);

  //if (abs(dRateValue) < 10) dRateValue = 0;
  //if (abs(dPitchValue) < 10) dPitchValue = 0;
  /*
  if(serialVal == 10)
  {
    dPitchValue = 0;
  }*/

  // adjust the rate and pitch depending on the X and Y position of the mouse, drawing a trace
  stroke(100);
  point(rate*width/10.0 + width/2.0, pitch*height/10.0 + height/2.0);
  rate += dRateValue;
 // rate = 2*dRateValue/width;
 // pitch = dPitchValue/height/10;
  stroke(255);
  point(rate*width/10.0 + width/2.0, pitch*height/10.0 + height/2.0);

  // print and set output
  println("Rate: " + rate + "; Pitch: " + pitch);
  rateValue.setValue(4.0*rate);
  pitchValue.setValue(-2.0*pitch + 1.0);
  sp1.setPitch(pitchValue);
  sp1.setRate(rateValue);

  delay(10);
}

void mousePressed() {
  rate = 0;
  pitch = 0;
}