Much like Charles’s hardware setup (see a pattern here?), I decided to use part of the touch screen as the keyboard for the computer.

Design

I roughed out the design using Inkscape. (Because unlike Photoshop or Gimp, you can easily move and resize shapes around after creating them).

I then recreated the design using JavaScript. I used Khan Academy’s coding environment* to do this because its live update feature made iteration and tweaking a breeze. Converting the Inkscape design to code using the Arduino IDE would have been a tedious process as I would have had to upload the sketch each time I wanted to see the effect of a change made.

In the JavaScript code, I used only basic functions that are supported by the Adafruit Graphics Library such as drawRect, drawText, and fillColor. The color scheme is based on this palette with tweaks here and there.

Below is the Javascript mockup. Try typing. Clicking the up arrow toggles the keyboard’s style.

I then ported the code to Arduino. I made this snippet to help convert the RGB colors used in the design to their equivalent 16 bit representations (since the Adafruit libraries use 16 bit color codes).

Here is the Arduino code. And here is a video of it in action:

The splotches you see near the end of the video is the debug mode in action (toggled by pressing the up arrow). In debug mode, a cyan dot is painted where a touch is detected. The random characters are unicode symbols not included in the font that comes with the Adafruit Graphics Library.

Touch detection

Touch detection is proving tricky. Specifically what to consider a keypress. My current solution is based on calculating the proximity from the last touch and using a low sampling rate. If the new touch value is too close to the previous one, I assume that the user has not moved the stylus intentionally (maybe they have slightly shifted the pressure they’re applying on the stylus or its position). I use a low sampling rate to increase the likelihood that the user will have intentionally moved the stylus when the next sample (touch position and pressure) is taken. A low sample rate however is a tradeoff between responsiveness and accuracy in favour of accuracy. I’m on the lookout for a more robust solution for touch detection.

void checkIfMouseMoved() {

  if ( tCount > 0 ) {

    mouseMoved = false;
  }
  else if ( p.x > p0.x - tMargin && p.x < p0.x + tMargin && 
            p.y > p0.y - tMargin && p.y < p0.y + tMargin )  
  {

    mouseMoved = false;
  }
  else {

    mouseMoved = true;
  }

  p0 = p; // store present position as previous
};

void advanceCount() {

  tCount += 1;

  if( tCount > samplePeriod ) {

    tCount = 0;
  }
}

*On the link, comment out the last line background(255) then press restart to use the keyboard.