║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
╟┐ ┌╢▐
║ The research project, nicknamed WiiAssist, focused on ║▐
║ adding more operating system user interfaces for people ║▐
║ without fine motor control in their arms or hands. This was ║▐
║ accomplished by developing a system where head tracking was ║▐
║ used to simulate the functions of a mouse. Head tracking ║▐
║ was accomplished by using the x and y positioning of an ║▐
║ infrared LED returned by two Wiimotes to calculate the ║▐
║ LED's coordinates in a 3-Dimensional space. The x and y ║▐
║ coordinates of the LED yields a mouse position on the users ║▐
║ monitor and allowed them to move the cursor around with ║▐
║ their head. ║▐
║ ║▐
║ DEFCON 17 Talk - Hacking the WiiMote to Help People With ║▐
║ Disabilities: ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ For this project, I worked on the Wiimote to PC ║▐
║ interaction. I used the c library, wiiuse, to convert the ║▐
║ Bluetooth messages into basic functions that provided my ║▐
║ algorithms with the information needed in order to ║▐
║ calculate a 3-Dimensional position for the infrared LED ║▐
║ attached to the user. After that, it was a matter of making ║▐
║ my outputs compatible with the Windows API in order to ║▐
║ control the cursor and simulate a physical mouse. Below are ║▐
║ a few diagrams that help explain the process. ║▐
║ ║▐
║ We were also featured on G4's 'Attack of the Show': ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ Calculations Overview ║▐
║ ║▐
║ This was our test setup. Notice the two WiiMotes set up to ║▐
║ the side of the monitor one on the left, and the other on ║▐
║ the right. These were used as cameras to keep track of an ║▐
║ infrared source attached to the user (shown in the second ║▐
║ picture), and they would relay the coordinates to the ║▐
║ computer. ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ WiiMotes are capable of reporting only 2-Dimensions for a ║▐
║ remote IR source, so by using two WiiMotes, we were able to ║▐
║ capture the third dimension, as shown below. It is also ║▐
║ important to point out that the WiiMote is capable of of ║▐
║ measuring the intensity of the LED and determine a third ║▐
║ dimension that way, but the results are much less reliable ║▐
║ than using two remotes. ║▐
║ ║▐
║ The image below gives a good mock-up of the typical setup ║▐
║ for our system, including measurements. The blue areas are ║▐
║ the areas that the WiiMote can see, and the darker blue ║▐
║ area is where both WiiMote view ranges overlap, so the user ║▐
║ must be within that area to be detected by both remotes in ║▐
║ order to produce a 3-Dimensional coordinate. Since the ║▐
║ WiiMote's resolution is 1024 by 768, we are able to scale ║▐
║ this and convert it into a measurement from 0 to 45 degrees ║▐
║ in order to detect where the infrared source is, since the ║▐
║ WiiMote's range is only 45 degrees. Below is the geometry ║▐
║ used to calculate the 3-Dimensional coordinate, and the ║▐
║ symbols should match up with second and third images. ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ ║▐
║ x_2 = (width) / ((tan(θ_2) / tan(θ_1)) + 1) ║▐
║ ║▐
║ To find x1, subtract x2 from the width: ║▐
║ ║▐
║ x_1 = width - x_2 ║▐
║ ║▐
║ Lastly, to get the distance x away from the left edge of ║▐
║ the monitor: ║▐
║ ║▐
║ x = x_1 - (width-screensize) / 2 ║▐
║ z = x_1 * tan(θ_1) ║▐
║ y = x_1 * tan(θ_y) ║▐
║ ║▐
║ Windows API Communication ║▐
║ ║▐
║ In order to simulate the mouse movement, every time the ║▐
║ user's coordinates updated, we would send that data to the ║▐
║ Windows API using the SENDINPUT function. An example of how ║▐
║ this is done (from our code) is below. ║▐
║ ║▐
║ INPUT *key; ║▐
║ key = (INPUT*)malloc(sizeof(INPUT)); ║▐
║ key->type = INPUT_MOUSE; ║▐
║ key->mi.dx = pointArray[0].x; ║▐
║ key->mi.dy = pointArray[0].y; ║▐
║ key->mi.mouseData = 0; ║▐
║ key->mi.dwFlags = (MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE); ║▐
║ key->mi.time = 0; ║▐
║ key->mi.dwExtraInfo = 0; ║▐
║ SendInput(1,key,sizeof(INPUT)); ║▐
║ free(key); ║▐
║ ║▐
║ Lastly, in order to avoid jittery movement from the user, ║▐
║ since it is difficult to keep one's head completely still, ║▐
║ we implemented a could filters into our algorithm. One ║▐
║ filter used was a window filter to check the current ║▐
║ coordinate against the previous ones and see if it differed ║▐
║ greatly, which would most likely mean the user was ║▐
║ intentionally trying to move the cursor. Little ║▐
║ unintentional movement could be filtered out, since it ║▐
║ would still have coordinates very similar to previous ║▐
║ movements. ║▐
║ ║▐
║ There were many other ideas for this project that were not ║▐
║ implemented, including the use of the third dimensional ║▐
║ coordinate to zoom in and out, a better detection setup for ║▐
║ the user to move around in, and a way to add clicking. We ║▐
║ were able to achieve clicking with the use of the a WiiFit ║▐
║ board. The user could either sit on the board or operate it ║▐
║ with their feet, and leaning to the right would right ║▐
║ click, and left would left click. Unfortunately, as the ║▐
║ user moved right or left, this would often affect the ║▐
║ position of the user's head and that made clicking ║▐
║ difficult, so a better way to click was needed. ║▐
╟┐ ┌╢▐
╟┐ ┌╢▐
║ Part hardware hacker, part creative technologist, and part ║▐
║ human-centered designer thriving at the cross-section of ║▐
║ engineering and design. ║▐
║ ║▐
║ Led to engineering through my love for music, art, and ║▐
║ math, I hold degrees in electrical, computer, and music ║▐
║ engineering. Following a stint in academia, I joined IDEO ║▐
║ as a senior electrical engineer where I contributed ║▐
║ creative problem solving and prototyping fluency to a ║▐
║ breadth of projects ranging from vehicles of the future, to ║▐
║ life-changing medical equipment, to innovative children's ║▐
║ toys. Most recently, I was at Delve and now Sundberg-Ferar, ║▐
║ diving deeper into engineering for production, and working ║▐
║ on bringing ideas and prototypes to reality. ║▐
║ ║▐
║ Outside of work, I enjoy creating rotoscopes and ║▐
║ illustrations, reverse engineering electronic toys, and ║▐
║ taking a break from screens by running, biking, and camping ║▐
║ in the great outdoors. ║▐
╟┐ ┌╢▐
║ -=≡≡≡≡≡≡≡≡ MS in Electrical and Computer Engineering ≡≡≡≡≡≡≡≡=- ║▐
║ School .... University of Delaware, Newark, DE ║▐
║ Grad year . 2012 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡ BEE in Electrical Engineering ≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ School .... University of Delaware, Newark, DE ║▐
║ Grad year . 2011 ║▐
╟┐ ┌╢▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡ Senior II Electrical Engineer ≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... Bresslergroup / Delve, Philadelphia, PA ║▐
║ Key roles . Developed embedded hardware for diagnostic devices ║▐
║ . Developed embedded firmware with precision-timing ║▐
║ . Designed for manufacturing ║▐
║ Years ..... Nov '20 - Apr '22 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡ Senior Electrical Engineer ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... IDEO, Chicago, IL ║▐
║ Key roles . Specialized in rapid hardware prototyping ║▐
║ . Developed embedded and front-end software ║▐
║ . Designed human-centered solutions for clients ║▐
║ Years ..... Sep '14 - Oct '20 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡ User Experience Intern ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... Shure, Niles, IL ║▐
║ Years ..... Summer '13 & Summer '14 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡ Graduate Research Assistant ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... Univ. of Miami, Music Eng Dept, Miami, FL ║▐
║ Years ..... Aug '12 - May '14 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡ Co-Founder and Shop Manager ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... Newark Bike Project, Newark, DE ║▐
║ Years ..... Oct '11 - Aug '12 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡≡≡ Undergrad/Grad Research Assistant ≡≡≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Company ... Univ. of Delaware, Elec/Comp Eng Dept, Newark, DE ║▐
║ Years ..... Feb '09 & Aug '12 ║▐
╟┐ ┌╢▐
║ -=≡≡≡≡≡≡≡≡ Languages ≡≡≡≡≡≡≡≡=- ║▐
║ C, C++, Obj-C HTML, CSS Spanish (int) ║▐
║ Basic Assembly Javascript Japanese (beg) ║▐
║ Matlab Python ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡ Software ≡≡≡≡≡≡≡≡≡=- ║▐
║ Adobe CC Autodesk Fusion Microsoft 365 ║▐
║ ║▐
║ -=≡≡≡≡≡≡≡≡≡≡ Other ≡≡≡≡≡≡≡≡≡≡=- ║▐
║ Screen Printing Bike Mechanic Furby Technician ║▐
║ Sketching Illustrating ║▐
╟┐ ┌╢▐
╟┐ ┌╢▐
╟┐ ┌╢▐
