World through touch

This is the transcript of the talk by Keith Kirkland presented at Git Commit Show 2019.

About the speaker

Keith Kirkland is an inclusive technologist and a TED Speaker. Keith built a device that helped a blind man run the marathon. He is founder of haptic design company - WearWorks. He has been an adjunct professor and a TED speaker. Keith is an Industrial Designer focused on developing products at the intersection of Touch, Design, Technology, and Navigation.

Transcript

I'm super excited to share what we've been working on with you for the past few years and my goal here and this master's class is to give you a sense of what the world looks like potentially to haptics through our example. Hopefully, it inspires some of your explorations. Now our company is called where we work and we build products and experiences that communicate through touch. The idea is that this skin is there and it's available and what if we could put some of the information directly into it. The way we explored this space is we built a wearable product called weight ban which is a haptic navigation device and I'll get into a bit about how it works and what we've used it for but mainly I want to start with this idea of haptics. I know in a lot of our presentations we come across people and also a lot of times. I've never heard of the word and so what we're trying to do is have fixed means of overlaying distance of touch that can be anything from burning the roof of your mouth you bite into a hot slice of pizza or you know the hug of a loved one that you haven't seen for a long time. Your skin is constantly taking in external information is one of five major senses and the largest sensory organ in the body and so we were looking at the possibilities of touch and wondering how we can explore that and this is what we decided to do with hope.

‌‌

Why in a sensitivity attend to the eye's ability to recognize millions of colours or the ability to recognize complex pitch and tone but as a communications channel, it's been largely relegated to more scope like cell phone notifications. If you look at the possibilities like if someone were to grab you could tell whether it was a direction or love almost in an instant. So we saw the possibility of what the human body was capable of understanding as far as to touch through our personal and interpersonal relationships. But we also saw what was being done with this medium as a form of design and as a form of communication. We thought that there was a lot of space in that gap to play and we got super excited about it. So I'm gonna talk a little bit.  I think it might enable people, it's the kind of stuff that happens to this idea of what tactics could be and by tapping into your own experience of filling through touch. The first thing that we realize is to understand touch, you have to close your eyes and if you look at it as a hard drive. Write a video file to accept a huge amount of space, an audio file a lot less space and then you look at a touch. It can be so dampening or so minuscule compared to the over sensory of information that you're getting through your visuals. It's easy to not pay attention but even right now I feel the fabric on your body like it's healing you sit in the chair where the pressure is on different points. How much tension is in your muscles? This is all touch-based information your body is communicating and often you can't pay attention to it with your eyes open because you're still overstimulated. So we wave a nap as part of our device and now once you'd imagine this imaginary route is, where you are in a new coffee shop that you're trying to find, you're trying to meet a friend there. Now I want you to imagine how you would walk that route. Now I want you to do me a favour, I want you to close your eyes and I want you to imagine that same brow without access to any sight at all. Now you'd be walking along the street probably like a blind person, maybe you'd have a cane or a dog to support you. You came at your dog, what's huggy, what's directly in front of you like the fire hydrant or this person or maybe that potted plant, that's on the side of the road but it's not gonna help you get to the coffee shop. Meanwhile, the audio and your GPS navigation is an amazing tool but your ears are your eyes. So sometimes it can be hard to hear what's going on in the surrounding environment and you need your ears to kind of keep yourself safe. That's how you do situational awareness, you can open your eyes now and so that leads directly into our next point of developing the deep empathy for the user and their particular situation when we first set out.

‌‌

We thought it'd be a universal thing that everyone would just understand but then we realized that it's not as language-based as we thought. I have to guess that it is haptic across every situation though in context the thing where the same way the word "computer "is a computer. What it is. I guess the best way to describe it is it's very situational, like a push from your wife underneath the kitchen table after you've said something might be very different than a push from your life at the kitchen table. After you give her a gift and so it's very contextual inside the attraction. So we try to start with this concept of what our user is doing. Now in our particular case, we weren't interested in the idea of navigation and we thought that visual navigation isn't necessarily the safest way to navigate. Sometimes we've always seen people walking down the streets, not paying attention to where they're going. Then we looked at some cases where it was like not only was it not safe but it was ridiculously not sensible. I've seen people riding down the streets in New York City all the time one hand on their phone, looking at a map while they're riding down the street and then, of course, there's an entire population of 285 million people worldwide for which this is impossible. So when we were digging into our users about what they were looking for, we were trying to understand what information might they weren't for. We started with the idea of tourists, we eventually started talking about running as a cyclist and we landed in a use case of the blind and visually impaired. It was pretty obvious that it was going to be a great use case for us but it was challenging and one of the things that we realized is that by doing the most challenging thing, we showed more of what was possible in space. So challenge yourself by the guest to build out something that you think is worthwhile. The idea of haptics is that your user doesn't care about haptics; they care about their experience and they care about the information they're getting. So as a haptic designer it can be easy to just geek out about it.  But that has to go into the context of what the user needs and what they want. The other thing is that you know in the case of haptics instead of seeing the feeling is really believing we built these devices and unless you were physically holding a device, you had no idea what the device was doing. So we tried to find different ways to visualize some of that but the important thing is that we stretch directly and are haptic. In the same way, if you're doing the sculpture, maybe you'll start with some line drawings, you get some inspiration but also in the lead you're gonna start sketching in clay as soon as you get a chance to. That's how you get a feel for what it is that you will be working with and what happens. It's just the same thing we sketch.

‌‌

I fell and we jumped indirectly to the experience of what was going on. This was an example of one of our prototypes before we recall its wave and our company was called beyond sight and all of our prototypes were aptly named BS. So this is BS number two and as you can see here we're playing with an accelerometer. We're going back and forth, we added the lights so that you can understand the vibrating motors. We're doing this because everyone would run in to grab the prototype and touch everybody and we redefined a whole new way of moving with touching the outfits. It's amazing but this was just a way better tool to visualize what's going on and so this is what one of our experiments is about.  The other thing that you have to work on is when exploring a haptic space, you will need a clear measure of what progress is like metrics defining the success of what the project is and for us when we set out we had a pretty audacious measure of what progress might look like.

‌‌‌‌Our navigation device for the blind or visually impaired figured out a way to guide people to their twin end destination using only operation without the need for visual or audio clues at all. You started with a use case of navigation initially, we just want to get people out of their phones and back into the real world but the more we dug into it the more we realized that the power of the blind use case was so profound because no one had come up with a solution. When I first started using the device, I wasn't sure what to expect and it ended up turning out to be even better than what I was imagining. The ability to convey information to me with just more vibrations and vibration patterns was amazing and not only that but when it wasn't vibrating, it's still gaming information that I am on the right path and navigating in the right direction. So it was completely intuitive while I was out using a device where there were three motorcycle abilities. If I was using a voiceover feature on the iPhone to navigate, I wouldn't be able to hear that over the outside sound that was an ability to bring in another one of my senses and be able to navigate with that science of touch rock with Marcus and then ultimately Simon who's our blind marathon runner. He reached out to us and said that if we could have our device ready for him he would run the New York City Marathon with it this year. We knew it could work, it had a lot of kinks that needed to be worked out and so over six months, we worked with Simon to send him the devices to test. We sent them new devices to test within a few weeks of the marathon, we finally had a product that we thought could do. On November 5th 2017, Simon got to make history as the first black person to run a New York City Marathon unassisted.

‌‌

So we had a pretty audacious goal and to be honest, we had no idea that could be done. The product and the idea were very simple but paired with Simon, connecting with us and reaching out to us and deciding for himself that like "hey you guys have this ready in the next six months. I'll run a marathon with it" was the push that we needed to improve the performance. So picking something a goal that's worthy of achieving with haptics a lot of times has to see if you looked at a  background BAC thing that occurs. So we challenge ourselves to make it happen, it's the central theme of whatever piece it is that we're working on and the thing is that in the case of Simon, the thing that we're excited about is that there are 285 million people like Simon in the world. So we saw the opportunity of making a huge impact by making this small proof case. I think that my final point is that you have to create a safe space to fail. Most people have not heard of this word before and most people have no idea of what the possibilities are outside of sending you a vibration notification and so we were testing out lots of different things. My team and I fail miserably regularly every single day.  I want to encourage a deep sense of exploration and also put some really big ideas out there. Don't be afraid to not hit them because a lot of times along the way and building these haptic experiences, you are doing stuff that no one's ever done before and no one's ever thought of before. Of course, it doesn't work the first time,  sometimes even the 20th time but the exploration is fun and it's exciting and every failure brings about new possibilities of what could be. Those failures add up to a wonderful potential that can take your product or your idea or your experience to the next level. Speaking of failure, Simon didn't complete the entire marathon. He only ran the first 15 miles of it. We had a failure along the way that was pretty catastrophic as far as the device is concerned Simon did finish the marathon. He was safe, we had backup miners just to make sure that if the technology did fail but I'll let Simon talk a little bit more about his own story.

‌‌‌‌I guess to wrap things up, this is a really interesting space to play in. We're drawing on inspiration and ideas from lots of different fields. We're looking at sound generation, a sound creation that is an inspiration for haptic explorations. We're looking at gestures and how people respond to one another in physical settings and trying to understand what things we can leverage from our personal experiences in the world to bring it to the haptic space.

‌‌We thought what if we gave the device a jerk so that like most people have the experience of being in a vehicle and having the vehicle stopped suddenly. So we felt that everyone would get that experience and it's an understanding that is going to mean stop. We found that when we tried it out with people that's exactly what happened as people would walk, then they would get the jerk and then they would all of a sudden stop not recognizing even with the jerk man. So it's like how can we tap into the psychology of cultural experiences to bring about gestures out of that and distil them into potential haptic possibilities.

‌‌Now it's time for questions.

‌‌1. How haptic design can come in this place, in this field. Would love to have your thoughts on that. How we can enable efficient work within these people who are not at the same location.

‌‌‌‌There's a lot of work being done in the telepresence space right now and we've already augmented audio, we've augmented video, we have people working on explorations around how to make humans better human. I think right now the thoughts around potentially connecting people in a space of using haptics. The idea that I like is the idea of interpersonal communication. Most of the time we're not in the same room together, we're all running around doing different parts of the business. We're often in different parts of the world and so we've gotten to this thing where we make these random calls to show that the lines of communication are always open. We design an experience that is like " you did a great job "and like everyone gives you a high-five and you just feel it piling up. We spent a lot of time working with the blind and visually impaired. We've talked to people who were deafblind the Helen Keller Foundation, we've talked to people who are deaf people who have Parkinson's and they all saw very different opportunities and possibilities and use cases for our technology. It was almost like making the device because people would write us emails and they would design experiences.

‌‌2. What kind of tools or technologies would you recommend that they get?

‌‌I think the main thing that you will need to arm yourself with is some basic Arduino programming and that whole ecosystem is so well set up for hardware like you can buy a haptic motor. You can buy haptic drivers, you can buy orientation ammo like orientation sensors, ultrasonic sensors, light detection sensors light. You can give it the ability to create hardware on your own from my off-the-shelf components and breakout boards and so I think that's the way I would go when we got into this space.  I don't have a background in electrical engineering, so I taught myself to code. In our prototypes, we taught ourselves hardware basically and so the Arduino platform was essential. We have a whole electrical engineering team and they do great work.  you know I can barely understand the work that

‌‌3. Do you have any message for the audience?

‌‌This was my super lofty goal, we've been in a haptic space and right now we're in a space of haptic navigation but of course, we see that is only the beginning of what's possible for haptics. We see an entire ecosystem, an entire world where hectic communications underlying on top of visuals and top of audio create a more cohesive and Experience for four people with super excited about the VR space and so there's a lot of opportunity to like pair haptics and VR and a pair hectic scene gaming that's kind of the obvious stuff but also - there's a lot of really non-obvious stuff that's cool out there. So I wanted to encourage you to explore this space.

‌‌

Thank you so much