This weekend, my co-worker, Darrel, and I (we work for TiKL, Inc. - YC W12) attended a Google Glass hackathon a Markerly’s headquarters in Menlo Park. We were working on a version of TiKL, a walkie-talkie app for Android and iOS (with 27 million users!), that works on Glass. We made some good progress. Hopefully over the next week we can carve some time out in the evenings to turn Glass into a walkie-talkie with TiKL.
It’s pretty barebones right now, but it does work!
Zapier and Google Glass
I’m just getting around to checking out Zapier’s Glass support, and it is more compelling than I expected. While it would be nice if there were simply more apps available for Glass, Zapier sort of fills in the gaps, and allows you to do some interesting things. For one thing, I can now get notified on my two other gmail accounts on Glass. Since I use my work email way more than my personal, this is going to be a very useful feature. I’ve also got it set up to allow me to share photos to Dropbox, which is nice, my phone does this automatically, but until now, if I wanted a photo from Glass on Dropbox, I would have needed to download the photo on my laptop, and upload it to Dropbox manually. I am trying out the Trello integration as well, since my wife and I use that for the grocery list, though I’m not sure how useful that one will be for me.
http://neatocode.tumblr.com/post/53113748050/super-human-vision-google-glass
Google’s new wearable computer and display, Google Glass, can be used to enhance your abilities beyond human! This fits with the recent idea covered by The Verge at Augmented World Expo that, “every game-chaning technology can be recast as a human superpower.“ At the recent AngelHackNYC I…
This week, I started prototyping a stripped down Java implementation of a Mirror service for Talkray, the awesome mobile communications app that I work on at work. The typical flow for installing Mirror Glassware is to visit a website, click an ‘install’ button, pick your Google account and grant access, and Google returns you back to the website, giving the website your token. This works fine for brand new services, or web services, that already have your account, and are adept at doing OAuth. However, for a mobile app, where the account only lives on the phone and servers that don’t serve web pages, this presents a bit of a challenge. I could implement the full OAuth dance server-side, but, if I did that on the web, I’d need a way to make sure that we’re tying this to the correct account. Then, I had a crazy idea, what if I could use Android’s built-in APIs that talk to Google services for talking with Glass? Could this work? Spoiler: the short answer is no, and the long answer is a qualified yes.
While I have not spent a ton of time with this, there seem to be a few major roadblocks in the way of making this a reasonable solution. First off, I am able to authenticate and push items to Glass from Android, and I will get into the details shortly. However, the roadblocks are not insignificant. For one thing, as far as I’m aware, I can only initiate requests from Android, I can’t handle POST requests to my app. This means that there would be no way to respond to an item on Glass back to my app.
Another issue, from what I’ve read, is that the tokens that you get from Google are only good for an hour. Google expects you to re-request tokens as you need them, and provides a way to do those re-requests without bugging the user. Unfortunately, I didn’t need to provide my OAuth key/secret to the Android app, which means that the platform is handling that. Initially, I was hoping that I could get the token on the phone and push it up to the server. That way, I could solve the difficult OAuth problem, and do just the Mirror API stuff on the server with the token that I already have. But, if the token expires every hour, I would need to be able to re-request for that token somehow, and I can’t do that from the server because the platform is handling that, instead of my app, which means that I would not be able to re-request a token from the server. (I think.) There are probably ways to work around this, but I would assume that they all involve lots of talking between your phone and the server - even if you could use that token on the server.
That said, if all you want to do is push data from the phone to Glass, and don’t need to get data back from Glass, this might work OK for you. Keep reading to see how.
I’ll break this down into two main sections, the Authentication piece, and the Mirror piece. Pretty self explanatory.
First off, I grabbed a lot of this code from the following blog post: Google OAuth on Android using AccountManager, which was incredibly helpful in getting started with this. So, go read that first, I’ll be here when you get back.
Alright, now that you’ve got an idea of how to use Android’s AccountManager, there are a few tweaks and notes that need to be made to that post. Most importantly, is an error in that post that I mentioned in the comments. When grabbing the user’s account, you need to make sure to filter on “com.google”, before I added this, I kept getting Dropbox: accountManager.getAccountsByType("com.google")
The SCOPE also needs to be changed to request permissions for the Mirror API:
Those are really the only changes to Thomas’s code that need to be made to get the authentication piece working.
Now that you’re authenticating with the AccountManager, we need to plug in some Mirror calls. I think that I’ve found a bug here as well, but I’ll need to ping the team to make sure. Basically, when you’re building the Mirror object, you don’t pass it any credentials, instead, the token gets added to the timeline insert request.
See how that third parameter in the
Builderis null?
Now, you can take a look at what we can do once we have our token:
See the call to
.setOauthToken(authPreferences.getToken())?
If you have Glass and you’re running this, at this point you should see “Hello from android” inserted into your timeline. It did work for me when I was testing it last night.
While I’m very interested in seeing if I can push this to make a work-able solution, I’m concerned that this will be a dead-end, at least for now, given the issues that I outlined above. However, I am certainly open to hearing ideas about how this might be expanded into something that is actually useful, or a way that I could actually use the token server-side. (Obviously, nothing that Google would frown upon.)
Forgot a couple of things, first, there are a few jars that you’ll need to pull in, check out the libs dir of the project. I’ll probably need to get this formatted to have maven suck those in auto-magically, but I didn’t have much time to get the code working and write this post, so hopefully soon. Second, I had an issue with getting AndroidHttp.newCompatibleTransport working, so I stuck with NetHttpTransport.
Looks like someone else had done this a couple months back and actually spent enough time to give it some polish. For those wondering, I don’t really bother with polishing experiments much. They might grow eventually, or they might just stay as they are. The idea is to just test something out quickly, and maybe write about it. I don’t have time for much more than that.
I just ran into the error: “Daily limit for unauthenticated use exceeded. Continued use requires signup.” Luckily, I figured out what the issue was, at least for me. I switched to Intellij, and one thing that I didn’t realize was that gradle was not grabbing my regular debug.keystore from ~/.android/debug.keystore. So, I tried specifying a particular keystore with gradle:
Then just copy the keystore into your module directory:
Now, you should be good to rebuild and have things work!
As I tend to do, I went to a Meetup last Thursday at Google, for a presentation on Chrome Packaged Apps. It was a very well delivered presentation, and the pizza was a bonus.
Yay pizza!
And, as always, I rode my bike to the meetup, and then from Google back home, which is almost twice my normal commute. I was getting fairly sweaty on the ride, and when I was getting close to home, Glass started freaking out. It was as if I were constantly tapping and swiping the touchpad. This was quite a distraction while riding, as the screen was doing all sorts of wacky things. When I got to a stop light, I powered Glass down, and finished up my ride. I waited a little bit to turn Glass back on, probably a half hour or so, and the problem persisted. I powered it down and waited an hour, and it was still acting up, but now it wasn’t recognizing touch input. I decided that I would give a factory reset a try, just in case this was an oddly timed software bug, and I had the thing in debug mode - for development. After the device rebooted it was actually quite difficult to get it signed in, because the touchpad was still not working properly.
The other wrinkle here was that I was flying out to Vancouver, for the weekend, early the next morning. I was disappointed, but decided to leave Glass behind for the trip, and figured that I would call the morning after returning home.
Leaving for the airport at 5:30am, without Glass was an odd experience. I had only had Glass for about two or three weeks at this point, and already, I’ve gotten quite used to it. I noticed myself several times wanting to flick my head up to see the time as we rushed to SFO. I noticed that I had the urge to check the flight status that had been showing in my timeline, to see if the flight was on time, and to check the weather - not that we could have done much about what we had packed at that point. The one thing that I did not miss that morning was trying to get Glass through airport security. I had no idea how they might react - not that I would keep it on, but who knows.
Throughout the trip, there were moments where I’d catch myself reaching for one of Glass’s features, whether it be quickly checking Google to figure out the difference between Steelhead Trout and Salmon, or to snap some quick photos at the concert. My phone works for both of these, however, my phone has also become quite slow in the last few months, and Glass is just so much faster for those things than a phone could be.
QOTSA at Squamish
Glass currently seems ok, but I have no idea if there has been permanent damage, or if was just temporary, or if/when it may happen again. There was a suggestion of using one of those sweat headbands to soak up the moisture and protect Glass a bit, and I will certainly use that from now on. It would be unfair to draw any conclusions about Glass’s eventual durability as a wearable from the experiences of Explorers, so I’m not going to even hint at that. I have a feeling that the most important part of this program for Google is to learn how this thing will fail out in the wild, so that they can make the consumer version that much better. At least, that would be my top priority if I were the one running the show.
As I mentioned in my last post, I recently picked up an Explorer Edition of Google Glass. It’s been a couple weeks now, so I figure that it’s been enough time for me to gather some thoughts about it and write them down. There is certainly an adjustment period, where when you first get it, interacting with it while walking might cause you to walk into a post, it takes a few days to get past this. Once you get used to it, you tend not to look at Glass while you’re interacting with it, and doing something else. It’s very liberating.
The experience so far has been pretty amazing. It feels a bit like I’m living in the future. Glass isn’t perfect though, so I’ll try to talk both about what I love about Glass, and what doesn’t seem quite right.
What I’ve been doing mostly has been taking pictures, videos, searching, sending messages and getting navigation directions. Snapping photos and taking a quick video has worked perfectly, and I really don’t think that there’s anything in that flow that I would touch.
Search is pretty good, I’ve been surprised more often than not about Google figuring out some of the specific things that I’m searching. It doesn’t always work properly, but I’d say 90% of the time, it gives me exactly what I ask it for. What’s more, is that search is quick enough, and the flow is good enough, that I can actually search for things mid-conversation. For example, I was describing an article that I had read about a guy who went a year without food, and I remembered that his starting weight was around 200kg, but didn’t know what that was in pounds, so I quickly searched convert 200 kilograms to pounds, and Google quickly told me 440lbs. I was also trying to make plans for the following day, I wanted to get up for an early coding session, and wanted to know if Red Rock Coffee or Hacker Dojo opened first, so, I googled the hours for each of them, which Google told me with one of their cards. (They both opened at 8am, so I picked Red Rock, because they have better coffee and snacks.)
The navigation takes a bit of adjustment, especially while driving. The first time I used it, I felt very disoriented. Think about this, it’s similar to using your smartphone on your dash for navigation, but then it follows your head around as you look left and right. The first time you experience this in the car is a bit confusing. After the first time, along with getting used to wearing Glass in general, will make a lot more sense. Once you grok it, I think it makes more sense than using a smartphone for nav, especially in that it turns its screen off when you’re going straight for a bit. Functionally, it has worked about the same as a simplified version of the navigation mode on Google Maps. I don’t think that there’s anything here that I would really change in the flow.
Messaging is the place where I’ve had the most trouble with Glass. While it’s sometimes convenient to use Glass for sending a quick hands-free message, but most of the time it runs into one of several issues. First, the quick contacts. You can add up to ten friends to your quick contacts, and they come from your GMail contacts, which means that the names come in exactly as they appear in your GMail contacts. You can’t set a nickname, and in order to change the name, you need to change how it appears in your GMail contacts. This is really annoying when you’ve got some people in your contacts last name first. It’s annoying because in order for Glass to recognize the name, you need to say it exactly as it’s entered. This means that you need to say the last name first, and, if it’s a difficult to pronounce name, you might need to try to guess how Glass thinks that name is pronounced. Needless to say, I’ve had lots of trouble with the send a message to command. Quick fix here seems like there could be a ‘nickname’ field in the MyGlass contacts that changes the name that shows up in the menu on Glass.
Once you get to the point of actually saying your message, there are a couple of issues that might come up. First, it might simply get you wrong, and translate something dumb. The only way around this is to re-record until you’ve got something close enough. I’ve gotten to the point of letting those little typos go, as long as the message seems at least understandable.
The second issue is that if you don’t know exactly everything you need to say when you start recording, and pause to think, Glass will stop listening, and will try to send your message. I wish that it would allow me to take some action to append the message, I realize that there’s not much that can be done to edit typos, but appending to a message seems like it should be doable. This is probably a bigger problem when interacting over email, since while you may be giving a quick response, there is still a bit more of an expectation with email that you’re going to take a few beats to think about what you’re going to say, and put it into a single message, as opposed to firing off a half-dozen emails in a row.
Let’s talk hardware for a bit. Overall, Glass is light-weight enough that I really don’t have a problem wearing it for hours, it’s no different from wearing sunglasses for me. (Since I don’t have the experience of wearing glasses on a regular basis.) Overall, I really like the design, but I’ll try to touch on everything.
The display is perfect for what it is and what it needs to do. It really doesn’t need more pixels shoved in there, and I don’t think that most people would be able to tell if they were able to push more pixels in there.
Not much to talk about here, it’s a touchpad, responds exactly how you’d expect it to. The touchpad has freaked out on me once while I was on a long bike ride. Hopefully it doesn’t happen again.
The battery needs to last longer. I think that the Glass team knows this, but it’s probably really tough. They could possibly add another battery on the other side, which would have the added benefit of balancing out Glass, but battery life is always one of those tough issues.
This one is a bit strange. It’s got Bluetooth and WiFi, and uses a tethered phone (via Bluetooth) for GPS and mobile network connection. Once you set your phone’s Bluetooth tethering options, it usually works fine, but sometimes you’ll see that you’re connected to Bluetooth without data. Sometimes this means that your phone doesn’t have a signal, sometimes your phone’s settings get screwed up. Not really Glass’s fault, but it’s confusing none-the-less.
As far as WiFi, there is the problem of networks that require you to visit a website and log in before you’re allowed to use their connection. I know that the Glass team is aware of the problem, though I’ve got no idea what they plan on doing about it.
The bone conduction speaker
This is one hardware component that I do think needs to be changed before it goes out for release. It’s very difficult to hear when there’s any amount of ambient noise. It gets a bit better once you get used to it, but I’m not sure that it’s enough. One trick that I picked up is that you can plug one of your ears, and you’ll be able to hear the bone conduction speaker better.
Shades attachment with nose-bridge clip
I like that there are shades attachments, but they work quite a bit differently than I would have expected. I would have expected that the clip was magnetic, instead, you have to line up the clip and rotate it in. It’s really annoying because it takes two hands, and you need to remove Glass to clip the shades on. I’m really thinking of going to the Hacker Dojo and 3D-printing myself a new clip that I could modify to make it the magnetic clip that I want.
One other part of the experience so far has been getting to demo Glass for people. It’s a lot of fun to see the expression on people’s face the first time that they try Glass on, and the screen lights up. Then, when they say 'OK Glass’, and either google something or take a picture, and it clicks for them. It’s a lot of fun. Wearing Glass around, especially in a bright color, like tangerine, instantly makes you the most interesting person on the street. You’ll get a lot of stares, and people whispering 'that was Google Glass’ as they walk past.
It’s still early days, and obviously, there’s no consumer product yet, so it’s way too early to pass judgement. I’m looking forward to seeing the improvements that the Glass team makes over the next few months.
I recently accepted an invite to purchase an Explorer Edition of Google Glass. Well, actually, the company I work for accepted (man, I love working for TiKL, Inc.), and I’ve been the one working on everything Glass at our office. (In case you’re wondering, a write-up on my thoughts about Glass so far is in the works.) So far, my efforts have been focused on building apks to run on Glass, which is incredibly easy if you’re and Android hacker like myself. What isn’t so easy for someone mainly focused on Android is trying to figure out the Mirror API.
The Glass team has put out a number of quick-start demos for working with the Mirror API, with a focus on getting over the OAuth2 hump. They demonstrate this in a number of languages, but they haven’t put out anything for node.js, which is what I use whenever I need a quick demo server for something. What Google has provided, is a github library that provides some simple mechanisms to interact with all of their services. It’s pretty awesome, and it allows you to pretty easily inspect the JSON data that is returned as the client object, and look at what’s available.
I figured out how to do the Mirror stuff by looking at the Google+ example in the googleapis github, implementing that first, so that I could get the authentication working. Then, making a couple of educated guesses about what I might need to change to get the Mirror API endpoints, along with looking at the Mirror documentation for Java and Python. Once I had G+ authentication up, moving it over to getting authenticated with the Mirror API was really easy, and getting valid requests written was not a big leap from there.
I still do not have much done, my main goal was to simply get node.js talking to Glass, which I did. Speaking of, here’s the github repo for my demo. Again, all this does right now is allows a user to authenticate, pushes the message ‘hello world’ to Glass, and requests the user’s timeline for the registered app.
Follow the instructions in the Mirror API documentation for registering a new app. It’s basically the same flow for starting any new project using a Google API. Just follow the instructions, it’s easy. Once you’ve registered, pull down my repository and run npm install. This will install the googleapis npm package, as well as express.js if you need it.
Next, copy sample_config.json to config.json, and replace the values with those from the Google Developer’s console.
Run node app.js. Finally, visit the app in your browser, and authenticate with Google. You should get a push to your Glass with the text 'Hello world’.
Now that you’ve seen this working, let’s look at some code. This is simply chunks from the app.js file, so if you want to just look at the whole thing at once, here it is.
First up, we’ve got some things to require, the googleapis, and the related oauth stuff.
We should look at the routes involved here first, before we look at anything specific to the API. Without the routes, you don’t really know what’s being called, and where the most important parameters come from, so here are the routes:
As you can see, there are only two routes, the root, and the oauth2 callback routes. We start by looking to see if we already have a token, if they don’t, we send them to Google to go request one. Otherwise, we use the token we have. If they didn’t have a token, they’ll return to us on the route oauth2callback (unless you change that route on your config), and we take the code provided in the request data to parse out their token.
**WARNING**: this is not intended for a multi-user case, just for you! Don't use this unmodified!
We’ll provide a couple of generic callbacks, for simplicity. Whenever you’re unsure of what to do next, or aren’t sure if something’s working, dump whatever data that you’ve got.
Alright, first thing’s first, we need to get ourselves a token. This method gets called when the user returns from giving us permission to connect to their account, and will use the provided oauth2Client to pull the token out of the code that is returned to us. We then store that token in the oauth2Client.
Once we have a token, we can do stuff with it. First thing to do is request a client that can make requests on the Mirror API, and in the callback, we can use the returned client to make those requests.
The following are two simple requests that you can make once you’re authenticated and get your client object:
I probably don’t really need to explain what’s going on there, it’s incredibly simple. But, in case you like reading things, the first one inserts the text 'hello world’ into your timeline on Glass. If you’re using this with the account that you’ve attached Glass to, then that text will show up in your eyeball. It’s neat. The second request simply shows the timeline items that your app has generated.
Currently, this is incredibly simple, and as such is completely lacking in style, structure, and functionality. This is intended to demonstrate just two things, first how to authenticate with Google for the Mirror API with OAuth2, using node.js. Second, it begins to show how you can use node.js to make requests with the Mirror API. Think of it as the quick-start for node.js.
Here it is again, in case you missed it, the github repo that this post was discussing.