Going From Google Glass Mirror API Quick Start to Product
Going from Google Glass Mirror API Quick Start to Product
At this point, you should have either the Java or the Python Mirror API Quick Start up and running. If you do not, please check out this post, and this post (apologies that I don’t have one for Python, just Java). This post is going to discuss how to use the quick start projects to begin making progress on your product.
I’m going to talk about this in the context of a service that I’m working on building up, Hearty.io. I think that it’s important to understand the product, and the goals of the product when approaching development. I also think that it is especially important with wearables to understand your product, and how that’s going to work with wearables.
Hearty.io
Let’s get started by taking a look at what we have for a service, understanding it, and thinking about how we can use the Mirror API to extend it and offer something useful to our users. One note, Hearty.io is currently very rough and is missing quite a bit. What we will do here, is to define a current, fictional set of functionality, things that we can assume are already working. Again, to be clear, the following is a fictionalized description fo the service, that we will use for our purposes.
Hearty.io is a fitness service with several components. There is an Android component, an Android Wear component, and Glassware. There is also a backend service that keeps all of the data synchronized.
Android
The Android app does automatic activity recognition, and tracking. It uses Google Play Services to determine what the user’s current activity is, and then automatically tracks times when the user is active. The Android app posts a notification during times that it is tracking with the current activity, and the length of time tracked.
The Android app also syncs with the server, and displays data from all three sources.
Android Wear
The Wear app does step counting, and syncs directly to Android using the Android Wear Data APIs. The Android notifications for Activity tracking show up on the watch, and there is a Wearable action that would allow the user to ‘ignore’ the current activity.
Glass
The Hearty.io Glassware uses the GDK and talks to a Bluetooth Heart Rate Monitor. The user launches the Glassware by saying 'Ok Glass, show my heart rate’. When Hearty launches, if there is no device paired, it redirects into a pairing UI. Once paired, Hearty.io inserts a LiveCard that displays the users’s current heart rate. The Glassware syncs the user’s heart rate directly with the server.
Server
The server just keeps track of three things for each user, each day’s step count, number of active minutes, and average heart rate.
New Mirror functionality
The most obvious thing that I can think to do with Mirror here is to send the daily summary to the user on a daily basis. We also want to display that data in a dashboard for the user.
There are a few things to consider here, beyond simply what we want to build. The primary consideration should be if it’s something that makes sense, and would be a good experience on Glass. Mirror can be used to do things that don’t result in a good experience. For example, news content is tricky to consume on Glass. Reading lots of text on a screen, or spending up to a minute listening is not ideal. WinkFeed does a great job of providing a good experience for news, mainly because of its Pocket integration. Sometimes, a clever feature can help your Glassware to go from a mediocre experience to a great one. Take a look at the Glass Design Principles and think about whether what you’re doing is in line with those principles.
What we are going to build
Let’s build the following, a web service that uses a fake database, displays the dummy data to the user on a web app, and then allows the user to send that data to Glass.
Wiring this up to our real service should be something that we know how to do, so we are not going to cover that here. We are also not going to do the work of automating this to run on a daily basis, that should be fairly straightforward.
I’ll cover Java first, if you’re more interested in Python, feel free to skip down to that section.
Java
I’m going to assume that you’ve already been introduced to the quick start code, and so I’m going to skip the overview here. What we need to know is where we need to go in and what to modify.
JSP
The index.jsp file is what is used to generate the main web view. We are going to start by adding a section at the top with some dummy, hardcoded data. Place this above where the timeline cards are shown.
<div class="span4">
<br>
<table class="table">
<tbody>
<tr>
<th>Heart Rate</th>
<td>
77
</td>
<th>Steps</th>
<td>
17,311
</td>
<th>Active Minutes</th>
<td>
89
</td>
<td>
</td>
</tr>
</tbody>
</table>
</div>
That will give us a table, without any actual data. Let’s add a button at the top that uses the same sort of paradigm that all of the other buttons do on this page.
<form action="<%= WebUtil.buildUrl(request, "/") %>" method="post">
<input type="hidden" name="operation" value="insertHeartyData">
<button class="btn btn-block" type="submit">
Insert a bundle with Hearty Data</button>
</form>
Pretty easy, right? Notice that the value of the input is insertHeartyData, that’s going to be important when we go to the servlet.
Now, let’s make a quick side-trip, and add our dummy database.
FakeDatabase:
public class FakeDatabase {
private static final ConcurrentHashMap<String, HeartyData> heartyData = new ConcurrentHashMap<String, HeartyData>();
private static Random rand = new Random(System.currentTimeMillis());
public static void generateFakeUserData(String userId) {
HeartyData data = new HeartyData();
data.activeMinutes = rand.nextInt(110);
data.stepCount = rand.nextInt(25000);
data.heartRate = rand.nextInt(50) + 50;
heartyData.put(userId, data);
}
public static HeartyData getUserData(String userId) {
return heartyData.get(userId);
}
private FakeDatabase(){ // not allowed to instantiate }
}
Ok, now that we have our database, we have to get data in there, so what we’ll do for that is to add the following to the NewUserBootstrap:
public static void bootstrapNewUser(HttpServletRequest req, String userId) throws IOException {
// snip ...
FakeDatabase.generateFakeUserData(userId);
// snip ...
}
Back to the jsp, let’s use our “real” data there:
<div class="span4">
<form action="<%= WebUtil.buildUrl(request, "/") %>" method="post">
<input type="hidden" name="operation" value="insertHeartyData">
<button class="btn btn-block" type="submit">
Insert a bundle with Hearty Data</button>
</form>
<br>
<table class="table">
<tbody>
<tr>
<th>Heart Rate</th>
<td>
<%= FakeDatabase.getUserData(userId).heartRate %>
</td>
<th>Steps</th>
<td>
<%= FakeDatabase.getUserData(userId).stepCount %>
</td>
<th>Active Minutes</th>
<td>
<%= FakeDatabase.getUserData(userId).activeMinutes %>
</td>
<td>
</td>
</tr>
</tbody>
</table>
</div>
That about does it for the viewing side, let’s try to make this do something.
Servlets
The quick start uses a Servlet for handling all of the POST requests from the web page, as well as all of the subsequent Mirror requests. We can copy one of the other insert methods, and modify it a bit for our purposes.
if (req.getParameter("operation").equals("insertHeartyData")) {
LOG.fine("Inserting Hearty Timeline Item");
String bundleId = String.valueOf(System.currentTimeMillis());
TimelineItem timelineItem = new TimelineItem();
Attachment bundleCover = new Attachment();
String imgLoc = WebUtil.buildUrl(req, "/static/images/hearty_640x360.png");
URL url = new URL(imgLoc);
bundleCover.setContentType("image/png");
timelineItem.setText("Hearty.io");
timelineItem.setBundleId(bundleId);
timelineItem.setIsBundleCover(true);
TimelineItem timelineItemHeart = new TimelineItem();
timelineItemHeart.setText("Heart Rate: " + FakeDatabase.getUserData(userId).heartRate);
timelineItemHeart.setBundleId(bundleId);
TimelineItem timelineItemSteps = new TimelineItem();
timelineItemSteps.setText("Steps: " + FakeDatabase.getUserData(userId).stepCount);
timelineItemSteps.setBundleId(bundleId);
TimelineItem timelineItemActivity = new TimelineItem();
timelineItemActivity.setText("Active minutes: " + FakeDatabase.getUserData(userId).activeMinutes);
timelineItemActivity.setBundleId(bundleId);
// Triggers an audible tone when the timeline item is received
timelineItem.setNotification(new NotificationConfig().setLevel("DEFAULT"));
MirrorClient.insertTimelineItem(credential, timelineItem, "image/png", url.openStream());
MirrorClient.insertTimelineItem(credential, timelineItemHeart);
MirrorClient.insertTimelineItem(credential, timelineItemSteps);
MirrorClient.insertTimelineItem(credential, timelineItemActivity);
message = "A timeline item has been inserted.";
}
That’s a big chunk of code, but there are lots of repeated bits in there. Let’s break it apart and look at it piece by piece.
There is a repeated pattern, that I’m going to show here:
TimelineItem timelineItemHeart = new TimelineItem();
timelineItemHeart.setText("Heart Rate: " + FakeDatabase.getUserData(userId).heartRate);
timelineItemHeart.setBundleId(bundleId);
MirrorClient.insertTimelineItem(credential, timelineItemHeart);
The above is the basics for inserting an item into the timeline. Notice that the bundleId is set as well, that allows this Card to be bundled with other Cards with the same bundleId.
You’ll notice that the full block of code that was posted just repeats that pattern a few times to create four cards, a cover and three children.
Wrapping up with Java
From here, what I did was to move the Hearty.io pieces out of MainServlet and into both its own Servlet, and its own jsp file. When you do this, you’ll need to modify web.xml to map the new path to the correct servlet. You can see the final version of the code here.
Python
As with Java, I’m going to assume that you’ve already been introduced to the quick start code, and so I’m going to skip the overview here. What we need to know is where we need to go in and what to modify.
As a note, this is going to be very similar to the Java portion. Partly because it does the same thing, partly because the quick start code structures are fairly similar, and mostly because I’m lazy.
HTML Template
The index.html file is what is used to generate the main web view. We are going to start by adding a section at the top with some dummy, hardcoded data. Place this above where the timeline cards are shown.
<div class="span4">
<br>
<table class="table">
<tbody>
<tr>
<th>Heart Rate</th>
<td>
77
</td>
<th>Steps</th>
<td>
17,311
</td>
<th>Active Minutes</th>
<td>
89
</td>
<td>
</td>
</tr>
</tbody>
</table>
</div>
That will give us a table, without any actual data. Let’s add a button at the top that uses the same sort of paradigm that all of the other buttons do on this page.
<form action="/hearty" method="post">
<input type="hidden" name="operation" value="insertHeartyData">
<input type="hidden" name="heart_rate" value="77">
<input type="hidden" name="step_count" value="17,311">
<input type="hidden" name="active_minutes" value="89">
<button class="btn btn-block" type="submit">
Insert a bundle with Hearty Data</button>
</form>
Pretty easy, right? Notice that the value of the input is insertHeartyData, that’s going to be important when we go to the handler. Also notice that I’ve got several input values, these are for passing values from the web frontend to the backend. There’s probably a way to attach this to a session, but I don’t know Python that well, and again, lazy.
Now, let’s make a quick side-trip, and add some sort of data generator.
Hearty:
class Hearty(object):
def __init__(self):
self.heart_rate = random.randrange(40,135,1)
self.step_count = random.randrange(2000,28000,1)
self.active_minutes = random.randrange(17,132,1)
Ok, now that we have our data, we have to get data in there, so what we’ll do for that is to add the following in the get method:
@util.auth_required
def get(self):
# snip ...
self.hearty = Hearty()
self._render_template(message)
And our render template method:
def _render_template(self, message=None):
"""Render the main page template."""
template_values = {'userId': self.userid,
'hearty': self.hearty }
# snip ...
This means that every time we refresh the page, we will see new values.
Back to the template, let’s use our “real” data there:
<div class="span4">
<form action="/hearty" method="post">
<input type="hidden" name="operation" value="insertHeartyData">
<input type="hidden" name="heart_rate" value="{{ hearty.heart_rate }}">
<input type="hidden" name="step_count" value="{{ hearty.step_count }}">
<input type="hidden" name="active_minutes" value="{{ hearty.active_minutes }}">
<button class="btn btn-block" type="submit">
Insert a bundle with Hearty Data</button>
</form>
<br>
<table class="table">
<tbody>
<tr>
<th>Heart Rate</th>
<td>
{{ hearty.heart_rate }}
</td>
<th>Steps</th>
<td>
{{ hearty.step_count }}
</td>
<th>Active Minutes</th>
<td>
{{ hearty.active_minutes }}
</td>
<td>
</td>
</tr>
</tbody>
</table>
</div>
That about does it for the viewing side, let’s try to make this do something.
Handlers
The quick start uses a handler for handling all of the POST requests from the web page, as well as all of the subsequent Mirror requests. We can copy one of the other insert methods, and modify it a bit for our purposes.
def _insert_hearty_item(self):
"""Insert a Hearty.io timeline item."""
logging.info('Inserting hearty timeline item')
bundle_id = `random.random()`
body = {
'notification': {'level': 'DEFAULT'},
'text': 'Hearty.io python',
'isBundleCover': True,
'bundleId': bundle_id
}
heart_rate = self.request.get('heart_rate')
step_count = self.request.get('step_count')
active_minutes = self.request.get('active_minutes')
body_a = {
'text': 'Heart Rate: ' + heart_rate,
'bundleId': bundle_id
}
body_b = {
'text': 'Steps: ' + step_count,
'bundleId': bundle_id
}
body_c = {
'text': 'Active Minutes: ' + active_minutes,
'bundleId': bundle_id
}
media_link = util.get_full_url(self, "/static/images/hearty_640x360.png")
resp = urlfetch.fetch(media_link, deadline=20)
media = MediaIoBaseUpload(
io.BytesIO(resp.content), mimetype='image/png', resumable=True)
# self.mirror_service is initialized in util.auth_required.
self.mirror_service.timeline().insert(body=body, media_body=media).execute()
self.mirror_service.timeline().insert(body=body_a).execute()
self.mirror_service.timeline().insert(body=body_b).execute()
self.mirror_service.timeline().insert(body=body_c).execute()
return 'A timeline item has been inserted.'
That’s a big chunk of code, but there are lots of repeated bits in there. Let’s break it apart and look at it piece by piece.
There is a repeated pattern, that I’m going to show here:
heart_rate = self.request.get('heart_rate')
body_a = {
'text': 'Heart Rate: ' + heart_rate,
'bundleId': bundle_id
}
self.mirror_service.timeline().insert(body=body_a).execute()
The above is the basics for inserting an item into the timeline. Notice that the bundleId is set as well, that allows this Card to be bundled with other Cards with the same bundleId.
You’ll notice that the full block of code that was posted just repeats that pattern a few times to create four cards, a cover and three children.
The only other thing going on here is adding the image to the bundle cover:
media_link = util.get_full_url(self, "/static/images/hearty_640x360.png")
resp = urlfetch.fetch(media_link, deadline=20)
media = MediaIoBaseUpload(
io.BytesIO(resp.content), mimetype='image/png', resumable=True)
# insert the cover body with the media into the timeline
self.mirror_service.timeline().insert(body=body, media_body=media).execute()
Wrapping up with Python
From here, what I did was to move the Hearty.io pieces out of main_handler and into both its own handler, and its own template html file. When you do this, you’ll need to modify main to map the new routes to the correct handler. You can see the final version of the code here.
Moving forward
This post should have given you some idea of how to take the Mirror API Quick Start and start taking steps towards morphing it into your own project. You will obviously want to have your own service, with your own data, and a real database. The idea here was to start turning knobs and seeing what happens when we change things. Really, the same approach can be taken with most sample code, where you start with what’s given, and poke at it, while reading the documentation, to figure out how to do what you want to do.