Google Maps Developers Live: Photo Spheres and Street Views

Google Maps Developers Live: Photo Spheres and Street Views


Hi, welcome to another Google
Maps Developers Live. I’m broadcasting today from
Sydney on this Australia Day special, hence the
ridiculous hat. Today I’ll be speaking about
Android Photo Sphere– give a little overview– and how to get those Photo
Sphere images and put them into Google Maps Street View. So first of all, a quick
overview of Photo Sphere. It’s one of my favorite features
of Android 4.2. So Photo Sphere lets you take a
complete immersive panoramic image very, very easily,
assisted by the Android camera, and it’s just awesome. So let me show you a
quick look at what the result is like. So this is when you’ve
shared in Google+. So going over to Google+, I’ve
got this awesome panorama that I took here in Sydney
a while ago. So this is the Sydney Harbor
Bridge, looking out over the harbor, all the way over to
Luna Park on the right. And you can see Balmain and
McMahon’s Point in between. This is an awesome, awesome
day in Sydney. So you can pan around
in Google+. You’ve got this great immersive
experience. So in this case, the panorama
that I took wasn’t a complete 360-degree panorama. I think it was about 90
or 100 degrees or so. So the Google+ viewer lets me
pan in between that range, and it’s very cool. And I think if I press this,
this is the flat image that’s come back. So for those that know
anything about image projections, this is an
equirectangular projection. So from there it then gets
warped into this 3D view. And we can also go full screen,
which is quite nice. So over to Street View, why
would you be interested in bringing this image
into Street View? For a few reasons. You can share the image
on your own site. So if you’d like to keep your
panoramas private, or if you don’t want to share it through
Google+, or for some reason your panorama hasn’t been
accepted onto Google Maps, you can use this to embed
it on your own site. You can also customize
the panorama. So if you, for example, wanted
to do some post processing bit of image editing, you could
change the colors, maybe put some image overlays. And finally, Maps API Street
View’s very powerful. You can link panoramas
together. So for example, if you’ve taken
a photo in the lobby of your business and you want to
link that through to a meeting room, or if you’re in a
restaurant, the waiting area through to some of the tables,
you can do all of that using the Maps API. So a quick look at the
Photo Sphere format. You can download an image. You can obtain this
image directly through the Android gallery. If you’ve shared it through
Google+, you can download it from the options menu. And finally, some panoramas get
accepted on Google Maps, so you can obtain imagery
through there. I think the easiest and the most
foolproof way is through the Android camera, but if
you’ve already got one on Google+, that’s quite
good, too. So in this case, the image
is 3,283 by 956 pixels. That becomes important later
on, and I’ll show you why. Actually, first of all, let’s
look at the end result– so a Photo Sphere inside
Google Maps. So this is a Google
Maps API map. I’ll show you the
code later on. But you can see this is a very
similar experience to the one on Google+. You’ve got this nice warping, if
you’ve got a browser that’s capable and fast enough
to support warping. So in this case I’m using
Chrome on a Mac. And you may notice that you can
actually scroll past the bounds, which you couldn’t
do on Google+. This is just a fact that I’m
using the Maps API, but you could write some custom
code to prevent that. So if you look up, everything’s
black around, and you’ll see why in a moment. So going back– Photo Sphere embeds a whole
bunch of metadata into the image similar to how
Exif, if you’re familiar with it, is embedded. So if I run the Strings Unix
utility over the JPEG file, you get back this interesting
information. So you can see here there’s a
URL ns.adobe.com, but the really interesting part is
a little bit after that. It’s some XML, it’s an RDF
embedded metadata, and you can see we’ve got all these
important parts highlighted– the cropped area image height
pixels, image width pixels. You can see, if I format it a
little better, we’ve got the cropped area left top, image
height, image width, full panel width, and full
panel height. So if you’re sharp, you would’ve
noticed that we didn’t see 3283 or 956
anywhere there. This is because a scaling
factor’s applied, so this is important later on. But this is basically what
all those numbers mean. The cropped area is the main
image that you’ve taken, and the full pano is the
full padded image. So if you imagine 360 degrees on
the x-axis, and 180 degrees on the y-axis, the image
width should be twice the image height. So the six important values that
come back from Android Photo Sphere metadata is the top
pixels, the left pixels, the image height and the image
width, and the full pano’s height and width. So this scaling factor needs
to be applied to all of these numbers. The easiest way to make use of
all of these numbers is to hop over to your favorite
image editor. I’ll show you an example
using Photoshop. And so essentially, you need to
take the photo that you’ve taken, pad some black
around it. So you basically move the image
down and to the right, and then pad it out so
that it fills up that full pano pixel values. Just looking here after scaling
those numbers, we take all of them and apply it
by that scale factor. So essentially you take that
cropped area image height and width, and compare them
to the actual values. And then you just
scale them down. In Photoshop you can see here
I’ve gone and added some padding to the left and the top,
using the relative canvas size option. And then I’ve just changed the
total image value, so the result of that is something
like this. So this is a full 360- and
180-degree panorama. So you see that the amount of
photos that I’ve taken is quite small in comparison
to the full panorama. So from here, we just have to
take that image and put it straight into Street View. So we’ve done a couple tutorials
on custom Street View before, but I’ll take you
through it very quickly. It’s really simple
to get started. You start with a Street View
panorama object, but you provide a pano provider object,
and this is where the custom thing comes in. So if I look at the Get Custom
Panorama function, it looks a little like this. You get a panorama ID– so in this case we’ve hard coded
it to Custom, and your returns in metadata are
about that panorama. So the important thing here is
the Get Tile URL function, and here we just return
static value. So you could cut up the image
and tile it if you like, if it’s very large– in fact, it is very large. And you just provide the tile
size and the world size, so this is just the image
dimensions. OK so once we piece all of that
together, we get that beautiful Maps API custom
street view. So let’s just take another look
at that very quickly. So all of that code
ends up something like this, just brilliant. And then from there you can
add links to other custom panoramas, which is great. OK so let’s talk a little
bit more about this embedded format. If we inspect the bytes of this
image file, you’ll see this FFE1 marker, which is
called an APP1 marker. So this is used for all sorts
of metadata, including Exif. So the next two bytes denote
how large this section is. So in this case, 03FE– that’s something like
1,000 bytes. So all we need to do then is
to get the next 1,000 bytes and parse that as XML. So I’ll show you some example
code using Go, but you could do this in any language
you like. So here I define an APP1
marker as E1, and we essentially scan through all of
the bytes until we get FF and then E1. And then from there we go and
call the Make Section function, which is
very simple. It just takes in the size and
then just returns the byte slice of that much of the file,
and then from there the file stops reading. So the code that sort of wraps
all of this is very simple. We continually call this next
section function looking for the APP1 markers until we
end up with something that is Adobe XMP. Once we’ve found something
that’s Adobe XMP, we parse it– I’ll show you in a second the
fields that we parse. But we use the Go’s
XML parser. Extract the XMP, and then
check that it’s a Photo Sphere panorama. If it’s not a Photo Sphere
panorama, it’s some other XMP metadata, so we just
keep looking. So from there, once we have all
that metadata, we can do these same image transformations
that I did in Photoshop using Go’s
image libraries. I’ll show you that
very quickly. So these are all the fields
defined as Go structs. So we have the XML name and
PanoOpts, which is the RDF description tag. So from there we just extract
the fields that we need, nothing extra. Go does not complain if it
gets fields that are unexpected. So from there we can resize
the image using the image library in Go, which
is very, very easy. We create some bounds, and these
are defined by the left top and the image’s
width and height. And then we simply draw that
to an image of the correct dimensions, and then
output it as JPEG. I’ll be making this
utility available. Hopefully I can convert it to an
App Engine app, and output all the required JavaScript
code, so look out for a post on the Google Maps
API Plus Page. You’ll be able to read all of
the code and hopefully iterate on it and do some cool things. So that’s it. That’s a little overview of
Android Photo Sphere and Google Maps Street View.

7 thoughts on “Google Maps Developers Live: Photo Spheres and Street Views

  • January 30, 2013 at 6:50 am
    Permalink

    Sounds excited

    Reply
  • January 30, 2013 at 9:54 am
    Permalink

    hhhh

    Reply
  • January 30, 2013 at 12:02 pm
    Permalink

    Do it for our country Chris. It was well over my head, but I enjoyed your video. I just loved that you're showing off our awesome city.

    Reply
  • March 8, 2013 at 8:16 am
    Permalink

    well done Google developer

    Reply
  • March 26, 2013 at 3:21 am
    Permalink

    You mentioned later in the video when you were parsing the image with Go that you did it originally in Photoshop. What do you mean by that? Also you mentioned using the strings Unix utility. Does this mean that in order to get photospheres into the google map, I would need some server side code to dynamically get the image dimensions from the photosphere? I imagined if you did a full photosphere that they would all be the same dimensions.

    Reply
  • March 26, 2013 at 3:27 am
    Permalink

    Also is there any way to incorporate elevation with different sets of linked photospheres. For instance: a 4 story library and you want to switch between a street view of each floor. Would I need to do this where each floor is a different layer and you switch between them with an outside controller (combobox) or is there a function within the GMaps js API where you can link each sphere by height. IE: there is arrow going N, S, E, W but also ones for N, S, E, W at a 45 degree angle

    Reply
  • April 24, 2013 at 9:40 am
    Permalink

    The dimensions are pretty much always different. Check out the open source code here: github.com/googlemaps/streestphere

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *