Two weeks remaining in the class and I finally was able to make progress on writing my own style transfer code. I didn't write it from scratch, I rewrote the existing code I found online to improve the code quality and make it fit my workflow. I'm going to continue working to make further improvements over the next few days.
Today I am meeting with ITP's Feedback Collective to discuss this project. I'm going to use this blog post to present what my project is about so this post will provide a review of what I've accomplished.
The basic idea is to take Google Street View photos and stylize them to make them more visually appealing. Although the Google Street View tool is an amazing application the photos taken can be fairly dull looking. Google is working to address this but they are aiming for a high degree of realism. My goal is to take a much more artistic approach.
Our class is half over and it is time for our midterm full-class feedback session. I've made a lot of progress recently and am excited to share my work with my fellow students. I will definitely achieve my original goals for this class, however, if that's the only thing I do with this, it would be a crime. This project is absolutely going to live on long after this class is over.
Milestone #2: Data Assembly¶
I'm comfortable saying I've completed this milestone. I've finished all the major features and have a nice interface for interacting with the downloaded data.
There are a few minor issues but none require a lot of time or brainpower to implement. Mostly nice-to-have enhancements like better error checking in my code that I feel compelled to do but aren't critical right now. I'll complete them as time allows.
The important thing is that I can now begin downloading lots of data without fear that I will need to download everything a second time later.
I made an interactive tool in matplotlib to visualize a spatial map of the locations I've downloaded data for. It looks like this:
Milestone #2: Data Assembly¶
The second step of this project is to access the Google Street View data and organize it in a suitable format. In my project plan my target was to reach this goal by February 21st (last Wednesday). Although I have accomplished a lot, I have not achieved all of the things I wanted to achieve for this milestone. I expect to hit it by next week at the latest.
Here's what I have achieved.
First, I can download all of the relevant data from Google. This includes all of the panorama image data and meta data. I can also access the panorama ids for the neighboring locations. All of the metadata is stored in a database.
This week I was busy writing Python code to download and store data and JPG files retrieved from Google's Streetview API. I am happy with my progress. I can reliably download data and images from Google without any problems. My code begins with the latitude and longitude of a start location and will download all of the available Streetview data within a specified radius linked to the start location. It will crawl the dataset from one location to another, testing if a new location is within the given radius, querying neighboring locations, and downloading the panorama images.
Next I need to make sure I am storing the data in an accessible way. I expect to download hundreds of thousands of files (but no more than 25,000 a day, as per API limits) over time and want to organize it in such a way that I don't have to download anything a second time. I am using a SQLite database to store metadata on the downloaded data. This will keep everything organized and retrievable. Once I have this done I will let it start downloading data for multiple locations as I begin stockpiling data.
I am on track to meet next week's milestone of having something ready to feed into a Neural Network for a style transfer.
Milestone #1: Data Investigation¶
My first step is to investigate my data options for this project. As discussed in my plan, I am considering Google Streetview data and LiDAR data. The Streetview data is my first choice but I realized that that data might be different from what I expect or have weird complications that make it difficult or impossible to do what I have in mind. I wanted to consider alternatives, and there's a lot that interests me about LiDAR data. Of course that data might be impossible to work with too. In any case, I needed to find out these things right away while it is still easy to change course on this project.
The summary of Google Streetview data is that it is easy to work with and close to what I expected. They provide a convenient API that is properly documented. Unfortunately, the depth data discussed in this blog post does not come from the API, and that information is compressed in a format I have not yet parsed. The author of that post does provide C++ code for doing so; I am optimistic that I will be able to translate that to Python and/or integrate their process into my code.
LiDAR data is also well documented but extremely complex. I've worked with complex data before and am confident I can manage this if I put in the time. My objection is that taking the project in that direction would take a good portion of the class. I would have less time to learn about the topics I want to be learning about.
Additionally, I feel the challenges I would face with the Google Streetview data is resonating with me in a way that the LiDAR data challenges are not.
My conclusion is that I will use the Google Streetview data for this project. Sometime after the semester is over I might spend more time with the LiDAR data and get some experience working with it. It would be a great choice for a future project.
My conceptual motivation for this project is to create visually compelling animations using large amounts of spatial data.
Data by itself is formless. Our present technology community collects vast amounts of data. Some of that data is a record of the physical space we humans live in. My dream is to access that data and reimagine it in a way that maintains the integrity of the data and inspires viewers to appreciate the beauty of the originating physical space.
Project Development Studio, taught by Daniel Rozin.
Class blog posts:
- Tuesday, January 30, 2018 15:21 PM Plan
- Tuesday, February 6, 2018 23:11 PM Data Investigation
- Tuesday, February 13, 2018 23:58 PM Data Progress
- Tuesday, February 27, 2018 23:14 PM Data Assembly
- Tuesday, March 6, 2018 23:58 PM Data Assembly Complete
- Wednesday, March 21, 2018 12:14 PM Midterm Status
- Tuesday, April 3, 2018 23:13 PM Style Transfer Progress
- Wednesday, April 11, 2018 10:00 AM Feedback Collective Review
- Wednesday, April 18, 2018 1:43 AM Two Weeks Remaining...