Adjacent Illustrations

Recently I was asked to create a few illustrations for Adjacent, the online journal published by ITP.

The next issue of Adjacent will have an article on the Chinese influence in Taiwan. The suggested idea was to use a style transfer to mix Chinese art or pictures with Google Street View pictures of Taiwan.

I could have used the same code I used for the videos I created for the Spring Show to make a short animation but I didn't think that was the best idea for this. Instead I came up with the idea that I would pick relevant locations and animate them by continuously rotating the view to the right or left. This required me to do a continuous style transfer on a panorama image because I didn't want the right and left edges of the panorama to be visible. If I just did a style transfer the normal way the style applied to the left and right edges would be different and would not match up correctly. There would be noticeable seam.

My existing code could not do what I wanted and writing new code to implement this would require some algorithmic creativity. TensorFlow isn't the best library for that sort of thing but I knew PyTorch was; therefore I decided to re-write my code using PyTorch and code my approach for an algorithm to do a continuous style transfer on a panorama. So in a blaze of computer science glory, I got up to speed on PyTorch, re-implemented all of the style transfer code using the new neural network library, and coded my new algorithm for panoramas. All in one weekend. And it worked!

I'm quite pleased with the result. I haven't seen anything like this on the Internet so I am naming my algorithm a "cylindrical style transfer." And the best part about this algorithm is it validates my ideas and approach for doing a 360 degree style transfer. There is still more work to be done to make that happen but I now see a clear path for achieving it.

And now let's have a look at the illustrations I made for the Adjacent article.

The first picture is located in Taipei near the Lungshan Temple and Bangka Park. The painting is Map of "Art and China after 1989: Theater of the World", 2017, by Qiu Zhijie.

The second picture is located in downtown Taipei near Taipei 101, one of the tallest buildings in the world. The painting is All the Bigness have their Scars, 2008, again by Qiu Zhijie.

The third picture is located on the Jingshan Suspension Bridge in Yangmingshan National Park in Taipei City. The painting is Sunshine and Solitude: Shadow, 2010, by Cai Guo-Qiang.