Thesis Project Idea

Portfolio

First, my portfolio, available on my Projects page.

There are a few projects I will highlight that I identify most strongly with.

• My first published research paper, Algorithmic Trading in the Iowa Electronic Markets. This was published in 2011 when I was still working in the finance industry, long before I had even heard of ITP. The paper documents a (still ongoing) project where I wrote a computer program that could profitably make financial transactions on my behalf. In addition, I am writing on a second paper with David Rothschild of Microsoft Research Labs that is currently working its way through the publication process.

• My first Processing library, Camera-3D. This library enhances Processing's functionality and allows users to create stereoscopic and 360 renderings. It is downloaded and installed 20-30 times a day by people all around the world. I also created a second library called ColorBlindness, which happens to be a fork of Camera3D.

• My first semester at ITP I took a 3D printing class and did a 3D Printed Tesseract project. I printed a series of objects depicting a tesseract rotating in 4 dimensional space and made a stop-motion animation with them.

• My first semester I also made an After Effects VFX Project that I spent more time on than any other assignment. For this project I figured out the math behind the visual effects in mustardcuffins' simulacra video and implemented it in After Effects.

There are common themes that run through these projects. All of them are computational, and all of them are mathematical. From the math behind the tesseract rotations to an optimal strategy for 360 renderings, I've always been driven to understand the math behind what I am doing. The goal is to understand things deeply because I know that achieving that is (for me) a source of many creative ideas.

These themes make sense. I've always been interested in math and computer science. For my undergraduate education I double majored in computer science and math and have taken many scientific computing classes as an undergrad and later as a grad student at NYU. I also worked in the finance industry as a quantitative researcher for over a decade. Math and computer science are important parts of who I am.

Something I enjoy very much is using my math and computational skills to create new artistic tools. My opinion of the creative-tech space is that there are a lot of things that are possible and computationally feasible that aren't being done because the right tools don't exist. I want to build those things, and then I want to use them to expand people's ideas about what is possible.

Thesis Idea

I am developing a new algorithm for applying style transfers to 360 photographs and video and will document my approach in a research paper. In addition, I want my thesis to be about creating something that has artistic value and not just a technical achievement. I’m going to be researching painters and artists and trying to figure out how to convey conceptual meaning through style choices.

This project is a continuation of my project from last year's Project Development Studio class. It's a challenge by brain has latched onto and I'm excited to immerse myself in this project.

I would like for the output of my thesis to be a published research paper, an open source code repository for 360 style transfers, and a series of 360 photos and videos that have artistic merit.

Progress So Far

Since the conclusion of last year's Project Development Studio class I have continued working on this idea. I have made two significant accomplishments.

First, I created illustrations for an adjacent article by Lee-Sean Huang. These illustrations are something I'm calling cylindrical style transfers. Observe that the style transfers are applied to the images in a way that is continuous and leaves no edges.

A major shortcoming of those adjacent illustrations is the computation time. It took several hours to compute each panorama with a size of 2000 by 500 pixels. At the time I estimated that a 4K 360 image (4096 by 2048 pixels) would take 20-30 hours. That's unacceptably slow.

Over the winter break I addressed the performance issues and improved the speed by an order of magnitude. The adjacent illustrations can now be recreated in under 5 minutes. Images twice the size can be computed in under 10.

And I still have a bag of tricks I haven't tried yet to make it even faster.

My next step is to do a style transfer for 360 photos. I have written code to do this but I am not satisfied with the results. I am going to keep working on it in parallel to my other thesis work. My goal is to get my code working properly as quickly as possible so I can spend most of the semester using the tool to create something artistic and not writing code.