Thursday, March 10, 2016

Grad School

What I've been up to - some of my academic work at NYU, for the Master's in Scientific Computing(MS) program in the Courant Institute for Mathematical Sciences-

Some highlights:

Computer Vision with Davi Geiger (and upcoming thesis in Stereo Matching)
Early Result from Stereo Matching/Depth Finding

Original Left/Right Images courtesy of the Middlebury College 2014 Stereo Datasets


Computer Graphics with Ken Perlin

"Marble Maze" - Final Project, using ThreeJS and CannonJS



Monday, February 20, 2012

Fight or Flight: Hello, Cinder.

I am trying to learn Cinder, the more advanced (meaning complicated) C++ version of Processing, so I can port my GPS program to it and hopefully get it to run a bit faster. In the mean time, I am messing around with the tutorials and came up with an idea to model pedestrian traffic as a way to learn Cinder's particle engine.

In the process, I stumbled upon what I think may be a significant milestone in the machine learning and artificial intelligence fields: I have created a behavior model for the most basic of human instincts: panic.

In the first shot, two "pedestrian" particles approach each other, and collide, because they fail to adapt to the situation as time unfolds. In the second shot, they are programmed to "turn right" if they sense an oncoming collision and feel threatened. However, I neglected to provide them a means of relief, and lacking a "fight" instinct, they are doomed to perpetual "flight".


I feel this way quite often on the streets of New York City...

Sunday, January 8, 2012

Hanging out in Rumney

After leading nothing below 5.10 all weekend, Giuseppe attempts the upside-down second pitch of Giant Man (5.12 b/c) in Rumney's Hinterlands...
The nice thing about sport crags is that there's bolts everywhere - perfect for setting up photo ops. Here is our protagonist looking good on the approach:
Behind the scenes exclusive:

And hanging out, looking for the move...

Sunday, November 6, 2011

Calculus and Christmas Lights (or, the flicker problem)

This is a problem that I was wrestling with a little over year ago, building an LED prop for a friend's movie. That was before I started taking math classes...

To simplify it, many "efficient" light sources are efficient because they flicker- they are not constantly "on" like incandescent lights. Filmmakers love incandescent lights because they don't flicker (see note).* More efficient sources like HMI's are very carefully engineered to flicker in sync with the camera shutter, so that the camera doesn't notice the flicker. And most LED light sources are battery powered with a constant DC voltage, so there is absolutely no flicker.

This is where we come to the problem of LED Christmas lights- long strings of LED lights in series, that plug into AC power. LED's don't normally run on AC power because the voltage is too high, and LED's are at their hearts, diodes, which means they only pass current in one direction. Most LED circuits you plug into your wall go through both a transformer, to step down the voltage to a safe level, and a rectifier, to invert the negative half of the AC wave, and provide a constistent DC power supply.

But LED Christmas lights are an ingenious solution to this problem... almost. They take 60-70 LED bulbs, and wire them in series. LED's tend to drop about 1.2-2.0 volts individually, but put 60 in series, and you've got yourself a circuit that can drop 120v. That eliminates the transformer (wall wart) from the equation. However, it does not adequately address the flicker problem. LED's are still diodes, and if you have 60 of them, all pointing in the same direction, they will only be "on" when the AC wave is positive. That means, they will only be on for half of the AC wave.

You may notice this flicker if you look with the corner of your eye. The first time I noticed this was rigging on the set of a pretty big movie that will go unnamed. If you can see a light source flicker with your eye, a movie camera will definitely pick it up, and possibly ruin the shot (I wonder how much it cost to fix that...).

That's because movie cameras generally shoot at 24 frames per second, and the shutter speed is 1/48 of a second. That means that the camera only sees HALF of the time you're filming (they say half of every movie is darkness), and the camera ONLY sees things 1/48 of a second at a time. This being a film camera with a mechanical shutter. Digital cameras (and digital shutters) make things even weirder.

Hence, some math:


I've approximated the 60Hz AC power with the red sine wave, and the 24 fps camera shutter with the green sine wave. The LED's are only on when the red wave is positive, and shutter is only open when the green wave is positive (shutter speed is calculated from when the shutter is half-open to half-closed, probably because it's constantly moving).

So each positive half-wave of the green curve is one frame in the movie. The exposure (brightness) of the LED's in each frame would be equal to the intersected area beneath both curves, when both are positive. Keep in mind there are 24 frames per second:


This is done by adding up piecewise integrals. A1 is the "brightness" of the first frame, and A2 is the "brightness" of the second frame, which is slightly more than the first. Keep in mind these are abstracted, and don't actually correlate to any real measure of brightness. However, the proportions should be accurate, and Frame 2 is 18% brighter than Frame 1. That's not good.


Another interesting thing is that 5 AC power cycles = 2 camera shutter cycles. The waves line up every 2 frames, or 5 flickers. So all the even frames are the same, and all the odd frames are the same- which means every other frame is 18% brighter than the one before it. There's your flicker!

Thanks to Maple for doing the heavy lifting for me. I need to figure out how to code in Maple. It would make my life much easier...

*Note: Incandescent lights also "flicker" because they are on AC power, but they are so hot (read: inefficient) that the flicker is negligible.

Wednesday, August 24, 2011

RoboRadar

Working on making a radar type sensor with a Parallax rangefinder mounted on top of a servo motor. Left is raw distance data (2-300 cm), drawn in polar coordinates, and right is *supposed* to be corrected into rectangular coordinates. But they look eerily similar so I'm not sure that's working yet.

But the big accomplishment is the beginnings of an object detection system, as note by the white boxes. Basically if the points are close enough together, there must be an object there.

The problem is figuring out how close is close enough. As you can see, there are lots of tiny boxes that should probably be combined into one big box, as well as really big boxes that are overlapping smaller discrete objects.

Too many little boxes are a problem for memory & speed (right now it's sensing over 4,000 objects, but there are only 90 dots!). It would make more sense to combine them into one "wall" object, with the same effect.

The really big box is a problem, because the robot would see that entire area as an object, and avoid it; it would never move forward.

The moral of the story: weak object detection makes for slow, paranoid robots. Smarter = faster = more risk tolerance = more exploration!

Tuesday, August 23, 2011

Richard Serra Driving

Pieces of a Richard Serra sculpture driving toward the George Washington Bridge...

Finally, some art in the suburbs...

Thursday, July 28, 2011

Inspecting the Internet Plumbing

Ever wondered how a website gets from there to here? Me too.

Especially when "there" is halfway around the world...


The Black Dot is Brooklyn (because Brooklyn is too cool for silly colors). The white dots in the upper right are a website in Italy. The other white dots are the servers in between that the website had to travel through to get to me.

The signal went from New York to New Jersey (which are so close they overlap), to California out left, and then magically to Italy.

Pay no attention to the line going straight through the Earth. I actually have no idea how the signal got from California to Italy. Must be either a satellite or an underwater cable... but an underwater cable wouldn't go direct to Italy... very curious indeed.

Also, the white dot on the lower right is an anomaly. It actually represents the IP 0.0.0.0, which is a dummy IP (which I believe is my computer talking to my router). Since it's a dummy IP, it was assigned (0,0) as a location, and so it pops up at the intersection of the Prime Meridian and the Equator.

Other points of note are Greenwich, England (green dot on the Prime Meridian), and the Magnetic North Pole (yellow dot on the yellow meridian). The rest are cities around the world.

At some point I will have to figure out how to add continents and draw arcs around the world rather than tunnel straight through it... however convenient that might be. For some reason my problems keep coming back to moving things around spheres...

Saturday, April 30, 2011

A Different Perspective

Finally getting things to line up... but the perspective is not quite right. It looks fine on the top half of the mountain, but the horizontal trail should be much lower; in fact, part of it is behind the camera, which just ain't right...
This is the same view with placemarkers drawn in: the green cross is the GPS peak of Blackhead, which is the point where the photo is inserted. The red cross is the camera position (with compass and XYZ directions drawn in), and the pink cross is the peak of Black Dome mountain. There enlies the problem-

This is a side view of the whole thing, looking North. The red cross is still the camera, and the white line is where it's looking. Black Dome is behind the camera, but somehow the camera still sees it. I think this has to do with the ortho() function, again because the perspective on the path is not quite right. I've tried adjusting the clipping planes, but that doesn't change the perspective, only what renders and what doesn't render. So how the camera is seeing behind itself, I don't know...

But this is what it looks like with the "standard" perspective:

And this is where I think the trail should be:

Wednesday, April 27, 2011

Into the Void... aka Progress!



Making progress orienting the OpenGL virtual camera, and fixed the rotation of the image so it's [very nearly] square to the virtual camera. Note to self: degrees are not the same as radians.

In theory, all I need to do is make some fine adjustments to the image orientation to compensate for the pan and tilt of the real camera (when the picture was actually taken). And then get it to scale properly- the image should take up the whole gray area. And hopefully the perspective will correct itself with the scaling... hopefully being the key word.

Wednesday, February 23, 2011

Blackhead (annotated)

This is Blackhead Mountain in the Catskills, from a lookout on Black Dome, and somewhere in the middle is the trail that links the two. If I can figure out the scale of the photo, and the direction that the camera is pointing, I can use my GPS log to draw the trail.

I know the GPS position of the Blackhead peak and the camera, and using those I can calculate the distance and bearing between the two. But since the peak is off center, I need to figure out the direction the camera was pointing, even though it's only off by a few degrees.

The peak is offset by 114 pixels horizontally (the black triangle). If I can figure out the scale of the photograph accurately, I can use this to figure out angle the camera is pointing. Of course, the original image is much larger, but the ratios should stay the same.

If I can figure out the scale and rotation of the image, I should be able to line up the GPS log of the trail with the picture...

This is a crudely oriented view of the GPS trail. The red cross is the camera position, and the green cross is the peak of Blackhead. So all I really need to do is line up those two points over the picture... simple enough, right?


Ultimately it should look something like this, but with more picture and less black void...

Monday, February 7, 2011

Mt. Hight, New Hampshire

3D view
Elevation vs. Time


Total Time: 5:51:49
7.93 miles
1.36 miles / hour (not taking stops into account)

1472.1 feet (base elevation)
4706.5 feet (peak elevation)
4079.3 feet (elevation gain)

Red: <>Yellow: <>
Green: > 1.0 m/s

Monday, December 27, 2010

Eclipse Timelapse- Second Attempt



Heard there was going to be an eclipse, so I broke out the 5D and tried my very best to catch it.

This is one frame every 30 seconds, starting around 2:30am and going for about an hour and a half, using a 105mm lens.

Timelapse is complicated because you really can't recompose in the middle of a shot- you have to plan it out ahead of time, lock off the tripod, and cross your fingers. Or build a motion control rig- maybe next time...

The moon was much smaller at 3:30am than it was earlier in the night, so I zoomed all the way in, and the moon was still tiny. This reduces the frame size which is why it kept falling out of the frame. Luckily I caught it, and for lack of anything better to do, recomposed the shot.

I also made a big no-no by changing the exposure toward the end (when it gets redder), which kind of killed the subtlety of the eclipse... but after seeing this, I have a lot more faith in the low end of the 5D's exposure range.

Next time I think I'll try shooting earlier in the night, when the moon is huge, so I can use a wider lens. It will probably take a few more tests to get a feel for the passage of the moon, so I can frame it better.

I need to make friends with an astronomer, and practice some more before the next eclipse...

Friday, December 17, 2010

Pictures!

Updated the Elevation Profile part of the program... same GPS Log (Catskills Blackhead Range) with new graphics & live animated elevation marker...

Will elaborate later, but here's some pictures for now!


Monday, November 1, 2010

Hiking the Catskills in 4D!

This past weekend I had my first introduction to what is known among the hiking community as "Peak bagging". From wikipedia:

Peak bagging (also hill bagging, mountain bagging, Munro bagging, or among enthusiasts, just bagging) is an activity in which hillwalkers and mountaineers attempt to reach the summit of some collection of peaks, usually those above some height in a particular region, or having a particular feature.

We bagged three peaks, two of them twice, since we had to retrace our route to get back to the car. This is known as a lollipop loop:

This is the overhead 2D map view. Latitude vs. Longitude. Pretty simple. And now that I finally have some interesting altitude data, I started working on the third (and, inadvertently, fourth) dimension to my program-

This is the Altitude Profile of our hike. Altitude vs. Time. If you notice, the first two peaks and last two peaks are nearly mirror images of each other, which is an interesting result of the lollipop loop.

The heights of the peaks are the same, but the widths are different, probably because I was going faster on the way out. It was starting to get dark...

So, now I have two 2D abstractions of our hike. Now, all I need to do is mash them together (that's a technical term) and I'll have 3D!

Thursday, October 28, 2010

Saturday, October 9, 2010

Shooting the Moon



Live-blogging the shoot!

Using Processing to control the moon. The physical controls are nice, but limited in their capabilities. Very handy for Backpack Mode, so the kid can run around in the woods with the thing. But Processing allows for much more fine control, and better random number effects (and processing speed!) too.

Friday, October 1, 2010

Moon Timelapse Test



Shot this "manually" by holding down the shutter button until the memory card got full, hence the jumpiness. Or as I like to call it, the "hand-cranked aesthetic".

Just one more reason to get back to work on the Arduino intervalometer- Check out the Camera Axe page for more info.

The beautiful thing is that the original images are really big- 21 Megapixels big, in fact. 35mm movie print, anyone?

Tuesday, September 28, 2010

IT'S A BOY!

I think I'm going to call it Red, short for "The Great Red Shark". It's fast. Of course, I dropped it within 5 minutes of leaving the store, but scratches build character.
And it's recycled, kind of. At least the frame is. Most of the important parts are new, which is a good thing. Custom built by Brooklyn Bike & Board. Same day service, who would have thought?

All this (and a lock!) for $100 less than my old bike. I feel like I might wind up spending less on the bike, and more on locking it up... but that's the world we live in, unfortunately.

For more information on how to tell a boy bicycle from a girl bicycle, ask your parents.

Thursday, September 23, 2010

So it goes...

If ever I was going to eulogize an inanimate object, this would be the one. But a bicycle is hardly an inanimate object. A bike embodies the phrase "I want to go fast." In the words of the late Kurt Vonnegut, so it goes...

[Cue sappy music and "trip down memory lane" montage]

This was the day I rode all the way to the end of Bedford Ave. Note the boat in the background. Not pictured: the Applebees on the corner. Nobody wants to see that...

Manhattan Beach, same day.

The greenest grip truck in the city. Not going to lie, this was a little unwieldy, but I didn't have far to go.

Saturday, September 18, 2010

Battery Stuff

Just a note- using a DeWalt XRP 14.4v battery for the "Backpack Mode" moon (portable & self contained, so the kid can run around with the moon lit up).

Ran most of the day. Forgot to meter the battery at the start of the day, but about halfway through the day, it metered 15.7v, and by the end of the day it was only down to 15.6v. I guess the LED load pales in comparison to the DeWalt drill motor... I have a feeling this will last for a while, despite not having done the math to confirm this fact...

Also can't find the mAh rating for the battery, but maybe I'm not looking hard enough.

Shoot day 2 coming up... just found out the moon has to get wet. That should be fun...