This not at all scary new computer program allows any camera to see around corners


The fact that we live in a world where surveillance is becoming more and more common will probably come as no surprise to you. But even when you’re out of sight, you might not be safely hidden: Researchers have developed a computer program that allows cameras to see around corners.

The technique is called computer periscopy, and it works by analyzing shadows cast on a wall and applying very powerful decoding algorithms to them. The end result isn’t perfect, but it’s very impressive (see image below).

While the thoughts of Big Brother watching you may immediately spring to mind, technology doesn’t need to have such a dystopian goal.

It could, for example, help autonomous cars avoid crashes or improve the navigation skills of autonomous robots working in disaster areas.

Some reconstructions of images. (Boston University Goyal Laboratory)

What really sets the program apart is the way it can be applied to an image captured by any digital camera – you don’t need any special equipment.

“It was thought that it was virtually impossible to reconstruct an image just from light scattered from a wall without any advanced instrumentation,” said optical physicist Allard Mosk of the University of Utrecht in the Netherlands, who does not did not participate in the study. Nature.

For a source image, the program needs an image of the wall receiving the light of a scene and the shadows cast by an object hidden in the corner. Specifically, it needs a penumbra – the outer edge of a shadow cast by an opaque object.

The penumbra are most often evoked in relation to the shadows cast by the planets and the moons, but here the algorithms developed by the researchers can work backwards from them to reconstruct an image of the original scene.

The algorithm is basically to decipher the light. When the light from a scene hits a mirror (as in a conventional periscope), no descrambling is necessary, as the light travels without interference.

In this case, heavy calculations are used to eliminate interference and turn a dull wall into something like a mirror.

clear corners 3The configuration of the experience. (Nature)

Above all, there must be an opaque object blocking the scene, with dimensions and a shape that the algorithm already knows. This helps the program understand how the light was scattered and how to replenish it.

“Based on the optics of the light rays, we can calculate and understand which subsets of the appearance of the scene influence the pixels of the camera,” explains one of the team members, the engineer. Boston University electrician Vivek Goyal.

“It becomes possible to calculate an image of the hidden scene.”

Cameras have been doing stuff like this with scattered light for several years, but here everything is done in software rather than in the camera itself.

Even though a specific scenario (with an opaque object) is required, as well as strong lighting illuminating the object, this is another tool that cameras of the future may be able to call upon when needed.

The team believes that eventually the algorithm may be able to determine the dimensions and shape of the opaque object itself.

“In the future, I imagine that there might be some kind of hybrid method, in which the system is able to locate opaque objects in the foreground and take them into account in the computer reconstruction of the scene”, explains Goyal.

The system will only get better over time. Right now, it takes around 50 seconds to reconstruct a scene from the light and shadows scattered across a wall, but the team thinks that could be improved.

Eventually, it might be able to process real-time video footage, the researchers say – but they hope it will be used for positive rather than sinister uses, like finding burning buildings or keeping people safe on the roads. .

“I’m not particularly excited about the surveillance, I don’t want to do scary things,” Goyal told Ian Sample at the Guardian.

“But being able to see that there is a child on the other side of a parked car, or seeing a little around the corner of an intersection could have a significant impact on safety.”

The research was published in Nature.


Gordon K. Morehouse