Years of video are recorded and uploaded to the cloud each day to sites like YouTube, Vine, and Facebook from devices like mobile phones, personal drones, or home surveillance cameras.With the new capabilities of computer vision technology, we have the ability to mine this visual data to get valuable insights about what’s happening in the world. However, video content has largely been ignored in the conversation about big data.
Video as a data source has the power to give us information about how people interact with the world around them unlike any other data source. Video provides real-time data about behavior happening in the present, not just the past — whether it’s traffic patterns for the commute in and out of San Francisco or what time most kids get home from school in Palo Alto, California.
Video data stored in the cloud is exploding. Nest, Google’s internet of things investment, just bought Dropcam, a home security camera company. Silicon Valley companies are building fleets of satellites that will orbit the earth and be able to take thousands of high resolution photos or record video for up to 90 seconds of any spot, and revisit that spot several times per day. Inexpensive consumer drones like the DJI Phantom are selling in the thousands per week. Much of that data captured by these technologies is being uploaded to public sites such as YouTube where anyone can access them.
How can we extract value from all this video, in the way that we do from other “big data” sources?
To move forward, we have to get past our current fixation on the devices — drones being the largest example — and focus on the software that will extract the useful data they collect, and the answers that can come from that data. To find these answers we need to have a way to mine video content for data.
Until now, extracting intelligence from video content has mostly “manual,” through human observation — literally, a human watching the video and writing down what is happening. Think about a documentary filmmaker “logging” their footage, marking down interesting occurrences in the video, what they are, and when they occur. This was the process for police and FBI who mined hours of surveillance video collected after the Boston Marathon Bombings to find moments where the bombers passed the cameras.
To make video data into useful big data, we need to leap beyond this. We need true video analytics, powered by computer vision. Current search technology looks at the metadata stored with videos — like tags you include when you upload a YouTube video (“science”, “kids,” “cat dancing” or other keywords). Or, some sites use still image techniques (like classification) to find objects, cars, or company logos in videos. True video analytics involves using computer vision algorithms to analyze video pixels automatically, and over time — not only identifying objects in the scene, but also tracking their movements and behavior. With true video analytics you can track patterns of movement of hundreds of objects in the scene, as well as their size, shape, speed, and direction of movement.
To see the influence that data gathered from video would have, let’s take a mundane government question like, “Is it worth it to invest in new infrastructure to address traffic issues in the Bay Area?” Here in Burlingame, local citizens wanted a new crosswalk built across a busy intersection. The town had to employ a full-time person to stand and manually count every adult and child that crossed that intersection over a week — in order to measure how many citizens would benefit from that investment. This could have been accomplished with a single surveillance camera and video analytics, measuring people (adults, kids), and vehicles, turning the data into something that is actionable (and provides good ROI).
We could assess things like how many, and what type of vehicles are using a new highway overpass? Were the tax breaks given to the company who asked for it worth it? Is the local sewage facility operating within regulations? Are there sewage outflows after heavy rains? How big are they?
Over the longer term, as video analytics of aerial or space data become more widely accessible, it will change the way we understand the world. It will give everyone access to the data that, until recently, only militaries and intelligence agencies had at their disposal. It will let everyone see for themselves what is really happening around the world and disrupt the current notion of the big data revolution. This is true freedom of information and it is on the horizon.
