At MotionDSP, we have a lot of business development meetings with hardware companies who need image processing and computer vision software to augment their products and differentiate them in competitive markets. We often see that many are trying to build software teams of their own, as their long-term strategy is to make software in-house. Several companies we have spoken to have vastly underestimated the challenges of creating an computer vision team, and all of them have eventually come back to us after their own internal efforts fell short of expectations.
We don’t fault the developers at those companies, nor their management. They simply didn’t know what they were getting into and the complexities involved. They may have thought that their existing expertise in embedded software design or DSPs could quickly translate to the high-performance cloud/desktop world, or they simply didn’t understand that open-source algorithms don’t always work for their specific use case. Just one example we’ve seen of this is body camera manufacturers using open-source tracking algorithms that aren’t designed for moving cameras.
Software is essential to add value to the hardware experience
Whether you make drones, body cameras, high-end ISR gimbals, video management systems, or even consumer cameras, software that processes video pixels is essential. As companies like GoPro are finding, it’s not enough to just capture and store video. A complete end-to-end solution means the user has software that enables them to do something with the video they capture rather than just keeping it stored somewhere.
We see a lot of companies coming to the same realization as GoPro. In the markets we mentioned earlier, we’ve regularly offered them advice based on our own experience over the past 11 years of making high-performance, real-time image processing and computer vision software. Below are five things hardware companies need to consider before getting into software development.
1) Accept that you don’t know what you don’t know.
As the Boston Consulting Group lays out in their article, “How Hardware Companies Can Win in the Software World,” hardware companies don’t have the skills or the agile culture required to succeed in software development. As we can attest to, writing cloud and desktop software that uses advanced image processing and computer vision is complex. It’s different than writing embedded software or DSP, requires a different type of expertise, and takes a team with years of experience and know-how. There are no short cuts. We’ve seen companies hire a bunch of young developers, stick them in nice offices with foosball tables and expect them to create miracles.
2) It’s going to take years.
Do not underestimate your opportunity cost. To do this in-house, you need to build a great team, give them a really clear objective, and keep them focused while they do the hard work. You can’t build these kind of teams overnight.
Once you have your team, they need clear goals, focus and time. Lots of time. We licensed our initial super-resolution algorithm from UC Santa Cruz in 2005. We had our first TechCrunch article 18 months later, and that was a demo of a simple algorithm, not a product. It took years, and many more algorithms, to make a mature product and years of effort since to adapt it to new market challenges. You can’t expect your new team to produce industry-leading capabilities in less than a few years.
3) Open-source software probably won’t work.
Open-source software is great for quick prototyping and experimentation (as is MATLAB, for that matter), but it rarely works in real-world applications that require robustness and performance. When our R&D team approaches a new problem, they try every available method including open-source options like OpenCV. These issues are common:
- Robustness: the algorithm just doesn’t work, or it doesn’t work reliably in your use case. Or worse, it works on some carefully-selected video sequences, but falls apart on real-world data. For example, some trackers work great with fixed cameras, but utterly fail with moving cameras such as body cameras.
- Performance: the algorithm doesn’t give you the performance you need. Some algorithms are really robust, but take an enormous amount of computation. We encountered this with a feature detector we were using for wide area/large format imagery. We ended up porting the feature detector to the GPU, parallelizing it, and we got the performance we needed. That required a very experienced team, and what we did is now patented.
- Scalability: an algorithm might do well with points 1 and 2 above, but fails when it needs to scale from a single prototype to thousands of nodes.
4) Engineering know-how is key.
I can’t stress the importance of an experienced team. There are no shortcuts to building great intellectual property. It takes years to create the software building blocks, as well as the experience it takes to both know how to do something and how to understand when something “doesn’t” work. Many companies experiment with open-source software, but if you don’t have the know-how, open-source software is a black box. When something doesn’t work, you are permanently stuck.
5) Competition is fierce.
Accept that your competitors are investing heavily in software. GoPro realized this a few years ago and has built up their software team to more than 100 people. Apple has hundreds of engineers devoted to the iPhone camera alone. Companies like Google, Facebook, and Twitter have hired numerous computer vision and image processing engineers. Uber hired an entire robotics department from CMU.
Want to hear more about our experiences? Reach out to us at email@example.com