Improving model results with quality data annotation
A sensor-as-a-service company (SSC) turned to CloudFactory to help improve the performance of their application after several unsuccessful attempts at outsourcing data work. The CloudFactory team tags pertinent objects in sensor imaging, from one frame to the next.
Retail
California, USA
11-50
For a machine learning startup, the fourth time was a charm.
When SSC came to CloudFactory, it had worked with two other vendors who couldn't achieve the accuracy needed for its business intelligence platform.
SSC also tried to do the work on its own. None of the options were working like its Chief Technology Officer wanted.
Once SSC collects sensor data, it needs to be manually analyzed and then run through its machine learning algorithm. The company tried three options before turning to CloudFactory.
“When we started the company, it was me sitting in a cafe all day tagging pieces of data and drawing boxes around relevant objects and doing that kind of thing. Obviously, that wasn't scalable,’’ the CTO says.
Next came attempts to use on-site contract workers. The contract workers came and went and were inconsistent even with the CTO there to train them. The crowdsourcing vendor chosen next wanted the work done in their tool, while the company had its own. None of the methods provided reliable results.
“With crowdsourcing, we would explain the task and kind of throw it over the fence and hope it was done correctly,’’ the CTO says. “But the accuracy was just horrible because it’s a task that requires just enough context and understanding.’’
Frustrated with these efforts, a company employee suggested CloudFactory as he had a positive experience with us in a previous position. “When I heard about CloudFactory, it seemed like a perfect fit because of the dedicated workforce,” the CTO says.
We have been through a full production cycle of data sent through CloudFactory. We’ve fed that back into our model, trained it, deployed it and seen much, much better results and model accuracy.
SSC Chief Technology Officer
CloudFactory provides a managed workforce with a project manager located where the work is being done, providing a vital layer of management to make sure that the labeling teams have the needed instruction and business context.
Our workforce can ask questions easily, which is helpful because some of the work is very nuanced. “We needed trained people to talk with and who could ask us questions like, ‘I saw this weird case, what should I do here?’ That just was a really important element of the process,” he says.
The CTO was thrilled by how quickly the CloudFactory workforce ramped up. “The team’s output has almost outpaced our ability to keep them fed with data. It’s making us look at what else we can do.”
“It’s easy to discount how hard it is to find people that are good, affordable, consistent and reliable. It’s hard to hire for that, but CloudFactory has it figured out. Not only is it way less expensive than if we did it ourselves, it just takes so much pain off our plate.’’
The tagged data is fed into SSC’s model and the results have been exactly what the CTO sought. “We have been through a full production cycle of data sent through CloudFactory. We’ve fed that back into our model, trained it, deployed it and seen much, much better results and model accuracy.’’
The CTO says he also feels confident that as the company grows, he’s got a partner that can help him grow. “The CloudFactory team can support any big customer use cases that might come up, as opposed to trying to come up with a mechanism for scaling up some other service ourselves.”
Even more importantly, it’s freeing the company to expand its horizons.
“We want to recommit to being a machine learning company, to go back to our roots of offering a do-it-all sensor,” the CTO says. “CloudFactory is not just the catalyst but the support to help us refocus on our vision. That is a cool story.”
Note: This client/company requested to remain confidential; therefore real names were not used.
We have 10+ years of experience helping our clients focus on what matters most. See what we can do to help your business.
Azavea interviewed a handful of leading data labeling firms, and studiously compared their pricing, approach, and cultural values. Learn why CloudFactory stood out.
Geospatial data analysis provides organizations with actionable intelligence about the visual world.
Geckomatics turned to CloudFactory to help scale their cost-effective solution to more customers around the world.
Fill out this form to speak to our team about how CloudFactory can help you reach your goals.
