The use of computer vision in healthcare is exploding as medical professionals continuously look to improve care, make more accurate medical decisions, and lower costs. In fact, the Global Computer Vision in Healthcare Market is expected to be around $416 billion by 2025 due to the increasing demand to extend the adoption of AI-based technologies.
While there is immediate benefit from physicians analyzing individual medical images at the point of care, there is potentially more value in viewing images from X-rays, CT scans, MRIs, and other technologies in the aggregate to identify patterns and trends not always visible to the human eye.
Computer vision using AI and machine learning makes this possible. The foundation of successful projects starts with annotating and labeling thousands of images to train algorithms. While there are many different tools and platforms available, it is critical to understand, especially in healthcare where precision is required, that every data annotation tool is meant to be used by a human workforce - even tools that include AI-based automation features.
Increasingly, medical AI companies are turning to experienced managed workforces to get started and scale data annotation projects, especially ones that have extensive experience with both the industry and annotation tools.
It’s equally important to understand that incorrectly annotated images will result in computer vision-based projects failing, and the cost of that failure can be high. Not only does it include potentially scrapping thousands of hours wasted time on a failed project, but basing medical decisions on incorrectly labeled medical images can result in delayed patient care, inaccurate medical diagnosis, or worse.
CloudFactory — with over a decade of experience annotating images for thousands of clients, including medical AI — has developed a set of best practices to make medical computer vision projects successful. Our medical AI team builds long-lasting relationships and true partnerships with our clients, and it shows.
As part of these successful engagements and our presence in the healthcare world, we’ve curated the top five questions on using a managed workforce that we get from our clients and prospects.
Managed Workforce for Computer Vision in Healthcare: Top 5 Questions
1. Medical experts have always been involved with annotating training data for complex AI use cases. How can a managed workforce deliver the same quality?
A managed workforce is able to focus on routine, rote tasks for long periods of time — it is their primary job, and they are vetted for their ability to do it well. Because they are trained by the experts (e.g., medical doctors, radiologists, etc.), and continuously iterate and improve over time, they develop data annotation competency that matches or exceeds the experts. The AI in turn can develop the ability to outperform experts’ ability to identify diagnoses. Building tight feedback loops with our clients and managing our teams closely enables Cloudfactory to consistently deliver high-quality annotations that meet or exceed quality goals.
The medical industry moves fast and it’s important that your managed workforce work as fast, understand fundamental industry shifts, while maintaining their focus on quality.
One example of this is CloudFactory’s work with Sartorius to segment and annotate complex cell imagery for AI cell identification to help spur medication development. In a two-year engagement with Sartorius, CloudFactory annotated 1.6 MM cells, exceeded quality goals, and saved years in developing the largest dataset in cell imaging.
There are four ways we measure quality for managed workforces:
- Gold standard - There’s a correct answer for the task. Measure quality based on correct and incorrect tasks.
- Sample review - Select a random sample of completed tasks. A more experienced worker, such as a team lead or project manager, reviews the sample for accuracy.
- Consensus - Assign several people to the same task, and the correct answer is the one that comes back from the majority of labelers.
- Intersection over Union (IoU) - This is a consensus model often used in object detection within images. It combines people and automation to compare the bounding boxes of your hand-labeled, ground truth images with the predicted bounding boxes from your model.
You’ll want the freedom to choose from these quality assurance methods instead of being locked into a single model to measure quality. At CloudFactory, we use one or more of these methods on each project to measure the work quality of our own data labeling teams.
2. How do I know a managed workforce can start and scale fast when working with medical AI machine learning models?This is not a problem for a data labeling team that has experience with medical, healthcare, and life sciences data. You should assure that your managed workforce provider has dedicated teams with the expertise to handle the most extensive healthcare datasets in the world and the ability to ramp quickly and complete projects with millions of annotations.
These projects would have taken years with in-house resources but can ramp up in as little as two weeks with a managed workforce provider. CloudFactory makes it easy to scale your data annotation work up or down as needed. To add more capacity, just let us know what your scaling needs are and your Client Success Manager will facilitate a process change. Our subscription model enables you to unlock more savings as your work volume increases.
3. Isn’t a managed workforce more expensive than using internal teams?The cost of scaling data labeling projects with in-house teams or a crowdsourced model can quickly become prohibitively expensive.
The right managed workforce can scale to millions of annotations in months faster and more affordably than internal teams. Our CloudWorkers have become experts in specific market segments for data annotation, including medical and life sciences image and video data annotation.
Additionally, with a managed workforce and tooling option combined, the expense and hassle of directly managing a data processing workforce and a complicated and evolving tech stack is eliminated.
Perhaps the most significant cost savings when using a managed workforce is the skills and expertise to manage change. AI "drift" — when the original context changes but the model doesn’t — is a real concern for production models that are not consistently well maintained.
Ask yourself: How expensive is scaling with the wrong data annotation workforce (i.e., developing an inaccurate or ineffective machine learning model)?
The right managed workforce will become an essential component of your machine learning team. They should be properly trained by medical professionals or data scientists on your team on how to perform the tasks at hand in a secure manner. While patient-specific information is necessarily anonymized, there are additional, important security and healthcare-specific aspects to the work covering people, processes, or technology — including HIPAA, GDPR — that must be considered by our clients.
Your data annotation workforce should be certified in SOC2 and ISO-9001. These are gold-standard certifications that ensure your service providers securely manage your data to protect the interests of your organization and the privacy of its clients. Both SOC2 and ISO-9001 broadly cover a number of quality management principles including a strong customer focus, the motivation and implication of top management, and the process approach and continual improvement.
Other certifications may or may not be necessary for your particular data needs, but whatever your needs, your data labeling vendor should be prepared to respond to your concerns about data privacy and security. In some cases, your project may require specific and tailored security protocols. CloudFactory aims to work with you to address, and in some situations customize, our approach to ensure data security and privacy meet your needs. Learn more about CloudFactory’s robust approach to data security and privacy.
5. When building AI in healthcare and life sciences, how important is building trust and communication with a managed workforce?
One of the things our clients value the most is that CloudFactory integrates managed teams with internal teams. From day one, this avoids trust and communications issues you may have experienced with other vendors. These issues often stem from low-tech collaboration tools, sporadic meetings, lack of real-time chatter, and poor feedback loop processes in place.
In addition to dedicated client teams, CloudFactory utilizes dedicated messaging platforms and collaboration tools that provide real-time access to project status. Another key element to building trust is our platform can predict worker productivity and identify areas for improvement before problems arise.
As the CEO of V7 says, “CloudFactory is interested in maintaining a relationship. You are interested in more than just taking the job and doing it. There is a larger interest in the whole project and it creates this feeling that you are an extended part of our team. That’s more attractive than an outsourcing company that is more transactional.”
Why is CloudFactory your best choice for medical AI data annotation?
CloudFactory has proven skills and a trained data annotation workforce to understand the nuances of labeling medical images. We’ve successfully annotated millions of medical images for a range of clients working in medical diagnostics, robotics, and research.
Learn how our managed workforce supports medical AI including labeling diagnostic and clinical imaging here.
