Automated Fraud Detection Using AI

Appen is a provider of training data for organizations that build AI systems. In April 2020, they incorporated Figure Eight, an AI/ML-powered platform, into their solution offering to assist them in transforming text, image, audio, and video data into tailored, high-quality training data more effectively.

Challenge

Appen was looking to make their partially automated but mostly manual fraud detection system more efficient, in order to:

• Scale the number of distributed workers it can monitor per day to raise the bar for detecting and preventing malicious activity on the platform;
• Build on their existing ability to train ML/DL models for their platform through automated data labeling, annotation, and categorization;
• Reduce the amount of manual work done by distributed workers, to increase the speed and efficiency of data processing and to eliminate human error.

Appen used crowd workers to label data sets to train their ML and DL models. However, the company’s fraud detection system, which was used to guarantee data quality by eliminating low-quality contributions, was a manually activated solution that relied on SQL and Python scripts. This approach did not allow Appen to optimize the efficiency of their crowd workers, nor did it enable them to scale their fraud detection efforts, which was critical for the company to grow and attract new enterprise clients.

With more than 50 jobs running per day, it was proving difficult for Appen to manage this process manually. They faced the decision of either hiring 20+ data analysts or investing in an intelligent fraud detection solution based on AI and machine learning. In the end, Appen chose the latter and partnered with Provectus to design and build an automated ML-powered fraud detection platform with a scalable SaaS architecture and a graphical user interface to ensure optimum user experience.

Solution

Provectus developed an automated and end-to-end fraud detection platform with human-in-the-loop by designing and constructing data pipelines to simplify the Appen team’s ability to label, annotate, categorize and moderate data.

We also designed, trained and tuned highly accurate machine learning and deep learning models, which formed the AI core of the fraud detection solution. Furthermore, we developed a user-friendly web application for the Appen team to facilitate more efficient handling and management of data and alerts. Finally, out team ensured that all components of the fraud detection solution were automated and properly integrated for maximum efficiency and ease of use.

Prior to the development process, the Provectus team conducted thorough research into the latest academic papers related to crowdsourcing and fraud detection in order to ensure that the envisioned solution met all requirements.

TensorFlow was used to create the machine learning and deep learning components, which made up the AI core of the solution. For deployment, serving, and monitoring, Hydrosphere.io was installed on Amazon ECS. Java microservices, SQS events, and a React.js UI application were used to automate the data pipelines and provide an end-to-end user experience. Continuous monitoring was enabled by employing Prometheus and Grafana, enabling Provectus to provide transparency into the performance of the solution to Appen’s engineers, fraud analysts, and business stakeholders.

Outcome

Appen’s new ML-powered fraud detection platform enabled the team to process and handle 20x more jobs per day, with almost 97% of all jobs handled automatically. This platform, with its more efficient deep learning algorithms at its core, allowed Appen to reduce scammer activity by no less than 25%, resulting in a 5x drop in churned judgements.

This fraud detection platform brought about increased productivity and efficiency of staff, in addition to the elimination of the need to hire 20 or more data analysts. This, in turn, resulted in considerable cost savings in the long run.

Most importantly, the new fraud detection platform helped Appen meet customers’ requirements in regard to data quality and service efficiency, allowing the company to better satisfy existing customers and attract new enterprise clients, thus accelerating and making their global expansion more sustainable.