Copy the document you’d like to run plagiarism detection on to Amazon Simple Storage Service (Amazon S3).
Amazon S3 event triggers start of AWS Step Functions workflow.
AWS Lambda function extracts text from document using Tika (a content analysis toolkit that detects and extracts metadata and text from over a thousand different file types.
For each paragraph in the document, text is passed to a pre-trained Bidirectional Encoder Representations from Transformers (BERT)-based model to extract word embedding vectors.
For each word embedding vector, a K-Nearest Neighbor (KNN) search is run using a cosine-similarity algorithm.
Amazon OpenSearch Service (OpenSearch Service) domain stores an index of pre-processed works that have been converted into word embedding vectors and indexed.
Based on the configured similarity threshold that is compared against the OpenSearch Service query result score, an event bridge event is raised, specifying source document information that has possibly been plagiarized with reference to relevant works.
