Diverse producers deliver data into the data lake built on Amazon S3. The data first lands in an Amazon S3bucket for raw data. This is the first trace since AWS X-Ray enables trace messages for Amazon S3 Event Notifications. Next, data lake curation blocks prepare the data for analytics. AWS Step Functions trigger these processing blocks. AWS X-Ray integrates with AWS Step Functions to help you identify performance bottlenecks and troubleshoot any request that resulted in an error. AWS X-Ray can trace AWS Lambda functions. Lambdaruns the X-Ray daemonand records a segment with details about the function invocation and results. For further instrumentation, bundle the X-Ray SDK with the function to record outgoing calls and add annotations and metadata. Note: This will not record the duration of a AWS Glue job. Data curation continues with the AWS Step Functionsstate machine triggering the next curation block. Like the first block, X-Raycan capture Step Functions and Lambda functions. The data lands in an aggregated bucket, ready for analytics. This is the last trace in the data lake ingestion flow. With traces emitting from all key checkpoints in the flow, you can diagnose data lake processing lifecycle issues. Diverse analytics tools enable consumers to visualize the data from the data lake built on Amazon S3
