Data is collected from multiple data sources across the enterprise, SaaS applications, edge devices, logs, streaming media, flat files, and social networks.
Based on the type of the data source, AWS Database Migration Service (AWS DMS), AWS DataSync, Amazon Kinesis, Amazon Managed Streaming for Apache Kafka, AWS IoT Core, Amazon AppFlow, and AWS Transfer Family ingest the data into a data lake in AWS.
AWS Data Exchange integrates third-party data into the data lake.
AWS Lake Formation builds the scalable data lake, and Amazon S3 is used as the data lake storage. AWS Glue Data Catalog is a centralized metadata repository.
AWS Lake Formation also enables unified governance to centrally manage the security, access control, and audit trails.
AWS Glue and AWS Glue DataBrew catalog, transform, enrich, move, and replicate data across multiple data stores and the data lake.
Amazon Managed Service for Apache Flink is used to transform and analyze streaming data in real time.
Amazon QuickSight provides machine learning (ML)-powered business intelligence.
Amazon OpenSearch Service offers operational analytics.
Amazon Redshift is a cloud data warehouse. With federated queries, you can query and analyze data across operational databases, data warehouses, and data lakes.
Amazon EMR provides the cloud big data platform for processing vast amounts of data using open-source tools.
Amazon SageMaker and AWS AI services can build, train and deploy ML models and add intelligence to your applications.
Amazon Redshift Spectrum and Amazon Athena enable interactive querying, analyzing, and processing capabilities. Athena supports Apache Iceberg for data and AWS Glue data catalog.
Amazon Aurora offers high performance and availability at global scale. Aurora supports zero-ETL integration with Amazon Redshift.
