All batch inputs, like SSIM, are loaded into a batch staging bucket in Amazon S3. All real-time data feeds, like SSM and FLIFO, are loaded into a real-time staging bucket in Amazon S3. AWS Glue and Amazon EMR based processes are used to discover, catalog, process inputs, and create the processed data in S3. These processes combine batch and real-time data to create the processed flight data. Flight data is created by converting the schedule files into individual flights and then loaded into Amazon DynamoDB for serving direct flight lookup. Flight lookups are enabled through AWS Lambda and Amazon DynamoDB, with inmemory caching provided by Amazon DynamoDB Accelerator (DAX). Additionally, FLIFO events can be ingested and served along with flight data. Routes with mileage, day of week, seasonality, and airline type are processed and loaded into Amazon Neptune Graph database to provide route lookups by origin and destination. Duration and mileage rules are applied, flights are retrieved for each route and combined with connection rules to create the full schedule. Amazon ElastiCache for Redis is used to improve performance. Connection rules are maintained in Amazon DynamoDB for fast retrieval for usage in connection building. These rules can be managed with a connection rules mana
