{"id":39602,"date":"2020-12-01T03:13:16","date_gmt":"2020-11-30T18:13:16","guid":{"rendered":"https:\/\/jirak.net\/wp\/amazon-managed-workflows-for-apache-airflowmwaa-%ec%a0%95%ec%8b%9d-%ec%b6%9c%ec%8b%9c\/"},"modified":"2020-12-01T03:34:11","modified_gmt":"2020-11-30T18:34:11","slug":"amazon-managed-workflows-for-apache-airflowmwaa-%ec%a0%95%ec%8b%9d-%ec%b6%9c%ec%8b%9c","status":"publish","type":"post","link":"https:\/\/jirak.net\/wp\/amazon-managed-workflows-for-apache-airflowmwaa-%ec%a0%95%ec%8b%9d-%ec%b6%9c%ec%8b%9c\/","title":{"rendered":"Amazon Managed Workflows for Apache Airflow(MWAA) \uc815\uc2dd \ucd9c\uc2dc"},"content":{"rendered":"<p>Amazon Managed Workflows for Apache Airflow(MWAA) \uc815\uc2dd \ucd9c\uc2dc<br \/>\n<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jirak.net\/wp\/wp-content\/uploads\/2020\/12\/mwaa-create-environment-1-1024x342-1.png\" width=\"1024\" height=\"342\"><\/p>\n<p>\ub370\uc774\ud130 \ucc98\ub9ac \ud30c\uc774\ud504\ub77c\uc778\uc758 \uc591\uacfc \ubcf5\uc7a1\uc131\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \uc804\uccb4 \ud504\ub85c\uc138\uc2a4\ub97c \uc77c\ub828\uc758 \uc791\uc740 \ud0dc\uc2a4\ud06c\ub85c \ubd84\ud574\ud558\uc5ec \uac04\uc18c\ud654\ud558\uace0 <strong>\uc6cc\ud06c\ud50c\ub85c<\/strong>\uc758 \uc77c\ubd80\ub85c \uc774\ub7ec\ud55c \ud0dc\uc2a4\ud06c\uc758 \uc2e4\ud589\uc744 \uc870\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub97c \uc704\ud574 \ub9ce\uc740 \uac1c\ubc1c\uc790\uc640 \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \ucee4\ubba4\ub2c8\ud2f0\uc5d0\uc11c \ub9cc\ub4e0 \ud50c\ub7ab\ud3fc\uc778 <a href=\"https:\/\/airflow.apache.org\/\">Apache Airflow<\/a>\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc6cc\ud06c\ud50c\ub85c\ub97c \ud504\ub85c\uadf8\ub798\ubc0d \ubc29\uc2dd\uc73c\ub85c \uc791\uc131, \uc608\uc57d \ubc0f \ubaa8\ub2c8\ud130\ub9c1\ud569\ub2c8\ub2e4. Airflow\ub97c \uc0ac\uc6a9\ud558\uba74 \uc6cc\ud06c\ud50c\ub85c\ub97c \uc2a4\ud06c\ub9bd\ud2b8\ub85c \uad00\ub9ac\ud558\uace0, \uc0ac\uc6a9\uc790 \uc778\ud130\ud398\uc774\uc2a4(UI)\ub97c \ud1b5\ud574 \ubaa8\ub2c8\ud130\ub9c1\ud558\uba70, \uac15\ub825\ud55c \ud50c\ub7ec\uadf8\uc778 \uc138\ud2b8\ub97c \uc0ac\uc6a9\ud574 \uae30\ub2a5\uc744 \ud655\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ub098 Airflow\ub97c \uc218\ub3d9\uc73c\ub85c \uc124\uce58, \uc720\uc9c0 \uad00\ub9ac \ubc0f \ud655\uc7a5\ud558\ub294 \ub3d9\uc2dc\uc5d0 \uc0ac\uc6a9\uc790\uc758 \ubcf4\uc548, \uc778\uc99d \ubc0f \uad8c\ud55c \ubd80\uc5ec\ub97c \ucc98\ub9ac\ud558\ub824\uba74 \uc2e4\uc81c \ube44\uc988\ub2c8\uc2a4 \ubb38\uc81c \ud574\uacb0\uc5d0 \uc3df\ub294 \uc2dc\uac04\ubcf4\ub2e4 \ub354 \ub9ce\uc740 \uc2dc\uac04\uc774 \ud560\uc560\ub429\ub2c8\ub2e4.<\/p>\n<p>\uc774\ub7ec\ud55c \uc774\uc720\ub85c AWS\uc5d0\uc11c \uc624\ud508 \uc18c\uc2a4 \ubc84\uc804\uc758 Apache Airflow\ub97c \uc27d\uac8c \uc2e4\ud589\ud558\uace0 <a href=\"https:\/\/en.wikipedia.org\/wiki\/Extract,_transform,_load\">\ucd94\ucd9c-\ubcc0\ud658-\ub85c\ub4dc(ETL)<\/a> \uc791\uc5c5 \ubc0f \ub370\uc774\ud130 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc2e4\ud589\ud560 \uc6cc\ud06c\ud50c\ub85c\ub97c \uad6c\ucd95\ud560 \uc218 \uc788\ub294 \uc644\uc804\uad00\ub9ac\ud615 \uc11c\ube44\uc2a4, <a title=\"\" href=\"https:\/\/aws.amazon.com\/managed-workflows-for-apache-airflow\/\">Amazon Managed Workflows for Apache Airflow(MWAA)<\/a>\uc758 \ucd9c\uc2dc \uc18c\uc2dd\uc744 \uc804\ud558\uac8c \ub418\uc5b4 \uc815\ub9d0 \uae30\uc069\ub2c8\ub2e4.<\/p>\n<p>Airflow \uc6cc\ud06c\ud50c\ub85c\ub294 <a title=\"\" href=\"https:\/\/aws.amazon.com\/athena\">Amazon Athena<\/a> \ucffc\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec <a title=\"\" href=\"https:\/\/aws.amazon.com\/s3\/\">Amazon Simple Storage Service(S3)<\/a>\uc640 \uac19\uc740 \uc18c\uc2a4\uc5d0\uc11c \uc785\ub825\uc744 \uac80\uc0c9\ud558\uace0, <a title=\"\" href=\"https:\/\/aws.amazon.com\/emr\">Amazon EMR<\/a> \ud074\ub7ec\uc2a4\ud130\uc5d0\uc11c \ubcc0\ud658\uc744 \uc218\ud589\ud558\uba70, \uacb0\uacfc \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uc5ec <a title=\"\" href=\"https:\/\/aws.amazon.com\/sagemaker\/\">Amazon SageMaker<\/a>\uc5d0\uc11c \uae30\uacc4 \ud559\uc2b5 \ubaa8\ub378\uc744 \ud6c8\ub828\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. Airflow\uc5d0\uc11c \uc6cc\ud06c\ud50c\ub85c\ub294 Python \ud504\ub85c\uadf8\ub798\ubc0d \uc5b8\uc5b4\ub97c \uc0ac\uc6a9\ud558\uc5ec <a href=\"https:\/\/airflow.apache.org\/docs\/stable\/concepts.html#dags\">Directed Acyclic Graph(DAG)<\/a>\ub85c \uc791\uc131\ub429\ub2c8\ub2e4.<\/p>\n<p>Airflow\uc758 \uc8fc\uc694 \uc774\uc810\uc740 <strong>\ud50c\ub7ec\uadf8\uc778<\/strong>\uc744 \ud1b5\ud55c \uac1c\ubc29\ub41c \ud655\uc7a5\uc131\uc5d0 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub97c \ud1b5\ud574 <a title=\"\" href=\"http:\/\/aws.amazon.com\/batch\">AWS Batch<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/cloudwatch\/\">Amazon CloudWatch<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/dynamodb\/\">Amazon DynamoDB<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/datasync\/\">AWS DataSync<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/ecs\/\">Amazon ECS<\/a> \ubc0f <a title=\"AWS Fargate\" href=\"https:\/\/aws.amazon.com\/fargate\/\">AWS Fargate<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/eks\/\">Amazon Elastic Kubernetes Service(EKS)<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/kinesis\/firehose\/\">Amazon Kinesis Firehose<\/a>, <a title=\"AWS Glue\" href=\"https:\/\/aws.amazon.com\/glue\/\">AWS Glue<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/lambda\/\">AWS Lambda<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/redshift\/\">Amazon Redshift<\/a>, <a title=\"\" href=\"https:\/\/aws.amazon.com\/sqs\/\">Amazon Simple Queue Service(SQS)<\/a> \ubc0f <a title=\"\" href=\"https:\/\/aws.amazon.com\/sns\/\">Amazon Simple Notification Service(SNS)<\/a>\ub97c \ud3ec\ud568\ud558\uc5ec \uc6cc\ud06c\ud50c\ub85c\uc5d0 \ud544\uc694\ud55c \uc628\ud504\ub808\ubbf8\uc2a4 \ub9ac\uc18c\uc2a4 \ub610\ub294 AWS\uc640 \uc0c1\ud638 \uc791\uc6a9\ud558\ub294 \ud0dc\uc2a4\ud06c\ub97c \uc0dd\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p>\uad00\uce21 \uac00\ub2a5\uc131\uc744 \uac1c\uc120\ud558\uae30 \uc704\ud574 Airflow <strong>\uc9c0\ud45c<\/strong>\ub294 CloudWatch \uc9c0\ud45c\ub85c \uac8c\uc2dc\ud560 \uc218 \uc788\uc73c\uba70, <strong>\ub85c\uadf8<\/strong>\ub97c CloudWatch Logs\uc5d0 \ubcf4\ub0bc \uc218 \uc788\uc2b5\ub2c8\ub2e4. <span title=\"\">Amazon MWAA<\/span>\ub294 \uc5c5\uadf8\ub808\uc774\ub4dc\uac00 \uc218\ud589\ub418\ub294 \uc720\uc9c0 \uad00\ub9ac \uae30\uac04\uc744 \uc9c0\uc815\ud558\ub294 \uc635\uc158\uc744 \ud1b5\ud574 \uae30\ubcf8\uc801\uc73c\ub85c \uc790\ub3d9 \ub9c8\uc774\ub108 \ubc84\uc804 <strong>\uc5c5\uadf8\ub808\uc774\ub4dc<\/strong> \ubc0f \ud328\uce58\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n<p>\ub2e4\uc74c\uacfc \uac19\uc740 3\ub2e8\uacc4\ub85c <span title=\"\">Amazon MWAA<\/span>\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<ol>\n<li style=\"padding-bottom: 0.5em\"><strong>\ud658\uacbd \uc0dd\uc131<\/strong> \u2013 \uac01 \ud658\uacbd\uc5d0\ub294 \uc2a4\ucf00\uc904\ub7ec, \uc791\uc5c5\uc790 \ubc0f \uc6f9 \uc11c\ubc84\ub97c \ud3ec\ud568\ud55c Airflow \ud074\ub7ec\uc2a4\ud130\uac00 \ud3ec\ud568\ub429\ub2c8\ub2e4.<\/li>\n<li style=\"padding-bottom: 0.5em\"><strong>DAG\uc640 \ud50c\ub7ec\uadf8\uc778\uc744 S3\uc5d0 \uc5c5\ub85c\ub4dc<\/strong> \u2013 Amazon MWAA\ub294 \ucf54\ub4dc\ub97c Airflow\uc5d0 \uc790\ub3d9\uc73c\ub85c \ub85c\ub4dc\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>Airflow\uc5d0\uc11c DAG \uc2e4\ud589<\/strong> \u2013 Airflow UI \ub610\ub294 \uba85\ub839\uc904 \uc778\ud130\ud398\uc774\uc2a4(CLI)\uc5d0\uc11c DAG\ub97c \uc2e4\ud589\ud558\uace0 CloudWatch\uc5d0\uc11c \ud658\uacbd\uc744 \ubaa8\ub2c8\ud130\ub9c1\ud569\ub2c8\ub2e4.<\/li>\n<\/ol>\n<p>\uc2e4\uc81c\ub85c \uc5b4\ub5bb\uac8c \uc791\ub3d9\ud558\ub294\uc9c0 \uc54c\uc544\ubcf4\uaca0\uc2b5\ub2c8\ub2e4!<\/p>\n<p><span style=\"text-decoration: underline\"><strong><span title=\"\">Amazon MWAA<\/span>\ub97c \uc0ac\uc6a9\ud558\uc5ec Airflow \ud658\uacbd\uc744 \uc0dd\uc131\ud558\ub294 \ubc29\ubc95<br \/> <\/strong><\/span><a href=\"https:\/\/console.aws.amazon.com\/mwaa\/home\"><span title=\"\">Amazon MWAA<\/span> \ucf58\uc194<\/a>\uc5d0\uc11c [<strong>\ud658\uacbd \uc0dd\uc131(Create environment)<\/strong>]\uc744 \ud074\ub9ad\ud569\ub2c8\ub2e4. \ud658\uacbd\uc5d0 \uc774\ub984\uc744 \uc9c0\uc815\ud558\uace0 \uc0ac\uc6a9\ud560 <strong>Airflow \ubc84\uc804<\/strong>\uc744 \uc120\ud0dd\ud569\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-create-environment-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-43885\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-create-environment-1-1024x342.png\" alt=\"\" width=\"1024\" height=\"342\" \/><\/a><\/p>\n<p>\uadf8\ub7f0 \ub2e4\uc74c, S3 \ubc84\ud0b7\uacfc \ud3f4\ub354\ub97c \uc120\ud0dd\ud558\uc5ec <strong>DAG \ucf54\ub4dc<\/strong>\ub97c \ub85c\ub4dc\ud569\ub2c8\ub2e4. \ubc84\ud0b7 \uc774\ub984\uc740 <code>airflow-<\/code>\ub85c \uc2dc\uc791\ud574\uc57c \ud569\ub2c8\ub2e4.<\/p>\n<p>\uc120\ud0dd\uc801\uc73c\ub85c \ud50c\ub7ec\uadf8\uc778 \ud30c\uc77c\uacfc \uc694\uad6c \uc0ac\ud56d \ud30c\uc77c\uc744 \uc9c0\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<ul>\n<li style=\"padding-bottom: 0.5em\"><strong>\ud50c\ub7ec\uadf8\uc778<\/strong> \ud30c\uc77c\uc740 DAG\uc5d0\uc11c \uc0ac\uc6a9\ud558\ub294 \ud50c\ub7ec\uadf8\uc778\uc744 \ud3ec\ud568\ud558\ub294 ZIP \ud30c\uc77c\uc785\ub2c8\ub2e4.<\/li>\n<li style=\"padding-bottom: 0.5em\"><strong>\uc694\uad6c \uc0ac\ud56d<\/strong> \ud30c\uc77c\uc740 DAG\ub97c \uc2e4\ud589\ud558\uae30 \uc704\ud55c Python \uc885\uc18d\uc131\uc744 \uc124\uba85\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<p>\ud50c\ub7ec\uadf8\uc778 \ubc0f \uc694\uad6c \uc0ac\ud56d\uc758 \uacbd\uc6b0 \uc0ac\uc6a9\ud560 <a href=\"https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/dev\/Versioning.html\">S3 \uac1d\uccb4 \ubc84\uc804<\/a>\uc744 \uc120\ud0dd\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc0ac\uc6a9\ud558\ub294 \ud50c\ub7ec\uadf8\uc778\uc774\ub098 \uc694\uad6c \uc0ac\ud56d\uc73c\ub85c \uc778\ud574 \ud658\uacbd\uc5d0\uc11c \ubcf5\uad6c \ubd88\uac00\ub2a5\ud55c \uc624\ub958\uac00 \ubc1c\uc0dd\ud558\ub294 \uacbd\uc6b0 <span title=\"\">Amazon MWAA<\/span>\ub294 \uc774\uc804 \uc791\uc5c5 \ubc84\uc804\uc73c\ub85c \uc790\ub3d9 \ub864\ubc31\ub429\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-dag-code-s3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-42934\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-dag-code-s3-1012x1024.png\" alt=\"\" width=\"1012\" height=\"1024\" \/><\/a><br \/> [<strong>\ub2e4\uc74c(Next)<\/strong>]\uc744 \ud074\ub9ad\ud558\uc5ec \uace0\uae09 \uc124\uc815\uc744 \uad6c\uc131\ud569\ub2c8\ub2e4. [<strong>\ub124\ud2b8\uc6cc\ud0b9(Networking)<\/strong>]\ubd80\ud130 \uc2dc\uc791\ud569\ub2c8\ub2e4. \uac01 \ud658\uacbd\uc740 2\uac1c\uc758 <a href=\"https:\/\/aws.amazon.com\/about-aws\/global-infrastructure\/regions_az\/#Availability_Zones\">\uac00\uc6a9 \uc601\uc5ed<\/a>\uc5d0\uc11c \ud504\ub77c\uc774\ube57 \uc11c\ube0c\ub137\uc744 \uc0ac\uc6a9\ud558\uc5ec <a title=\"\" href=\"https:\/\/aws.amazon.com\/vpc\/\">Amazon Virtual Private Cloud<\/a>\uc5d0\uc11c \uc2e4\ud589\ub429\ub2c8\ub2e4. Airflow UI\uc5d0 \ub300\ud55c \uc6f9 \uc11c\ubc84 \uc561\uc138\uc2a4\ub294 \ud56d\uc0c1 <a title=\"\" href=\"https:\/\/aws.amazon.com\/iam\/\">AWS Identity and Access Management(IAM)<\/a>\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubcf4\uc548 \ub85c\uadf8\uc778\uc73c\ub85c \ubcf4\ud638\ub429\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \ud37c\ube14\ub9ad \ub124\ud2b8\uc6cc\ud06c\uc5d0\uc11c \uc6f9 \uc11c\ubc84 \uc561\uc138\uc2a4\ub97c \ud5c8\uc6a9\ud558\ub3c4\ub85d \uc120\ud0dd\ud558\uc5ec \uc778\ud130\ub137\uc774\ub098 VPC\uc758 \ud504\ub77c\uc774\ube57 \ub124\ud2b8\uc6cc\ud06c\uc5d0\uc11c \ub85c\uadf8\uc778\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uac04\ub2e8\ud558\uac8c [<strong>\ud37c\ube14\ub9ad \ub124\ud2b8\uc6cc\ud06c(Public network)<\/strong>]\ub97c \uc120\ud0dd\ud569\ub2c8\ub2e4. <span title=\"\">Amazon MWAA<\/span>\uc5d0\uc11c \uc62c\ubc14\ub978 \uc778\ubc14\uc6b4\ub4dc \ubc0f \uc544\uc6c3\ubc14\uc6b4\ub4dc \uaddc\uce59\uc744 \uc0ac\uc6a9\ud558\ub294 \uc0c8 <a href=\"https:\/\/docs.aws.amazon.com\/vpc\/latest\/userguide\/VPC_SecurityGroups.html\">\ubcf4\uc548 \uadf8\ub8f9<\/a>\uc744 \uc0dd\uc131\ud569\ub2c8\ub2e4. \uc120\ud0dd\uc801\uc73c\ub85c \ud558\ub098 \uc774\uc0c1\uc758 \uae30\uc874 \ubcf4\uc548 \uadf8\ub8f9\uc744 \ucd94\uac00\ud558\uc5ec \ud658\uacbd\uc5d0 \ub300\ud55c \uc778\ubc14\uc6b4\ub4dc \ubc0f \uc544\uc6c3\ubc14\uc6b4\ub4dc \ud2b8\ub798\ud53d\uc758 \uc81c\uc5b4\ub97c \ubbf8\uc138 \uc870\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-networking-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-43886\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-networking-1-881x1024.png\" alt=\"\" width=\"881\" height=\"1024\" \/><\/a><\/p>\n<p>\uc774\uc81c [<strong>\ud658\uacbd \ud074\ub798\uc2a4(Environment class)<\/strong>]\ub97c \uad6c\uc131\ud569\ub2c8\ub2e4. \uac01 \ud658\uacbd\uc5d0\ub294 \uc2a4\ucf00\uc904\ub7ec, \uc6f9 \uc11c\ubc84 \ubc0f \uc791\uc5c5\uc790\uac00 \ud3ec\ud568\ub429\ub2c8\ub2e4. \uc791\uc5c5\uc790\ub294 \uc6cc\ud06c\ub85c\ub4dc\uc5d0 \ub530\ub77c \uc790\ub3d9\uc73c\ub85c \ud655\uc7a5 \ubc0f \ucd95\uc18c\ub429\ub2c8\ub2e4. DAG\uc758 \uc218\uc5d0 \ub530\ub77c \uc0ac\uc6a9\ud560 \ud074\ub798\uc2a4\uc5d0 \ub300\ud55c \uc81c\uc548 \uc0ac\ud56d\uc744 \uc81c\uacf5\ud558\uc9c0\ub9cc \uc5b8\uc81c\ub4e0\uc9c0 \uc0ac\uc6a9\uc790 \ud658\uacbd\uc758 \ub85c\ub4dc\ub97c \ubaa8\ub2c8\ud130\ub9c1\ud558\uace0 \ud074\ub798\uc2a4\ub97c \uc218\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-environment-class.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-42939\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-environment-class-1024x555.png\" alt=\"\" width=\"1024\" height=\"555\" \/><\/a><\/p>\n<p>[<strong>\uc554\ud638\ud654(Encryption)<\/strong>]\ub294 \ud56d\uc0c1 \uc720\ud734 \ub370\uc774\ud130\uc5d0 \ub300\ud574 \ud65c\uc131\ud654\ub429\ub2c8\ub2e4. <a title=\"\" href=\"https:\/\/aws.amazon.com\/kms\/\">AWS Key Management Service(KMS)<\/a>\uc5d0\uc11c \uad00\ub9ac\ud558\ub294 \uc0ac\uc6a9\uc790 \uc9c0\uc815\ub41c \ud0a4\ub97c \uc120\ud0dd\ud560 \uc218 \uc788\uc9c0\ub9cc, AWS\uc5d0\uc11c \uc18c\uc720\ud558\uace0 \uc0ac\uc6a9\uc790 \ub300\uc2e0 \uad00\ub9ac\ud558\ub294 \uae30\ubcf8 \ud0a4\ub97c \ub300\uc2e0 \uc720\uc9c0\ud558\ub824\uace0 \ud569\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-encryption-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-43887\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-encryption-1-1024x286.png\" alt=\"\" width=\"1024\" height=\"286\" \/><\/a><\/p>\n<p>[<strong>\ubaa8\ub2c8\ud130\ub9c1(Monitoring)<\/strong>]\uc5d0\uc11c\ub294 CloudWatch \uc9c0\ud45c\ub85c \ud658\uacbd \uc131\ub2a5\uc744 \uac8c\uc2dc\ud569\ub2c8\ub2e4. \uc774 \uae30\ub2a5\uc740 \uae30\ubcf8\uc801\uc73c\ub85c \ud65c\uc131\ud654\ub418\uc5b4 \uc788\uc9c0\ub9cc \uc2e4\ud589 \ud6c4 CloudWatch \uc9c0\ud45c\ub97c \ube44\ud65c\uc131\ud654\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. <strong>\ub85c\uadf8<\/strong>\uc5d0\uc11c\ub294 \ub85c\uadf8 \uc218\uc900\uacfc CloudWatch Logs\uc5d0 \ub85c\uadf8\ub97c \uc804\uc1a1\ud574\uc57c \ud558\ub294 Airflow \uad6c\uc131 \uc694\uc18c\ub97c \uc9c0\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud0dc\uc2a4\ud06c \ub85c\uadf8\ub9cc \ubcf4\ub0b4\ub294 \uae30\ubcf8\uac12\uc744 \uc720\uc9c0\ud558\uace0 \ub85c\uadf8 \uc218\uc900 <code>INFO<\/code>\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-monitoring-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-43888\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/24\/mwaa-monitoring-1-1024x715.png\" alt=\"\" width=\"1024\" height=\"715\" \/><\/a><\/p>\n<p><code>default_task_retries<\/code> \ub610\ub294 <code>worker_concurrency<\/code>\uc640 \uac19\uc740 Airflow [<strong>\uad6c\uc131 \uc635\uc158(configuration options)<\/strong>]\uc758 \uae30\ubcf8 \uc124\uc815\uc744 \uc218\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc9c0\uae08\uc740 \uc774\ub7ec\ud55c \uac12\uc744 \ubcc0\uacbd\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-airflow-configuration.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-42944\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-airflow-configuration-1024x603.png\" alt=\"\" width=\"1024\" height=\"603\" \/><\/a><\/p>\n<p>\ub9c8\uc9c0\ub9c9\uc73c\ub85c, \uac00\uc7a5 \uc911\uc694\ud55c \uc791\uc5c5\uc740 \ud658\uacbd\uc5d0\uc11c DAG\uc5d0 \uc561\uc138\uc2a4\ud558\uace0, \ub85c\uadf8\ub97c \uc791\uc131\ud558\uba70, \ub2e4\ub978 AWS \ub9ac\uc18c\uc2a4\uc5d0 \uc561\uc138\uc2a4\ud558\ub294 DAG\ub97c \uc2e4\ud589\ud558\ub294 \ub370 \uc0ac\uc6a9\ud560 <strong>\uad8c\ud55c<\/strong>\uc744 \uad6c\uc131\ud558\ub294 \uac83\uc785\ub2c8\ub2e4. [<strong>\uc0c8 \uc5ed\ud560 \uc0dd\uc131(Create a new role)<\/strong>]\uc744 \uc120\ud0dd\ud558\uace0 [<strong>\ud658\uacbd \uc0dd\uc131(Create environment)<\/strong>]\uc744 \ud074\ub9ad\ud569\ub2c8\ub2e4. \uba87 \ubd84 \ud6c4 \uc0c8\ub85c\uc6b4 Airflow \ud658\uacbd\uc744 \uc0ac\uc6a9\ud560 \uc900\ube44\uac00 \ub429\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-permissions.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-42945\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-permissions-1024x361.png\" alt=\"\" width=\"1024\" height=\"361\" \/><\/a><\/p>\n<p><span style=\"text-decoration: underline\"><strong>Airflow UI \uc0ac\uc6a9<br \/> <\/strong><\/span><span title=\"\">Amazon MWAA<\/span> \ucf58\uc194\uc5d0\uc11c \ubc29\uae08 \uc0dd\uc131\ud55c \uc0c8 \ud658\uacbd\uc744 \ucc3e\uc544 [<strong>Airflow UI \uc5f4\uae30(Open Airflow UI)<\/strong>]\ub97c \ud074\ub9ad\ud569\ub2c8\ub2e4. \uc0c8 \ube0c\ub77c\uc6b0\uc800 \ucc3d\uc774 \uc0dd\uc131\ub418\uace0 AWS IAM\uc744 \ud1b5\ud55c \ubcf4\uc548 \ub85c\uadf8\uc778\uc73c\ub85c \uc778\uc99d\ub429\ub2c8\ub2e4.<\/p>\n<p>\uc5ec\uae30\uc5d0\uc11c <code>movie_list_dag.py<\/code> \ud30c\uc77c\uc5d0\uc11c S3\uc5d0 \ub123\uc740 DAG\ub97c \ucc3e\uc2b5\ub2c8\ub2e4. DAG\ub294 <a href=\"https:\/\/grouplens.org\/datasets\/movielens\/\">MovieLens \ub370\uc774\ud130 \uc138\ud2b8<\/a>\ub97c \ub2e4\uc6b4\ub85c\ub4dc\ud558\uace0 <a title=\"\" href=\"https:\/\/aws.amazon.com\/athena\">Amazon Athena<\/a>\ub97c \uc0ac\uc6a9\ud558\uc5ec S3\uc5d0\uc11c \ud30c\uc77c\uc744 \ucc98\ub9ac\ud558\uba70 \uacb0\uacfc\ub97c Redshift \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub85c\ub4dc\ud558\uace0 \ub204\ub77d\ub41c \uacbd\uc6b0 \ud14c\uc774\ube14\uc744 \uc0dd\uc131\ud569\ub2c8\ub2e4.<\/p>\n<p>\ub2e4\uc74c\uc740 DAG\uc758 \uc804\uccb4 \uc18c\uc2a4 \ucf54\ub4dc\uc785\ub2c8\ub2e4.<\/p>\n<pre><code class=\"lang-python\">from airflow import DAG\nfrom airflow.operators.python_operator import PythonOperator\nfrom airflow.operators import HttpSensor, S3KeySensor\nfrom airflow.contrib.operators.aws_athena_operator import AWSAthenaOperator\nfrom airflow.utils.dates import days_ago\nfrom datetime import datetime, timedelta\nfrom io import StringIO\nfrom io import BytesIO\nfrom time import sleep\nimport csv\nimport requests\nimport json\nimport boto3\nimport zipfile\nimport io\ns3_bucket_name = 'my-bucket'\ns3_key='files\/'\nredshift_cluster='my-redshift-cluster'\nredshift_db='dev'\nredshift_dbuser='awsuser'\nredshift_table_name='movie_demo'\ntest_http='https:\/\/grouplens.org\/datasets\/movielens\/latest\/'\ndownload_http='http:\/\/files.grouplens.org\/datasets\/movielens\/ml-latest-small.zip'\nathena_db='demo_athena_db'\nathena_results='athena-results\/'\ncreate_athena_movie_table_query=&quot;&quot;&quot;\nCREATE EXTERNAL TABLE IF NOT EXISTS Demo_Athena_DB.ML_Latest_Small_Movies (\n  `movieId` int,\n  `title` string,\n  `genres` string \n)\nROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'\nWITH SERDEPROPERTIES (\n  'serialization.format' = ',',\n  'field.delim' = ','\n) LOCATION 's3:\/\/my-bucket\/files\/ml-latest-small\/movies.csv\/ml-latest-small\/'\nTBLPROPERTIES (\n  'has_encrypted_data'='false',\n  'skip.header.line.count'='1'\n); \n&quot;&quot;&quot;\ncreate_athena_ratings_table_query=&quot;&quot;&quot;\nCREATE EXTERNAL TABLE IF NOT EXISTS Demo_Athena_DB.ML_Latest_Small_Ratings (\n  `userId` int,\n  `movieId` int,\n  `rating` int,\n  `timestamp` bigint \n)\nROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'\nWITH SERDEPROPERTIES (\n  'serialization.format' = ',',\n  'field.delim' = ','\n) LOCATION 's3:\/\/my-bucket\/files\/ml-latest-small\/ratings.csv\/ml-latest-small\/'\nTBLPROPERTIES (\n  'has_encrypted_data'='false',\n  'skip.header.line.count'='1'\n); \n&quot;&quot;&quot;\ncreate_athena_tags_table_query=&quot;&quot;&quot;\nCREATE EXTERNAL TABLE IF NOT EXISTS Demo_Athena_DB.ML_Latest_Small_Tags (\n  `userId` int,\n  `movieId` int,\n  `tag` int,\n  `timestamp` bigint \n)\nROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'\nWITH SERDEPROPERTIES (\n  'serialization.format' = ',',\n  'field.delim' = ','\n) LOCATION 's3:\/\/my-bucket\/files\/ml-latest-small\/tags.csv\/ml-latest-small\/'\nTBLPROPERTIES (\n  'has_encrypted_data'='false',\n  'skip.header.line.count'='1'\n); \n&quot;&quot;&quot;\njoin_tables_athena_query=&quot;&quot;&quot;\nSELECT REPLACE ( m.title , '&quot;' , '' ) as title, r.rating\nFROM demo_athena_db.ML_Latest_Small_Movies m\nINNER JOIN (SELECT rating, movieId FROM demo_athena_db.ML_Latest_Small_Ratings WHERE rating &gt; 4) r on m.movieId = r.movieId\n&quot;&quot;&quot;\ndef download_zip():\n    s3c = boto3.client('s3')\n    indata = requests.get(download_http)\n    n=0\n    with zipfile.ZipFile(io.BytesIO(indata.content)) as z:       \n        zList=z.namelist()\n        print(zList)\n        for i in zList: \n            print(i) \n            zfiledata = BytesIO(z.read(i))\n            n += 1\n            s3c.put_object(Bucket=s3_bucket_name, Key=s3_key+i+'\/'+i, Body=zfiledata)\ndef clean_up_csv_fn(**kwargs):    \n    ti = kwargs['task_instance']\n    queryId = ti.xcom_pull(key='return_value', task_ids='join_athena_tables' )\n    print(queryId)\n    athenaKey=athena_results+&quot;join_athena_tables\/&quot;+queryId+&quot;.csv&quot;\n    print(athenaKey)\n    cleanKey=athena_results+&quot;join_athena_tables\/&quot;+queryId+&quot;_clean.csv&quot;\n    s3c = boto3.client('s3')\n    obj = s3c.get_object(Bucket=s3_bucket_name, Key=athenaKey)\n    infileStr=obj['Body'].read().decode('utf-8')\n    outfileStr=infileStr.replace('&quot;e&quot;', '') \n    outfile = StringIO(outfileStr)\n    s3c.put_object(Bucket=s3_bucket_name, Key=cleanKey, Body=outfile.getvalue())\ndef s3_to_redshift(**kwargs):    \n    ti = kwargs['task_instance']\n    queryId = ti.xcom_pull(key='return_value', task_ids='join_athena_tables' )\n    print(queryId)\n    athenaKey='s3:\/\/'+s3_bucket_name+&quot;\/&quot;+athena_results+&quot;join_athena_tables\/&quot;+queryId+&quot;_clean.csv&quot;\n    print(athenaKey)\n    sqlQuery=&quot;copy &quot;+redshift_table_name+&quot; from '&quot;+athenaKey+&quot;' iam_role 'arn:aws:iam::163919838948:role\/myRedshiftRole' CSV IGNOREHEADER 1;&quot;\n    print(sqlQuery)\n    rsd = boto3.client('redshift-data')\n    resp = rsd.execute_statement(\n        ClusterIdentifier=redshift_cluster,\n        Database=redshift_db,\n        DbUser=redshift_dbuser,\n        Sql=sqlQuery\n    )\n    print(resp)\n    return &quot;OK&quot;\ndef create_redshift_table():\n    rsd = boto3.client('redshift-data')\n    resp = rsd.execute_statement(\n        ClusterIdentifier=redshift_cluster,\n        Database=redshift_db,\n        DbUser=redshift_dbuser,\n        Sql=&quot;CREATE TABLE IF NOT EXISTS &quot;+redshift_table_name+&quot; (title\tcharacter varying, rating\tint);&quot;\n    )\n    print(resp)\n    return &quot;OK&quot;\nDEFAULT_ARGS = {\n    'owner': 'airflow',\n    'depends_on_past': False,\n    'email': ['airflow@example.com'],\n    'email_on_failure': False,\n    'email_on_retry': False \n}\nwith DAG(\n    dag_id='movie-list-dag',\n    default_args=DEFAULT_ARGS,\n    dagrun_timeout=timedelta(hours=2),\n    start_date=days_ago(2),\n    schedule_interval='*\/10 * * * *',\n    tags=['athena','redshift'],\n) as dag:\n    check_s3_for_key = S3KeySensor(\n        task_id='check_s3_for_key',\n        bucket_key=s3_key,\n        wildcard_match=True,\n        bucket_name=s3_bucket_name,\n        s3_conn_id='aws_default',\n        timeout=20,\n        poke_interval=5,\n        dag=dag\n    )\n    files_to_s3 = PythonOperator(\n        task_id=&quot;files_to_s3&quot;,\n        python_callable=download_zip\n    )\n    create_athena_movie_table = AWSAthenaOperator(task_id=&quot;create_athena_movie_table&quot;,query=create_athena_movie_table_query, database=athena_db, output_location='s3:\/\/'+s3_bucket_name+&quot;\/&quot;+athena_results+'create_athena_movie_table')\n    create_athena_ratings_table = AWSAthenaOperator(task_id=&quot;create_athena_ratings_table&quot;,query=create_athena_ratings_table_query, database=athena_db, output_location='s3:\/\/'+s3_bucket_name+&quot;\/&quot;+athena_results+'create_athena_ratings_table')\n    create_athena_tags_table = AWSAthenaOperator(task_id=&quot;create_athena_tags_table&quot;,query=create_athena_tags_table_query, database=athena_db, output_location='s3:\/\/'+s3_bucket_name+&quot;\/&quot;+athena_results+'create_athena_tags_table')\n    join_athena_tables = AWSAthenaOperator(task_id=&quot;join_athena_tables&quot;,query=join_tables_athena_query, database=athena_db, output_location='s3:\/\/'+s3_bucket_name+&quot;\/&quot;+athena_results+'join_athena_tables')\n    create_redshift_table_if_not_exists = PythonOperator(\n        task_id=&quot;create_redshift_table_if_not_exists&quot;,\n        python_callable=create_redshift_table\n    )\n    clean_up_csv = PythonOperator(\n        task_id=&quot;clean_up_csv&quot;,\n        python_callable=clean_up_csv_fn,\n        provide_context=True     \n    )\n    transfer_to_redshift = PythonOperator(\n        task_id=&quot;transfer_to_redshift&quot;,\n        python_callable=s3_to_redshift,\n        provide_context=True     \n    )\n    check_s3_for_key &gt;&gt; files_to_s3 &gt;&gt; create_athena_movie_table &gt;&gt; join_athena_tables &gt;&gt; clean_up_csv &gt;&gt; transfer_to_redshift\n    files_to_s3 &gt;&gt; create_athena_ratings_table &gt;&gt; join_athena_tables\n    files_to_s3 &gt;&gt; create_athena_tags_table &gt;&gt; join_athena_tables\n    files_to_s3 &gt;&gt; create_redshift_table_if_not_exists &gt;&gt; transfer_to_redshift<\/code><\/pre>\n<p>\ucf54\ub4dc\uc5d0\uc11c\ub294 \uc77c\ubc18 Python \ucf54\ub4dc\ub97c \uc704\ud55c <code>PythonOperator<\/code> \ub610\ub294 <a title=\"\" href=\"https:\/\/aws.amazon.com\/athena\">Amazon Athena<\/a>\uc640\uc758 \ud1b5\ud569\uc744 \uc0ac\uc6a9\ud558\uae30 \uc704\ud55c <code>AWSAthenaOperator<\/code>\uc640 \uac19\uc740 \uc5f0\uc0b0\uc790\ub97c \uc0ac\uc6a9\ud558\uc5ec \ub2e4\ub978 \ud0dc\uc2a4\ud06c\uac00 \uc0dd\uc131\ub429\ub2c8\ub2e4. \uc774\ub7ec\ud55c \ud0dc\uc2a4\ud06c\uac00 \uc6cc\ud06c\ud50c\ub85c\uc5d0\uc11c \uc5f0\uacb0\ub41c \ubc29\uc2dd\uc744 \ubcf4\ub824\uba74 \ucd5c\uadfc \uba87 \uac1c \uc904\uc744 \ubcf4\uba74 \ub429\ub2c8\ub2e4. \uac04\ub2e8\ud558\uac8c \uc5ec\uae30\uc5d0 \ub4e4\uc5ec\uc4f0\uc9c0 \uc54a\uace0 \ubc18\ubcf5\ud569\ub2c8\ub2e4.<\/p>\n<pre><code class=\"lang-python\">check_s3_for_key &gt;&gt; files_to_s3 &gt;&gt; create_athena_movie_table &gt;&gt; join_athena_tables &gt;&gt; clean_up_csv &gt;&gt; transfer_to_redshift\nfiles_to_s3 &gt;&gt; create_athena_ratings_table &gt;&gt; join_athena_tables\nfiles_to_s3 &gt;&gt; create_athena_tags_table &gt;&gt; join_athena_tables\nfiles_to_s3 &gt;&gt; create_redshift_table_if_not_exists &gt;&gt; transfer_to_redshift<\/code><\/pre>\n<p>Airflow \ucf54\ub4dc\ub294 Python\uc5d0\uc11c \uc624\ub978\ucabd \uc2dc\ud504\ud2b8 <code>&gt;&gt;<\/code> \uc5f0\uc0b0\uc790\ub97c <a href=\"https:\/\/docs.python.org\/3\/reference\/datamodel.html#emulating-numeric-types\">\uc624\ubc84\ub85c\ub4dc<\/a>\ud558\uc5ec \uc885\uc18d\uc131\uc744 \uc0dd\uc131\ud569\ub2c8\ub2e4. \uc989, \uc67c\ucabd\uc758 \ud0dc\uc2a4\ud06c\uac00 \uba3c\uc800 \uc2e4\ud589\ub418\uace0 \ucd9c\ub825\uc774 \uc624\ub978\ucabd\uc758 \ud0dc\uc2a4\ud06c\ub85c \uc804\ub2ec\ub418\uc5b4\uc57c \ud569\ub2c8\ub2e4. \ucf54\ub4dc\ub97c \uc0b4\ud3b4\ubcf4\uba74 \uaf64 \uc27d\uac8c \ud30c\uc545\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc704\uc758 \uac01 4\uac1c \uc904\uc5d0\uc11c\ub294 \uc885\uc18d\uc131\uc744 \ucd94\uac00\ud558\uace0 \uc788\uc73c\uba70, \uc62c\ubc14\ub978 \uc21c\uc11c\ub85c \ud0dc\uc2a4\ud06c\ub97c \uc2e4\ud589\ud558\uae30 \uc704\ud574 \ubaa8\ub450 \ud568\uaed8 \ud3c9\uac00\ub429\ub2c8\ub2e4.<\/p>\n<p>Airflow \ucf58\uc194\uc5d0\uc11c DAG\uc758 <strong>\uadf8\ub798\ud504 \ubcf4\uae30<\/strong>\ub97c \ubcf4\uace0 \ud0dc\uc2a4\ud06c \uc2e4\ud589 \ubc29\uc2dd\uc744 \uba85\ud655\ud558\uac8c \ud45c\ud604\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-graph-view.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-43025\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/da4b9237bacccdf19c0760cab7aec4a8359010b0\/2020\/11\/17\/mwaa-graph-view-1024x466.png\" alt=\"\" width=\"1024\" height=\"466\" \/><\/a><\/p>\n<p><span style=\"text-decoration: underline\"><strong>\uc815\uc2dd \ucd9c\uc2dc<br \/> <\/strong><\/span><a title=\"\" href=\"https:\/\/aws.amazon.com\/managed-workflows-for-apache-airflow\/\">Amazon Managed Workflows for Apache Airflow(MWAA)<\/a>\ub294 \uc624\ub298 \ubbf8\uad6d \ub3d9\ubd80(\ubc84\uc9c0\ub2c8\uc544 \ubd81\ubd80), \ubbf8\uad6d \uc11c\ubd80(\uc624\ub808\uace4), \ubbf8\uad6d \ub3d9\ubd80(\uc624\ud558\uc774\uc624), \uc544\uc2dc\uc544 \ud0dc\ud3c9\uc591(\uc2f1\uac00\ud3ec\ub974), \uc544\uc2dc\uc544 \ud0dc\ud3c9\uc591(\ub3c4\ucfc4), \uc544\uc2dc\uc544 \ud0dc\ud3c9\uc591(\uc2dc\ub4dc\ub2c8), EU(\uc544\uc77c\ub79c\ub4dc), EU(\ud504\ub791\ud06c\ud478\ub974\ud2b8), EU(\uc2a4\ud1a1\ud640\ub984)\uc5d0\uc11c \uc815\uc2dd \ucd9c\uc2dc\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \ucf58\uc194, <a title=\"\" href=\"https:\/\/aws.amazon.com\/cli\/\">AWS \uba85\ub839\uc904 \uc778\ud130\ud398\uc774\uc2a4(CLI)<\/a> \ub610\ub294 <a title=\"\" href=\"https:\/\/aws.amazon.com\/tools\/\">AWS SDK<\/a>\uc5d0\uc11c \uc0c8 <span title=\"\">Amazon MWAA<\/span> \ud658\uacbd\uc744 \uc2dc\uc791\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub7f0 \ub2e4\uc74c, Airflow\uc758 \ud1b5\ud569 \uc5d0\ucf54\uc2dc\uc2a4\ud15c\uc744 \uc0ac\uc6a9\ud558\uc5ec Python\uc5d0\uc11c \uc6cc\ud06c\ud50c\ub85c\ub97c \uac1c\ubc1c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p><span title=\"\">Amazon MWAA<\/span>\uc5d0\uc11c\ub294 \uc0ac\uc6a9\uc790\uac00 \uc0ac\uc6a9\ud558\ub294 \ud658\uacbd \ud074\ub798\uc2a4 \ubc0f \uc791\uc5c5\uc790\uc5d0 \uae30\ubc18\ud558\uc5ec \ube44\uc6a9\uc744 \uc9c0\ubd88\ud569\ub2c8\ub2e4. \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 <a href=\"https:\/\/aws.amazon.com\/managed-workflows-for-apache-airflow\/pricing\/\">\uc694\uae08 \ud398\uc774\uc9c0<\/a>\ub97c \ucc38\uc870\ud558\uc138\uc694.<\/p>\n<p>\uc5c5\uc2a4\ud2b8\ub9bc \ud638\ud658\uc131\uc740 <span title=\"\">Amazon MWAA<\/span>\uc758 \uc8fc\uc694 \uc6d0\uce59\uc785\ub2c8\ub2e4. AirFlow \ud50c\ub7ab\ud3fc\uc5d0 \ub300\ud55c \ucf54\ub4dc\uac00 \ubcc0\uacbd\ub418\ub294 \uacbd\uc6b0 \uc624\ud508 \uc18c\uc2a4\ub85c \ub2e4\uc2dc \ucd9c\uc2dc\ub429\ub2c8\ub2e4.<\/p>\n<p><span title=\"\">Amazon MWAA<\/span>\ub97c \ud1b5\ud574 \uc5d4\uc9c0\ub2c8\uc5b4\ub9c1 \ubc0f \ub370\uc774\ud130 \uacfc\ud559 \ud0dc\uc2a4\ud06c\ub97c \uc704\ud55c \uc6cc\ud06c\ud50c\ub85c\ub97c \uad6c\ucd95\ud558\ub294 \ub370 \ub354 \ub9ce\uc740 \uc2dc\uac04\uc744 \ud560\uc560\ud560 \uc218 \uc788\uace0, Airflow \ud50c\ub7ab\ud3fc\uc758 \uc778\ud504\ub77c\ub97c \uad00\ub9ac\ud558\uace0 \ud655\uc7a5\ud558\ub294 \uc2dc\uac04\uc744 \uc904\uc77c \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p><a href=\"https:\/\/aws.amazon.com\/managed-workflows-for-apache-airflow\/\"><strong><span title=\"\">Amazon MWAA<\/span>\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc54c\uc544\ubcf4\uace0 \uc624\ub298 \uc2dc\uc791\ud574\ubcf4\uc138\uc694!<\/strong><\/a><\/p>\n<p>\u2014 <a title=\"Twitter\uc758 Danilo\" href=\"https:\/\/twitter.com\/danilop\">Danilo<\/a><\/p>\n<p>Source: <a href=\"https:\/\/aws.amazon.com\/ko\/blogs\/korea\/introducing-amazon-managed-workflows-for-apache-airflow-mwaa\/\" target=\"_blank\" rel=\"noopener noreferrer\">Amazon Managed Workflows for Apache Airflow(MWAA) \uc815\uc2dd \ucd9c\uc2dc<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"mh-excerpt\"><p>Amazon Managed Workflows for Apache Airflow(MWAA) \uc815\uc2dd \ucd9c\uc2dc \ub370\uc774\ud130 \ucc98\ub9ac \ud30c\uc774\ud504\ub77c\uc778\uc758 \uc591\uacfc \ubcf5\uc7a1\uc131\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \uc804\uccb4 \ud504\ub85c\uc138\uc2a4\ub97c \uc77c\ub828\uc758 \uc791\uc740 \ud0dc\uc2a4\ud06c\ub85c \ubd84\ud574\ud558\uc5ec \uac04\uc18c\ud654\ud558\uace0 \uc6cc\ud06c\ud50c\ub85c\uc758 \uc77c\ubd80\ub85c \uc774\ub7ec\ud55c \ud0dc\uc2a4\ud06c\uc758 \uc2e4\ud589\uc744 \uc870\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub97c \uc704\ud574 \ub9ce\uc740 \uac1c\ubc1c\uc790\uc640 \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \ucee4\ubba4\ub2c8\ud2f0\uc5d0\uc11c \ub9cc\ub4e0 \ud50c\ub7ab\ud3fc\uc778 Apache Airflow\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc6cc\ud06c\ud50c\ub85c\ub97c \ud504\ub85c\uadf8\ub798\ubc0d \ubc29\uc2dd\uc73c\ub85c \uc791\uc131, \uc608\uc57d \ubc0f \ubaa8\ub2c8\ud130\ub9c1\ud569\ub2c8\ub2e4. Airflow\ub97c \uc0ac\uc6a9\ud558\uba74 \uc6cc\ud06c\ud50c\ub85c\ub97c \uc2a4\ud06c\ub9bd\ud2b8\ub85c \uad00\ub9ac\ud558\uace0, \uc0ac\uc6a9\uc790 \uc778\ud130\ud398\uc774\uc2a4(UI)\ub97c \ud1b5\ud574 \ubaa8\ub2c8\ud130\ub9c1\ud558\uba70, \uac15\ub825\ud55c \ud50c\ub7ec\uadf8\uc778 \uc138\ud2b8\ub97c \uc0ac\uc6a9\ud574 \uae30\ub2a5\uc744 \ud655\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ub098 Airflow\ub97c \uc218\ub3d9\uc73c\ub85c \uc124\uce58, \uc720\uc9c0 \uad00\ub9ac \ubc0f \ud655\uc7a5\ud558\ub294 \ub3d9\uc2dc\uc5d0 \uc0ac\uc6a9\uc790\uc758 \ubcf4\uc548, \uc778\uc99d \ubc0f \uad8c\ud55c \ubd80\uc5ec\ub97c \ucc98\ub9ac\ud558\ub824\uba74 \uc2e4\uc81c \ube44\uc988\ub2c8\uc2a4 \ubb38\uc81c \ud574\uacb0\uc5d0 \uc3df\ub294 \uc2dc\uac04\ubcf4\ub2e4 \ub354 \ub9ce\uc740 \uc2dc\uac04\uc774 \ud560\uc560\ub429\ub2c8\ub2e4. \uc774\ub7ec\ud55c \uc774\uc720\ub85c AWS\uc5d0\uc11c \uc624\ud508 \uc18c\uc2a4 \ubc84\uc804\uc758 Apache Airflow\ub97c \uc27d\uac8c \uc2e4\ud589\ud558\uace0 \ucd94\ucd9c-\ubcc0\ud658-\ub85c\ub4dc(ETL) \uc791\uc5c5 \ubc0f \ub370\uc774\ud130 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc2e4\ud589\ud560 \uc6cc\ud06c\ud50c\ub85c\ub97c \uad6c\ucd95\ud560 \uc218 \uc788\ub294 \uc644\uc804\uad00\ub9ac\ud615 \uc11c\ube44\uc2a4, Amazon Managed Workflows for Apache Airflow(MWAA)\uc758 \ucd9c\uc2dc \uc18c\uc2dd\uc744 \uc804\ud558\uac8c \ub418\uc5b4 \uc815\ub9d0 \uae30\uc069\ub2c8\ub2e4. Airflow \uc6cc\ud06c\ud50c\ub85c\ub294 Amazon Athena \ucffc\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec Amazon Simple Storage Service(S3)\uc640 \uac19\uc740 \uc18c\uc2a4\uc5d0\uc11c \uc785\ub825\uc744 \uac80\uc0c9\ud558\uace0, <a class=\"mh-excerpt-more\" href=\"https:\/\/jirak.net\/wp\/amazon-managed-workflows-for-apache-airflowmwaa-%ec%a0%95%ec%8b%9d-%ec%b6%9c%ec%8b%9c\/\" title=\"Amazon Managed Workflows for Apache Airflow(MWAA) \uc815\uc2dd \ucd9c\uc2dc\">[ more&#8230; ]<\/a><\/p>\n<\/div>","protected":false},"author":1,"featured_media":39603,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[169],"tags":[656],"class_list":["post-39602","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-aws"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/39602","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/comments?post=39602"}],"version-history":[{"count":1,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/39602\/revisions"}],"predecessor-version":[{"id":39604,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/39602\/revisions\/39604"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/media\/39603"}],"wp:attachment":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/media?parent=39602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/categories?post=39602"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/tags?post=39602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}