{"id":36778,"date":"2020-05-14T20:05:48","date_gmt":"2020-05-14T11:05:48","guid":{"rendered":"https:\/\/jirak.net\/wp\/apache-airflow-genie-%eb%b0%8f-amazon-emr%ec%9d%84-%ed%86%b5%ed%95%9c-%eb%b9%85%eb%8d%b0%ec%9d%b4%ed%84%b0-%ec%9b%8c%ed%81%ac%ed%94%8c%eb%a1%9c-%ec%98%a4%ec%bc%80%ec%8a%a4%ed%8a%b8%eb%a0%88%ec%9d%b4-2\/"},"modified":"2020-05-14T20:34:18","modified_gmt":"2020-05-14T11:34:18","slug":"apache-airflow-genie-%eb%b0%8f-amazon-emr%ec%9d%84-%ed%86%b5%ed%95%9c-%eb%b9%85%eb%8d%b0%ec%9d%b4%ed%84%b0-%ec%9b%8c%ed%81%ac%ed%94%8c%eb%a1%9c-%ec%98%a4%ec%bc%80%ec%8a%a4%ed%8a%b8%eb%a0%88%ec%9d%b4-2","status":"publish","type":"post","link":"https:\/\/jirak.net\/wp\/apache-airflow-genie-%eb%b0%8f-amazon-emr%ec%9d%84-%ed%86%b5%ed%95%9c-%eb%b9%85%eb%8d%b0%ec%9d%b4%ed%84%b0-%ec%9b%8c%ed%81%ac%ed%94%8c%eb%a1%9c-%ec%98%a4%ec%bc%80%ec%8a%a4%ed%8a%b8%eb%a0%88%ec%9d%b4-2\/","title":{"rendered":"Apache Airflow, Genie \ubc0f Amazon EMR\uc744 \ud1b5\ud55c \ube45\ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \u2013 1\ubd80"},"content":{"rendered":"

Apache Airflow, Genie \ubc0f Amazon EMR\uc744 \ud1b5\ud55c \ube45\ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \u2013 1\ubd80
\n<\/p>\n

AWS\uc5d0\uc11c \ube45 \ub370\uc774\ud130 ETL \uc6cc\ud06c\ud50c\ub85c\ub97c \uc2e4\ud589\ud558\ub294 \ub300\uae30\uc5c5\uc740 \ub9ce\uc740 \ub0b4\ubd80 \ucd5c\uc885 \uc0ac\uc6a9\uc790\ub97c \uc9c0\uc6d0\ud558\ub294 \ub300\uaddc\ubaa8\ub85c \uc6b4\uc601\ud558\uba70 \uc218\ucc9c \uac1c\uc758 \ub3d9\uc2dc \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc2e4\ud589\ud569\ub2c8\ub2e4. \uc774\ub7ec\ud55c \uc0c1\ud669\uacfc \uc0c8\ub85c\uc6b4 \ud504\ub808\uc784\uc6cc\ud06c \ubc0f \ube45\ub370\uc774\ud130 \ud504\ub85c\uc138\uc2f1 \ud504\ub808\uc784\uc6cc\ud06c\uc758 \ucd5c\uc2e0 \ub9b4\ub9ac\uc2a4\uc5d0 \ubcf4\uc870\ub97c \ub9de\ucd94\uae30 \uc704\ud574 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc9c0\uc18d\uc801\uc73c\ub85c \uc5c5\ub370\uc774\ud2b8 \ubc0f \ud655\uc7a5\ud574\uc57c \ud558\ub294 \ud544\uc694\uc131\uc5d0 \ub530\ub77c, \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc758 \uad00\ub9ac\ub97c \uac04\uc18c\ud654\ud560 \ubfd0 \uc544\ub2c8\ub77c \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \ub300\ud55c \uac04\ud3b8\ud55c \uc561\uc138\uc2a4\ub97c \uc9c0\uc6d0\ud558\ub294 \ud6a8\uc728\uc801\uc778 \uc544\ud0a4\ud14d\ucc98 \ubc0f \uc870\uc9c1\uc801 \uad6c\uc870\uac00 \uc694\uad6c\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

\uc774 \uac8c\uc2dc\ubb3c\uc5d0\uc11c\ub294 \uc218\ucc9c \uac1c\uc758 \ub3d9\uc2dc ETL \uc6cc\ud06c\ud50c\ub85c\ub97c \uc9c0\uc6d0\ud558\uace0 \uc774\ub97c \uc2e4\ud604\ud558\ub294 \ub370 \ud544\uc694\ud55c \uc6b4\uc601 \uc791\uc5c5\uc744 \uac04\uc18c\ud654\ud558\uae30 \uc704\ud55c \uc911\uc559 \uc9d1\uc911\uc2dd \ud50c\ub7ab\ud3fc \ud300\uc758 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc \uc720\uc9c0 \uad00\ub9ac \uc791\uc5c5\uc744 \ub3d5\ub294 \uc544\ud0a4\ud14d\ucc98\ub97c \uc18c\uac1c\ud569\ub2c8\ub2e4.<\/p>\n

\uc544\ud0a4\ud14d\ucc98 \uad6c\uc131 \uc694\uc18c<\/h2>\n

\ub192\uc740 \uc218\uc900\uc5d0\uc11c \ubcf4\uba74 \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 Amazon EMR<\/a>\uacfc \ud568\uaed8 \ub450 \uac00\uc9c0\uc758 \uc624\ud508 \uc18c\uc2a4 \uae30\uc220\uc744 \uc0ac\uc6a9\ud558\uc5ec ETL \uc6cc\ud06c\ud50c\ub85c \uc791\uc131, \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \ubc0f \uc2e4\ud589\uc744 \uc704\ud55c \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. Genie<\/a>\ub294 \ube45 \ub370\uc774\ud130 \uc791\uc5c5 \ub3d9\uc2dc \uc81c\ucd9c, \ub3d9\uc801 \uc791\uc5c5 \ub77c\uc6b0\ud305, \uc911\uc559 \uc9d1\uc911\uc2dd \uad6c\uc131 \uad00\ub9ac \ubc0f Amazon EMR \ud074\ub7ec\uc2a4\ud130\uc758 \ucd94\uc0c1\ud654\ub97c \uc704\ud55c \uc911\uc559 \uc9d1\uc911\uc2dd REST API\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4. Apache Airflow<\/a>\ub294 \ubcf5\uc7a1\ud55c \ub370\uc774\ud130 \ud30c\uc774\ud504\ub77c\uc778\uc5d0 \ub300\ud55c \ud504\ub85c\uadf8\ub798\ubc0d \ubc29\uc2dd\uc758 \uc791\uc131, \uc608\uc57d \ubc0f \ubaa8\ub2c8\ud130\ub9c1\uc744 \uc9c0\uc6d0\ud558\ub294 \uc791\uc5c5 \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158\uc744 \uc704\ud55c \ud50c\ub7ab\ud3fc\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. Amazon EMR\uc740 Apache Hadoop, Apache Spark \ubc0f \uae30\ud0c0 \ube45 \ub370\uc774\ud130 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \uc2e4\ud589 \ubc0f \uc870\uc815\ud560 \uc218 \uc788\ub294 \uad00\ub9ac\ud615 \ud074\ub7ec\uc2a4\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n

\ub2e4\uc74c \ub2e4\uc774\uc5b4\uadf8\ub7a8\uc740 \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c \ubcf4\uc5ec\uc90d\ub2c8\ub2e4.<\/p>\n

<\/span>\"\"<\/p>\n

Apache Airflow<\/h3>\n

Apache Airflow \ube45 \ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c\uc758 \uc791\uc131 \ubc0f \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158\uc744 \uc704\ud55c \uc624\ud508 \uc18c\uc2a4 \ub3c4\uad6c\uc785\ub2c8\ub2e4.<\/p>\n

\ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\ub294 Apache Airflow\ub97c \ud1b5\ud574 DAG<\/em>(Direct Acyclic Graph)\ub97c \uc815\uc758\ud569\ub2c8\ub2e4. DAG\ub294 \uc6cc\ud06c\ud50c\ub85c\ub97c \uc2e4\ud589\ud558\ub294 \ubc29\ubc95\uc744 \uc124\uba85\ud558\uba70 Python\uc73c\ub85c \uc791\uc131\ub429\ub2c8\ub2e4. \uc6cc\ud06c\ud50c\ub85c\ub294 \ub3c5\ub9bd\uc801\uc73c\ub85c \uc2e4\ud589\ub418\ub294 \uc791\uc5c5\uc744 \uadf8\ub8f9\ud654\ud558\ub294 DAG \ud615\ud0dc\ub85c \uc124\uacc4\ub429\ub2c8\ub2e4. DAG\ub294 \uc791\uc5c5 \uac04\uc758 \uad00\uacc4\uc640 \uc885\uc18d\uc131\uc744 \ucd94\uc801\ud569\ub2c8\ub2e4.<\/p>\n

\uc624\ud37c\ub808\uc774\ud130<\/em>\ub294 \uc6cc\ud06c\ud50c\ub85c \ub0b4\uc758 \ub2e8\uc77c \uc791\uc5c5\uc744 \uc815\uc758\ud558\ub294 \ud15c\ud50c\ub9bf\uc744 \uc815\uc758\ud569\ub2c8\ub2e4. Airflow\ub294 \uc624\ud37c\ub808\uc774\ud130\uc5d0 \uc77c\ubc18\uc801\uc778 \uc791\uc5c5\uc744 \uc81c\uacf5\ud558\uba70, \uc0ac\uc6a9\uc790 \uc815\uc758 \uc791\uc5c5\uc790\ub97c \uc815\uc758\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \uac8c\uc2dc\ubb3c\uc740 Genie\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud558\uae30 \uc704\ud55c \uc0ac\uc6a9\uc790 \uc815\uc758 \uc624\ud37c\ub808\uc774\ud130(GenieOperator)\uc5d0 \ub300\ud574 \uc124\uba85\ud569\ub2c8\ub2e4.<\/p>\n

\uc791\uc5c5<\/em>\uc740 \uc624\ud37c\ub808\uc774\ud130\uc758 \ud30c\ub77c\ubbf8\ud130\ud654\ub41c \uc778\uc2a4\ud134\uc2a4\uc785\ub2c8\ub2e4. \uc624\ud37c\ub808\uc774\ud130\uac00 \uc778\uc2a4\ud134\uc2a4\ud654\ub41c \ud6c4\uc5d0\ub294 \uc774\ub97c \uc791\uc5c5\uc774\ub77c\uace0 \ubd80\ub985\ub2c8\ub2e4. \uc791\uc5c5 \uc778\uc2a4\ud134\uc2a4<\/em>\ub294 \uc791\uc5c5\uc758 \uad6c\uccb4\uc801\uc778 \uc2e4\ud589\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4. \uc791\uc5c5 \uc778\uc2a4\ud134\uc2a4\ub294 \uad00\ub828\ub41c DAG, \uc791\uc5c5 \ubc0f \uc2dc\uc810\uc744 \uac00\uc9d1\ub2c8\ub2e4.<\/p>\n

DAG\uc640 \uc791\uc5c5\uc740 \uc628\ub514\ub9e8\ub4dc\ub85c \uc2e4\ud589\ud558\uac70\ub098 DAG\uc5d0 cron \ud45c\ud604\uc2dd\uc73c\ub85c \uc815\uc758\ub41c \ud2b9\uc815 \uc2dc\uac04\uc5d0 \uc2e4\ud589\ub418\ub3c4\ub85d \uc608\uc57d\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

Apache Airflow\uc5d0 \ub300\ud55c \ucd94\uac00\uc801\uc778 \uc138\ubd80 \uc815\ubcf4\ub294 Apache Airflow \uc124\uba85\uc11c\uc758 Concepts<\/a>\ub97c \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

Genie<\/h3>\n

Genie\ub294 Amazon EMR \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub300\ud55c \uc561\uc138\uc2a4\uc758 \ucd94\uc0c1\ud654\ub97c \ud1b5\ud574 \uad6c\uc131 \uad00\ub9ac \uae30\ub2a5 \ubc0f \uc791\uc5c5\uc758 \ub3d9\uc801 \ub77c\uc6b0\ud305\uc744 \uc81c\uacf5\ud558\ub294 \uc624\ud508 \uc18c\uc2a4 \ub3c4\uad6c\uc785\ub2c8\ub2e4.<\/p>\n

Genie\ub294 Apache Hadoop MapReduce \ub610\ub294 Apache Spark\uc640 \uac19\uc740 \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0\uc11c \uc791\uc5c5\uc744 \uc81c\ucd9c\ud560 \uc218 \uc788\uac8c \ud574 \uc8fc\ub294 REST API\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4. Genie\ub294 \ud074\ub7ec\uc2a4\ud130\uc640 \ud574\ub2f9 \ud074\ub7ec\uc2a4\ud130\uc5d0\uc11c \uc2e4\ud589\ub418\ub294 \uba85\ub839 \ubc0f \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc758 \uba54\ud0c0\ub370\uc774\ud130\ub97c \uad00\ub9ac\ud569\ub2c8\ub2e4.<\/p>\n

Genie\ub294 \ud074\ub7ec\uc2a4\ud130\uc5d0 \ud558\ub098 \uc774\uc0c1\uc758 \ud0dc\uadf8\ub97c \uc5f0\uacb0\ud558\uc5ec \ucc98\ub9ac \uc911\uc778 \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub300\ud55c \uc561\uc138\uc2a4\ub97c \ucd94\uc0c1\ud654\ud569\ub2c8\ub2e4. \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc774 \uc9c0\uc6d0\ud558\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uacfc \uba85\ub839\uc5d0 \ub300\ud55c \uba54\ud0c0\ub370\uc774\ud130 \uc138\ubd80 \uc815\ubcf4\uc5d0 \ud0dc\uadf8\ub97c \uc5f0\uacb0\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. Genie\ub294 \ud2b9\uc815 \ud0dc\uadf8\uc5d0 \ub300\ud55c \uc791\uc5c5 \uc81c\ucd9c\uc744 \uc218\uc2e0\ud558\uace0, \ud074\ub7ec\uc2a4\ud130\/\uba85\ub839 \ud0dc\uadf8\uc758 \uc870\ud569\uc744 \uc0ac\uc6a9\ud558\uc5ec \uac01 \uc791\uc5c5\uc744 \uc62c\ubc14\ub978 EMR \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub3d9\uc801\uc73c\ub85c \ub77c\uc6b0\ud305\ud569\ub2c8\ub2e4.<\/p>\n

Genie\uc758 \ub370\uc774\ud130 \ubaa8\ub378<\/h4>\n

Genie\ub294 \ube45 \ub370\uc774\ud130 \ud658\uacbd \ub0b4\uc758 \ub9ac\uc18c\uc2a4\uc640 \uad00\ub828\ub41c \uba54\ud0c0\ub370\uc774\ud130\ub97c \ucea1\ucc98\ud558\ub294 \ub370\uc774\ud130 \ubaa8\ub378\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n

\uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ub9ac\uc18c\uc2a4<\/em>\ub294 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud558\ub294 Genie \ub178\ub4dc\uc758 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc774 \uc9c0\uc6d0\ud558\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc744 \uc124\uce58 \ubc0f \uad6c\uc131\ud558\uae30 \uc704\ud55c, \uc7ac\uc0ac\uc6a9 \uac00\ub2a5\ud55c \ubc14\uc774\ub108\ub9ac, \uad6c\uc131 \ud30c\uc77c \ubc0f \uc124\uc815 \ud30c\uc77c\uc758 \uc138\ud2b8\uc785\ub2c8\ub2e4. Genie\uc5d0\uc11c \uc791\uc5c5\uc744 \uc218\uc2e0\ud558\uba74, Genie \ub178\ub4dc\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \uad00\ub828\ub41c \ubaa8\ub4e0 \uc885\uc18d\uc131, \uad6c\uc131 \ud30c\uc77c \ubc0f \uc124\uc815 \ud30c\uc77c\uc744 \ub2e4\uc6b4\ub85c\ub4dc\ud558\uc5ec \uc791\uc5c5 \ub514\ub809\ud130\ub9ac\uc5d0 \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc740 \uba85\ub839\uc5d0 \uc5f0\uacb0\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc774 \uba85\ub839\uc744 \uc2e4\ud589\ud558\uae30 \uc804\uc5d0 \ud544\uc694\ud55c \ubc14\uc774\ub108\ub9ac\uc640 \uad6c\uc131\uc744 \ub098\ud0c0\ub0b4\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.<\/p>\n

\uba85\ub839 \ub9ac\uc18c\uc2a4<\/em>\ub294 \uc791\uc5c5\uc744 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc81c\ucd9c\ud558\uae30 \uc704\ud574 \uba85\ub839\uc904\uc744 \uc0ac\uc6a9\ud560 \ub54c\uc758 \ud30c\ub77c\ubbf8\ud130 \ubc0f \uba85\ub839\uc744 \uc2e4\ud589\ud560 \ub54c PATH\uc5d0\uc11c \uc0ac\uc6a9 \uac00\ub2a5\ud574\uc57c \ud558\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4. \uba85\ub839 \ub9ac\uc18c\uc2a4\ub294 \uba54\ud0c0\ub370\uc774\ud130 \uad6c\uc131 \uc694\uc18c\ub97c \uc11c\ub85c \uacb0\ud569\ud574 \uc90d\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, Hive \uba85\ub839\uc744 \ub098\ud0c0\ub0b4\ub294 \uba85\ub839 \ub9ac\uc18c\uc2a4\ub294 hive-site.xml\uc744 \ud3ec\ud568\ud558\uba70 \uba85\ub839\uc744 \uc2e4\ud589\ud558\ub294 \ub370 \ud544\uc694\ud55c Hive \ubc0f Hadoop \ubc14\uc774\ub108\ub9ac\ub97c \uc81c\uacf5\ud558\ub294 \uc77c\ub828\uc758 \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ub9ac\uc18c\uc2a4\uc5d0 \uc5f0\uad00\ub429\ub2c8\ub2e4. \uadf8 \ubfd0 \uc544\ub2c8\ub77c, \uba85\ub839 \ub9ac\uc18c\uc2a4\ub294 \ud574\ub2f9 \ub9ac\uc18c\uc2a4\uac00 \uc2e4\ud589\ub420 \uc218 \uc788\ub294 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc5f0\uacb0\ub429\ub2c8\ub2e4.<\/p>\n

\ud074\ub7ec\uc2a4\ud130 \ub9ac\uc18c\uc2a4<\/em>\ub294 \uc5f0\uacb0 \uc138\ubd80 \uc815\ubcf4, \ud074\ub7ec\uc2a4\ud130 \uc0c1\ud0dc, \ud0dc\uadf8 \ubc0f \ucd94\uac00 \uc18d\uc131\uc744 \ud3ec\ud568\ud55c, \uc2e4\ud589 \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub300\ud55c \uc138\ubd80 \uc815\ubcf4\ub97c \uc2dd\ubcc4\ud569\ub2c8\ub2e4. \ud074\ub7ec\uc2a4\ud130 \ub9ac\uc18c\uc2a4\ub294 \uc790\ub3d9\uc73c\ub85c \uc2dc\uc791 \uc2dc Genie\uc5d0 \ub4f1\ub85d\ub418\uace0 \uc885\ub8cc \uc2dc \ub4f1\ub85d \ud574\uc81c\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud074\ub7ec\uc2a4\ud130\ub294 \ud574\ub2f9 \ud074\ub7ec\uc2a4\ud130\uc5d0\uc11c \uc2e4\ud589\ub420 \uc218 \uc788\ub294 \ud558\ub098 \uc774\uc0c1\uc758 \uba85\ub839\uc5d0 \uc5f0\uacb0\ub429\ub2c8\ub2e4. \uba85\ub839\uc774 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc5f0\uacb0\ub41c \ud6c4\uc5d0\ub294 Genie\uac00 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud558\uae30 \uc2dc\uc791\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

\ub9c8\uc9c0\ub9c9\uc73c\ub85c, \uc791\uc5c5 \ub9ac\uc18c\uc2a4<\/em>\uc5d0\ub294 \uc791\uc5c5 \uc694\uccad<\/em>, \uc791\uc5c5<\/em> \ubc0f \uc791\uc5c5 \uc2e4\ud589<\/em>\uc758 \uc138 \uac00\uc9c0 \uc720\ud615\uc774 \uc788\uc2b5\ub2c8\ub2e4. \uc791\uc5c5 \uc694\uccad \ub9ac\uc18c\uc2a4\ub294 \uc791\uc5c5 \uc2e4\ud589\uc744 \uc704\ud55c \uc138\ubd80 \uc815\ubcf4\uac00 \ud3ec\ud568\ub41c \uc694\uccad \uc81c\ucd9c\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4. \uc694\uccad\uc744 \ud1b5\ud574 \uc81c\ucd9c\ub41c \ud30c\ub77c\ubbf8\ud130\ub97c \uae30\ubc18\uc73c\ub85c \uc791\uc5c5 \ub9ac\uc18c\uc2a4\uac00 \uc0dd\uc131\ub429\ub2c8\ub2e4. \uc791\uc5c5 \ub9ac\uc18c\uc2a4\ub294 \uc791\uc5c5\uacfc \uad00\ub828\ub41c \uba85\ub839, \ud074\ub7ec\uc2a4\ud130 \ubc0f \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ub4f1\uc758 \uc138\ubd80 \uc815\ubcf4\ub97c \ucea1\ucc98\ud569\ub2c8\ub2e4. \uadf8 \ubfd0 \uc544\ub2c8\ub77c, \uc791\uc5c5 \ub9ac\uc18c\uc2a4\uc5d0\ub294 \uc0c1\ud0dc, \uc2dc\uc791 \uc2dc\uac04 \ubc0f \ub05d \uc2dc\uac04\uc5d0 \ub300\ud55c \uc815\ubcf4\ub3c4 \uc81c\uacf5\ub429\ub2c8\ub2e4. \uc791\uc5c5 \uc2e4\ud589 \ub9ac\uc18c\uc2a4\ub294 \uc791\uc5c5\uc774 \uc5b4\ub514\uc5d0\uc11c \uc2e4\ud589\ub418\uc5c8\ub294\uc9c0 \uc774\ud574\ud560 \uc218 \uc788\ub3c4\ub85d \uad00\ub9ac \uc138\ubd80 \uc815\ubcf4\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n

\uc790\uc138\ud55c \ub0b4\uc6a9\uc740 Genie Reference Guide\uc758 Data Model<\/a>\uc744 \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

Amazon EMR \ubc0f Amazon S3<\/h3>\n

Amazon EMR\uc740 \ubc29\ub300\ud55c \uc591\uc758 \ub370\uc774\ud130\ub97c \ucc98\ub9ac \ubc0f \ubd84\uc11d\ud558\uae30 \uc704\ud574 AWS\uc5d0\uc11c Apache Hadoop \ubc0f Apache Spark \uac19\uc740 \ube45 \ub370\uc774\ud130 \ud504\ub808\uc784\uc6cc\ud06c\uc758 \uc2e4\ud589\uc744 \uac04\uc18c\ud654\ud574 \uc8fc\ub294 \uad00\ub9ac\ud615 \ud074\ub7ec\uc2a4\ud130 \ud50c\ub7ab\ud3fc\uc785\ub2c8\ub2e4. \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 Amazon EMR \uc544\ud0a4\ud14d\ucc98 \uac1c\uc694<\/a> \ubc0f Amazon EMR \uac1c\uc694<\/a>\ub97c \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

\ub370\uc774\ud130\ub294 \ud655\uc7a5 \uac00\ub2a5\ud55c \uc131\ub2a5, \uc0ac\uc6a9\uc774 \ud3b8\ub9ac\ud55c \uae30\ub2a5 \ubc0f \ub124\uc774\ud2f0\ube0c \uc554\ud638\ud654 \ubc0f \uc561\uc138\uc2a4 \uc81c\uc5b4 \uae30\ub2a5\uc744 \uac16\ucd98 \uac1d\uccb4 \uc2a4\ud1a0\ub9ac\uc9c0 \uc11c\ube44\uc2a4\uc778 Amazon S3<\/a>\uc5d0 \uc800\uc7a5\ub429\ub2c8\ub2e4. S3\uc5d0 \ub300\ud55c \uc138\ubd80 \uc815\ubcf4\ub294 Amazon S3 as the Data Lake Storage Platform<\/a>\uc744 \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

\uc544\ud0a4\ud14d\ucc98 \uc2ec\uce35 \ubd84\uc11d<\/h3>\n

\ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\uc640 \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\ub77c\ub294 \ub450 \ubd80\ub958\uc758 \uc0ac\uc6a9\uc790\uac00 \uc774 \uc544\ud0a4\ud14d\ucc98\uc640 \uc0c1\ud638 \uc791\uc6a9\ud569\ub2c8\ub2e4.<\/p>\n

\ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\ub294 \ubaa8\ub4e0 \uad6c\uc131 \uc694\uc18c\uc5d0 \ub300\ud574 \uad00\ub9ac\uc790 \uad8c\ud55c\uc774 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub7ec\ud55c \uc0ac\uc6a9\uc790\ub294 \ud074\ub7ec\uc2a4\ud130\ub97c \ucd94\uac00 \ubc0f \uc81c\uac70\ud558\uace0, \ud50c\ub7ab\ud3fc\uc774 \uc9c0\uc6d0\ud558\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ubc0f \uba85\ub839\uc744 \uad6c\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

\ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\ub294 \uc120\ud638\ud558\ub294 \ud504\ub808\uc784\uc6cc\ud06c(Apache Spark, Apache Hadoop MR, Apache Sqoop, Apache Hive, Apache Pig \ubc0f Presto)\ub97c \uc0ac\uc6a9\ud55c \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \uc791\uc131\uacfc DAG\ub97c \ub098\ud0c0\ub0b4\ub294 Python \uc2a4\ud06c\ub9bd\ud2b8 \uc791\uc131\uc5d0 \uc9d1\uc911\ud569\ub2c8\ub2e4.<\/p>\n

\ub192\uc740 \uc218\uc900\uc5d0\uc11c \ubcf4\uba74 \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4 \ud300\uc740 \uc9c0\uc6d0\ub418\ub294 \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uacfc \uad00\ub828 \uc885\uc18d\uc131\uc744 \uc900\ube44\ud558\uace0 \uc774\ub97c Genie\uc5d0 \ub4f1\ub85d\ud569\ub2c8\ub2e4. \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4 \ud300\uc740 \uc2dc\uc791 \uc2dc Genie\uc5d0 \ub4f1\ub85d\ub418\ub294 Amazon EMR \ud074\ub7ec\uc2a4\ud130\ub97c \uc2dc\uc791\ud569\ub2c8\ub2e4.<\/p>\n

\ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4 \ud300\uc740 \uac01 Genie \uba54\ud0c0\ub370\uc774\ud130 \ub9ac\uc18c\uc2a4(\uc560\ud50c\ub9ac\ucf00\uc774\uc158, \uba85\ub839 \ubc0f \ud074\ub7ec\uc2a4\ud130)\ub97c Genie \ud0dc\uadf8\uc5d0 \uc5f0\uacb0\ud569\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, \ud074\ub7ec\uc2a4\ud130 \ub9ac\uc18c\uc2a4\ub97c environment\ub77c\ub294 \uc774\ub984\uc758 \ud0dc\uadf8\uc5d0 \uc5f0\uacb0\ud560 \uc218 \uc788\uc73c\uba70 \uc774 \ud0dc\uadf8\uc758 \uac12\uc740 \u201cProduction Environment<\/code>\u201d, \u201cTest Environment<\/code>\u201d \ub610\ub294 \u201cDevelopment Environment<\/code>\u201d\uac00 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

\ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\ub294 \uc6cc\ud06c\ud50c\ub85c\ub97c Airflow DAG \ud615\ud0dc\ub85c \uc791\uc131\ud558\uace0 \uc0ac\uc6a9\uc790 \uc815\uc758 Airflow \uc624\ud37c\ub808\uc774\ud130(GenieOperator)\ub97c \uc0ac\uc6a9\ud558\uc5ec Genie\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud569\ub2c8\ub2e4. \uc774\ub7ec\ud55c \uc0ac\uc6a9\uc790\ub294 \ud0dc\uadf8\uc758 \uc870\ud569\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc2e4\ud589 \uc911\uc778 \uc791\uc5c5\uc758 \uc720\ud615\uacfc \uc791\uc5c5\uc774 \uc2e4\ud589\ub418\uc5b4\uc57c \ud558\ub294 \uc704\uce58\ub97c \uc2dd\ubcc4\ud569\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, \u201cProduction Environment<\/code>\u201d \ud0dc\uadf8\uc5d0 \uc758\ud574 \uc2dd\ubcc4\ub41c \ud658\uacbd\uc5d0\uc11c Apache Spark 2.4.3 \uc791\uc5c5\uc744 \uc2e4\ud589\ud574\uc57c \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub97c \uc218\ud589\ud558\ub824\uba74 Airflow GenieOperator\uc5d0\uc11c \ud074\ub7ec\uc2a4\ud130 \ubc0f \uba85\ub839 \ud0dc\uadf8\ub97c \ub2e4\uc74c \ucf54\ub4dc \ud615\ud0dc\ub85c \uc124\uc815\ud569\ub2c8\ub2e4.<\/p>\n

(cluster_tags=['emr.cluster.environment:production'],command_tags=['type:spark-submit','version:2.4.3'])<\/code><\/p>\n

\ub2e4\uc74c \ub2e4\uc774\uc5b4\uadf8\ub7a8\uc740 \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c \ubcf4\uc5ec\uc90d\ub2c8\ub2e4.<\/p>\n

\"\"\uc774 \ub2e4\uc774\uc5b4\uadf8\ub7a8\uc758 \uc22b\uc790\uc5d0 \ud574\ub2f9\ud558\ub294 \uc6cc\ud06c\ud50c\ub85c\ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.<\/p>\n

    \n
  1. \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \uc9c0\uc6d0\ub418\ub294 \uc560\ud50c\ub9ac\ucf00\uc774\uc158(Spark-2.4.5, Spark-2.1.0, Hive-2.3.5 \ub4f1)\uc758 \ubc14\uc774\ub108\ub9ac \ubc0f \uc885\uc18d\uc131\uc744 \uc900\ube44\ud569\ub2c8\ub2e4. \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \uba85\ub839(spark-submit, hive)\uc744 \uc900\ube44\ud569\ub2c8\ub2e4. \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uacfc \uba85\ub839\uc744 Genie\uc5d0 \ub4f1\ub85d\ud569\ub2c8\ub2e4. \ub610\ud55c, \ud50c\ub7ab\ud3fc \uad00\ub9ac \uc5d4\uc9c0\ub2c8\uc5b4\ub294 \uba85\ub839\uc744 \ub2e8\uacc4 2(\uc544\ub798)\uac00 \uc644\ub8cc\ub41c \ud6c4 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \uc5f0\uacb0\ud558\uace0, \uba85\ub839\uc744 \uc77c\ub828\uc758 \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc5f0\uacb0\ud569\ub2c8\ub2e4.<\/li>\n
  2. Amazon EMR \ud074\ub7ec\uc2a4\ud130\uac00 \uc2dc\uc791 \uc2dc Genie\uc5d0 \ub4f1\ub85d\ub429\ub2c8\ub2e4.<\/li>\n
  3. \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\uac00 Airflow DAG\ub97c \uc791\uc131\ud558\uace0 Genie \ud0dc\uadf8\ub97c \uc0ac\uc6a9\ud558\uc5ec \ud658\uacbd, \uc560\ud50c\ub9ac\ucf00\uc774\uc158, \uba85\ub839 \ub610\ub294 \uc774\ub4e4\uc758 \uc870\ud569\uc744 \ucc38\uc870\ud569\ub2c8\ub2e4. \uc6cc\ud06c\ud50c\ub85c \ucf54\ub4dc\uc5d0\uc11c \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\uac00 GenieOperator\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4. GenieOperator\uac00 \uc791\uc5c5\uc744 Genie\uc5d0 \uc81c\ucd9c\ud569\ub2c8\ub2e4.<\/li>\n
  4. \uc77c\uc815\uc5d0 \ub530\ub77c \uc6cc\ud06c\ud50c\ub85c \uc2e4\ud589\uc774 \ud2b8\ub9ac\uac70\ub418\uac70\ub098 \ub370\uc774\ud130 \uc5d4\uc9c0\ub2c8\uc5b4\uac00 \uc218\ub3d9\uc73c\ub85c \uc6cc\ud06c\ud50c\ub85c \uc2e4\ud589\uc744 \ud2b8\ub9ac\uac70\ud569\ub2c8\ub2e4. \uc6cc\ud06c\ud50c\ub85c\ub97c \uad6c\uc131\ud558\ub294 \uc791\uc5c5\uc774 \uc791\uc5c5 \uc2e4\ud589 \uc704\uce58\ub97c \uc9c0\uc815\ud558\ub294 \uc77c\ub828\uc758 Genie \ud0dc\uadf8\uc640 \ud568\uaed8 \uc2e4\ud589\uc744 \uc704\ud574 Genie\uc5d0 \uc81c\ucd9c\ub429\ub2c8\ub2e4.<\/li>\n
  5. \ud074\ub77c\uc774\uc5b8\ud2b8 \uac8c\uc774\ud2b8\uc6e8\uc774 \uc5ed\ud560\uc744 \ud558\ub294 Genie \ub178\ub4dc\uac00 \ubaa8\ub4e0 \ubc14\uc774\ub108\ub9ac\uc640 \uc885\uc18d\uc131\uc774 \ud3ec\ud568\ub41c \uc791\uc5c5 \ub514\ub809\ud130\ub9ac\ub97c \uc124\uc815\ud569\ub2c8\ub2e4. Genie\ub294 \uc81c\uacf5\ub41c Genie \ud0dc\uadf8\uc5d0 \uc5f0\uad00\ub41c \ud074\ub7ec\uc2a4\ud130\uc5d0 \uc791\uc5c5\uc744 \ub3d9\uc801\uc73c\ub85c \ub77c\uc6b0\ud305\ud569\ub2c8\ub2e4. Amazon EMR \ud074\ub7ec\uc2a4\ud130\uac00 \uc791\uc5c5\uc744 \uc2e4\ud589\ud569\ub2c8\ub2e4.<\/li>\n<\/ol>\n

    Apache Airflow\uc640 Genie\uc5d0\uc11c \uc9c0\uc6d0\ud558\ub294 \uc2b9\uc778 \ubc0f \uc778\uc99d \uba54\ucee4\ub2c8\uc998\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 Apache Airflow \uc124\uba85\uc11c\uc758 Security<\/a> \ubc0f Genie \uc124\uba85\uc11c\uc758 Security<\/a>\ub97c \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.  \uc774 \uc544\ud0a4\ud14d\ucc98 \ud328\ud134\uc740 SSH \uc561\uc138\uc2a4\ub97c Amazon EMR \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub178\ucd9c\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. EMRFS(EMR \ud30c\uc77c \uc2dc\uc2a4\ud15c)\ub97c \ud1b5\ud574 Amazon S3\uc758 \ub370\uc774\ud130\uc5d0 \uc11c\ub85c \ub2e4\ub978 \uc218\uc900\uc758 \uc561\uc138\uc2a4\ub97c \uc81c\uacf5\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 Amazon S3\uc5d0 \ub300\ud55c EMRFS \uc694\uccad\uc758 IAM \uc5ed\ud560 \uad6c\uc131<\/a>\uc744 \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

    \ud65c\uc6a9 \uc0ac\ub840<\/h2>\n

    \ub2e4\uc74c \uc0ac\uc6a9 \uc0ac\ub840\ub294 \uc774 \uc544\ud0a4\ud14d\ucc98\uac00 \uc81c\uacf5\ud558\ub294 \uae30\ub2a5\uc744 \uc2dc\uc5f0\ud569\ub2c8\ub2e4.<\/p>\n

    \uc5c5\uadf8\ub808\uc774\ub4dc \ubc0f \ubc30\ud3ec\uc758 \ub2e4\uc6b4\ud0c0\uc784 \uc5c6\ub294 \uad00\ub9ac \ubc0f \ucd5c\uc2e0 \uc624\ud508 \uc18c\uc2a4 \ub9b4\ub9ac\uc2a4 \ub3c4\uc785<\/h3>\n

    \ub300\uaddc\ubaa8 \uc870\uc9c1\uc5d0\uc11c\ub294 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc0ac\uc6a9\ud558\ub294 \uc5ec\ub7ec \ud300\uc774 \uc774\uae30\uc885\uc758 \ud504\ub808\uc784\uc6cc\ud06c\uc640 \uc11c\ub85c \ub2e4\ub978 \ubc84\uc804\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4. \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c \uc0ac\uc6a9\ud558\uba74 \ub2e4\uc6b4\ud0c0\uc784 \uc5c6\ub294 \uc5c5\uadf8\ub808\uc774\ub4dc\ub97c \uc9c0\uc6d0\ud558\uace0 \uc9e7\uc740 \uc2dc\uac04\uc5d0 \ucd5c\uc2e0 \ubc84\uc804\uc758 \uc624\ud508 \uc18c\uc2a4 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \uc81c\uacf5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

    Genie\uc640 Amazon EMR\uc740 \uc774 \uc0ac\uc6a9 \uc0ac\ub840\ub97c \uc9c0\uc6d0\ud558\ub294 \ub370 \uc788\uc5b4 \uc8fc\uc694 \uad6c\uc131 \uc694\uc18c\uc785\ub2c8\ub2e4. Amazon EMR \uc11c\ube44\uc2a4 \ud300\uc774 \uc9e7\uc740 \ucd9c\uc2dc \uc8fc\uae30\ub85c Amazon EMR\uc5d0\uc11c \uc2e4\ud589\ub418\ub294 \ucd5c\uc2e0 \ubc84\uc804\uc758 \uc624\ud508 \uc18c\uc2a4 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \ucd94\uac00\ud558\uae30 \uc704\ud574 \ucd5c\uc120\uc744 \ub2e4\ud558\ubbc0\ub85c \uadc0\uc0ac\ub294 \uc120\ud638\ud558\ub294 \uc624\ud508 \uc18c\uc2a4 \ud504\ub808\uc784\uc6cc\ud06c\uc758 \ucd5c\uc2e0 \uae30\ub2a5\uc5d0 \ub300\ud55c \uc0ac\ub0b4 \ud300\uc758 \uc694\uad6c\ub97c \ucda9\uc871\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

    \uc0c8 \ubc84\uc804\uc758 \uc624\ud508 \uc18c\uc2a4 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uac8c \ub418\uba74, \uc774\ub97c \ud14c\uc2a4\ud2b8\ud558\uace0, \uc0c8\ub85c \uc9c0\uc6d0\ub418\ub294 \ubc84\uc804\uacfc \uad00\ub828 \uc885\uc18d\uc131\uc744 Genie\uc5d0 \ucd94\uac00\ud558\uace0, \uc774\uc804 \ud074\ub7ec\uc2a4\ud130\uc758 \ud0dc\uadf8\ub97c \uc0c8 \ud074\ub85c\uc2a4\ud130\ub85c \uc774\uc804\ud574\uc57c \ud569\ub2c8\ub2e4. \uc0c8 \ud074\ub7ec\uc2a4\ud130\ub294 \uc0c8 \uc791\uc5c5 \uc81c\ucd9c\uc744 \uc218\uc2e0\ud558\uba70 \uc774\uc804 \ud074\ub7ec\uc2a4\ud130\ub294 \uc544\uc9c1 \uc2e4\ud589 \uc911\uc778 \uc791\uc5c5\uc744 \uc644\ub8cc\ud569\ub2c8\ub2e4.<\/p>\n

    \uadf8 \ubfd0 \uc544\ub2c8\ub77c, Genie\uc5d0\uc11c \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ubc14\uc774\ub108\ub9ac \ubc0f \uad00\ub828 \uc885\uc18d\uc131\uc758 \uc704\uce58\ub97c \uc911\uc559 \uc9d1\uc911\ud654\ud558\ubbc0\ub85c, Genie\uc758 \ubc14\uc774\ub108\ub9ac \ubc0f \uad00\ub828 \uc885\uc18d\uc131\uc744 \uc5c5\uadf8\ub808\uc774\ub4dc\ud558\uba74 \ubaa8\ub4e0 \uc5c5\uc2a4\ud2b8\ub9bc \ud074\ub77c\uc774\uc5b8\ud2b8\ub3c4 \uc790\ub3d9\uc73c\ub85c \uc5c5\uadf8\ub808\uc774\ub4dc\ub429\ub2c8\ub2e4. Genie\ub97c \uc0ac\uc6a9\ud558\uba74 \ubaa8\ub4e0 \uc5c5\uc2a4\ud2b8\ub9bc \ud074\ub77c\uc774\uc5b8\ud2b8\ub97c \uc5c5\uadf8\ub808\uc774\ub4dc\ud560 \ud544\uc694\uac00 \uc5c6\uc5b4\uc9d1\ub2c8\ub2e4.<\/p>\n

    \uc911\uc559 \uc9d1\uc911\uc2dd \uad6c\uc131, \uc791\uc5c5 \uc0c1\ud0dc, \ud074\ub7ec\uc2a4\ud130 \uc0c1\ud0dc \ubc0f \ub85c\uae45 \uad00\ub9ac<\/h3>\n

    \uc218\ucc9c \uac1c\uc758 \uc791\uc5c5\uacfc \ubcf5\uc218\uc758 \ud074\ub7ec\uc2a4\ud130\uac00 \uc874\uc7ac\ud558\ub294 \ud658\uacbd\uc5d0\uc11c\ub294 \ud2b9\uc815 \uc791\uc5c5\uc774 \uc5b4\ub514\uc5d0\uc11c \uc2e4\ud589 \uc911\uc778\uc9c0 \uc2dd\ubcc4\ud574\uc57c \ud558\uace0 \ub85c\uae45 \uc138\ubd80 \uc815\ubcf4\ub97c \ube60\ub974\uac8c \uc561\uc138\uc2a4\ud574\uc57c \ud569\ub2c8\ub2e4. \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc5d0\uc11c \uc2e4\ud589 \uc911\uc778 \uc791\uc5c5, \uc791\uc5c5 \ub85c\uae45, \ud074\ub7ec\uc2a4\ud130 \ubc0f \uad00\ub828 \uad6c\uc131\uc5d0 \ub300\ud55c \uac00\uc2dc\uc131\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n

    \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc5d0 \ub300\ud55c \ud504\ub85c\uadf8\ub798\ubc0d \ubc29\uc2dd\uc758 \uc561\uc138\uc2a4<\/h3>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 Genie\uc758 REST API\ub97c \uc0ac\uc6a9\ud55c \ub2e8\uc77c \uc791\uc5c5 \uc81c\ucd9c \uc9c0\uc810\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4. \uae30\ubc18 \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub300\ud55c \uc561\uc138\uc2a4\ub294 \ud074\ub7ec\uc2a4\ud130\uc5d0 \ub300\ud55c \uad00\ub9ac \uc791\uc5c5\uacfc \uc791\uc5c5 \uc81c\ucd9c \ubaa8\ub450\ub97c \uc9c0\uc6d0\ud558\ub294 \uc77c\ub828\uc758 API\ub97c \ud1b5\ud574 \ucd94\uc0c1\ud654\ub429\ub2c8\ub2e4. REST API \ud638\ucd9c\uc740 \ube44\ub3d9\uae30\uc2dd\uc73c\ub85c Genie\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud569\ub2c8\ub2e4. \uc218\ub77d\ub41c \uacbd\uc6b0, \uc791\uc5c5 ID\uac00 \ubc18\ud658\ub429\ub2c8\ub2e4. \uc774 \uc791\uc5c5 ID\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc791\uc5c5 \uc0c1\ud0dc\ub97c \uc5bb\uc744 \uc218 \uc788\uc73c\uba70, API \ub610\ub294 \uc6f9 UI\ub97c \ud1b5\ud574 \ud504\ub85c\uadf8\ub798\ubc0d \ubc29\uc2dd\uc73c\ub85c \ud574\ub2f9 \uc815\ubcf4\ub97c \ucd9c\ub825\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. Genie \ub178\ub4dc\ub294 \uc791\uc5c5 \ub514\ub809\ud130\ub9ac\ub97c \uc124\uc815\ud558\uace0 \ubcc4\ub3c4 \ud504\ub85c\uc138\uc2a4\uc5d0\uc11c \uc791\uc5c5\uc744 \uc2e4\ud589\ud569\ub2c8\ub2e4.<\/p>\n

    \ub610\ud55c \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ubc0f Apache Airflow DAG\ub97c \uc704\ud574 \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c CI\/CD(\uc9c0\uc18d\uc801 \ud1b5\ud569 \ubc0f \uc9c0\uc18d\uc801 \uc804\ub2ec) \ud30c\uc774\ud504\ub77c\uc778\uc5d0 \ud1b5\ud569\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

    \ud655\uc7a5 \uac00\ub2a5\ud55c \ud074\ub77c\uc774\uc5b8\ud2b8 \uac8c\uc774\ud2b8\uc6e8\uc774 \ubc0f \ub3d9\uc2dc \uc791\uc5c5 \uc81c\ucd9c \uc9c0\uc6d0<\/h3>\n

    Genie \ub178\ub4dc\ub294 \ud074\ub77c\uc774\uc5b8\ud2b8 \uac8c\uc774\ud2b8\uc6e8\uc774 \uc5ed\ud560(\uc5e3\uc9c0 \ub178\ub4dc)\uc744 \ud558\uba70, \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc5d0 \uc791\uc5c5\uc744 \uc81c\ucd9c\ud558\ub294 \ub370 \uc0ac\uc6a9\ub418\ub294 \ud074\ub77c\uc774\uc5b8\ud2b8 \uac8c\uc774\ud2b8\uc6e8\uc774 \ub9ac\uc18c\uc2a4\uac00 \uc218\uc694\ub97c \ucda9\ub2f9\ud558\ub3c4\ub85d \uc218\ud3c9 \ud655\uc7a5\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \ubfd0 \uc544\ub2c8\ub77c, Genie\uc5d0\uc11c\ub294 \ub3d9\uc2dc \uc791\uc5c5\uc758 \uc81c\ucd9c\uc774 \ud5c8\uc6a9\ub429\ub2c8\ub2e4.<\/p>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c \uc0ac\uc6a9\ud574\uc57c \ud558\ub294 \uc0ac\ub840<\/h2>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 \uc77c\uc2dc\uc801\uc778 \ud074\ub7ec\uc2a4\ud130 \ub300\uc2e0 \ubcf5\uc218\uc758 \ub300\uaddc\ubaa8 \ub2e4\uc911 \ud14c\ub10c\ud2b8 \ud504\ub85c\uc138\uc2f1 \ud074\ub7ec\uc2a4\ud130\ub97c \uc0ac\uc6a9\ud558\ub294 \uc870\uc9c1\uc5d0 \uad8c\uc7a5\ub429\ub2c8\ub2e4. \uc870\uc9c1\uc774 \uc5b8\uc81c \uc77c\uc2dc\uc801\uc778 \ud074\ub7ec\uc2a4\ud130 \ub300\uc2e0 \uc0c1\uc2dc \uc2e4\ud589\ub418\ub294 \ud074\ub7ec\uc2a4\ud130\ub97c \uc0ac\uc6a9\ud574\uc57c \ud560\uc9c0\uc5d0 \ub300\ud55c \uc124\uba85\uc740 \uc774 \uac8c\uc2dc\ubb3c\uc758 \ubc94\uc704\ub97c \ubc97\uc5b4\ub0a9\ub2c8\ub2e4(EMR Airflow \uc624\ud37c\ub808\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uba74 Genie\uc5d0 \ub4f1\ub85d\ub418\ub294 Amazon EMR \ud074\ub7ec\uc2a4\ud130\ub97c \uc2dc\uc791\ud558\uace0 \uc791\uc5c5\uc744 \uc2e4\ud589\ud558\uace0 \ud074\ub7ec\uc2a4\ud130\ub97c \uc885\ub8cc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4). \uc774 \uc544\ud0a4\ud14d\ucc98\uc5d0\ub294 \uc608\uc57d \uc778\uc2a4\ud134\uc2a4\ub97c \uc0ac\uc6a9\ud574\uc57c \ud569\ub2c8\ub2e4. \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 \uc608\uc57d \uc778\uc2a4\ud134\uc2a4 \uc0ac\uc6a9<\/a>\uc744 \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 \ub9ce\uc740 \uc0ac\ub0b4 \ud300\uc744 \uc9c0\uc6d0\ud558\uba74\uc11c \uc218\ucc9c \uac1c\uc758 \uc791\uc5c5\uc744 \ub3d9\uc2dc\uc5d0 \uc2e4\ud589\ud574\uc57c \ud558\ub294 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc6b4\uc601 \ubc0f \uc720\uc9c0 \uad00\ub9ac\ud558\ub294 \uc911\uc559 \ud50c\ub7ab\ud3fc \ud300\uc744 \ubcf4\uc720\ud558\uace0\uc790 \ud558\ub294 \uc870\uc9c1\uc5d0 \ud2b9\ud788 \uad8c\uc7a5\ub429\ub2c8\ub2e4.<\/p>\n

    \uaddc\ubaa8\uac00 \uadf8\ub807\uac8c \ud06c\uc9c0 \uc54a\uac70\ub098 \uc774 \uc815\ub3c4 \uaddc\ubaa8\ub85c \uc131\uc7a5\ud560 \uac83\uc73c\ub85c \uc608\uc0c1\ub418\uc9c0 \uc54a\ub294 \uc870\uc9c1\uc5d0\uac8c\ub294 \uc774 \uc544\ud0a4\ud14d\ucc98\uac00 \uc801\ud569\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \ud074\ub7ec\uc2a4\ud130 \ucd94\uc0c1\ud654 \ubc0f \uc911\uc559 \uc9d1\uc911\uc2dd \uad6c\uc131 \uad00\ub9ac\ub294 \uc218\ucc9c \uac1c\uc758 \ub3d9\uc2dc \uc6cc\ud06c\ud50c\ub85c \ubc0f \uc218\ubc31 \uac1c \ud300\uc73c\ub85c \uad6c\uc131\ub418\uc5b4 \ud63c\ub3c8\uc2a4\ub7ec\uc6b8 \uc218 \uc788\ub294 \ud658\uacbd\uc5d0 \uad6c\uc870\ud654\ub41c \uc561\uc138\uc2a4\ub97c \uc81c\uacf5\ud558\ub294 \ub370 \uc774\uc0c1\uc801\uc785\ub2c8\ub2e4.<\/p>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 \ub192\uc740 \ube44\uc728\uc758 \ub2e4\uc2dc\uac04 \ub610\ub294 \uc911\ucca9 \uc6cc\ud06c\ud50c\ub85c \ubc0f \uc774\uae30\uc885 \ud504\ub808\uc784\uc6cc\ud06c(Apache Spark, Apache Hive, Apache Pig, Apache Hadoop MapReduce, Apache Sqoop \ub610\ub294 Presto)\ub97c \uc9c0\uc6d0\ud558\ub294 \uc870\uc9c1\uc5d0\uac8c\ub3c4 \uad8c\uc7a5\ub429\ub2c8\ub2e4.<\/p>\n

    \uadc0\uc0ac\uac00 Apache Spark\ub9cc \uc0ac\uc6a9\ud558\uba70 \uc774\uc804\uc5d0 \ub17c\uc758\ub41c \uad8c\uc7a5 \uc0ac\ud56d\uc5d0 \uc815\ub82c\ub418\uc5b4 \uc788\ub294 \uacbd\uc6b0\uc5d0\ub294 \uc774 \uc544\ud0a4\ud14d\ucc98\uac00 \uc5ec\uc804\ud788 \uc801\uc6a9\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc791\uc5c5 \uc81c\ucd9c\uc744 \uc704\ud55c \uc911\uc559 \uc9d1\uc911\uc2dd REST API, \ud074\ub7ec\uc2a4\ud130 \ucd94\uc0c1\ud654, \ub3d9\uc801 \uc791\uc5c5 \ub77c\uc6b0\ud305 \ub610\ub294 \uc911\uc559 \uc9d1\uc911\uc2dd \uad6c\uc131 \uad00\ub9ac\uc758 \ud544\uc694\uc131\uc744 \uc815\ub2f9\ud654\ud560 \ub9cc\ud55c \uaddc\ubaa8\uac00 \uc544\ub2cc \uc870\uc9c1\uc758 \uacbd\uc6b0 Apache Livy\uc640 Amazon EMR\uc758 \uc870\ud569\uc774 \uc801\uc808\ud55c \uc635\uc158\uc774 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4. Genie\ub294 \uc5e3\uc9c0 \ud074\ub77c\uc774\uc5b8\ud2b8 \uc5ed\ud560\uc744 \uc218\ud589\ud558\ub294 \ud655\uc7a5 \uac00\ub2a5\ud55c \uc790\uccb4 \uc778\ud504\ub77c\ub97c \ubcf4\uc720\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub294 Apache Livy\uc640 \ub2ec\ub9ac Genie\ub294 Amazon EMR \ub9c8\uc2a4\ud130 \uc778\uc2a4\ud134\uc2a4 \ub9ac\uc18c\uc2a4\uc640 \uacbd\ud569\ud558\uc9c0 \uc54a\uc74c\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.<\/p>\n

    \uadc0\uc0ac \uc6cc\ud06c\ud50c\ub85c\uc758 \ub300\ubd80\ubd84\uc774 \uc9e7\uc740 \uc218\uba85\uc758 \uc791\uc5c5 \uba87 \uac1c\uc5d0 \ubd88\uacfc\ud55c \uacbd\uc6b0, \uc11c\ubc84\ub9ac\uc2a4 \ud504\ub85c\uc138\uc2f1 \ub808\uc774\uc5b4, \uc11c\ubc84\ub9ac\uc2a4 \uc784\uc2dc \ucffc\ub9ac \ub808\uc774\uc5b4 \ub610\ub294 \uc6cc\ud06c\ud50c\ub85c\ubcc4\ub85c \uc804\uc6a9 \uc784\uc2dc Amazon EMR \ud074\ub7ec\uc2a4\ud130\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc774 \ub354 \uc801\ud569\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadc0\uc0ac \uc6cc\ud06c\ud50c\ub85c\uc758 \ub300\ubd80\ubd84\uc774 \uc9e7\uc740 \uc218\uba85\uc758 \uc791\uc5c5 \uc218\ucc9c \uac1c\ub85c \uad6c\uc131\ub41c \uacbd\uc6b0, \ud074\ub7ec\uc2a4\ud130 \uc2dc\uc791 \ubc0f \uc885\ub8cc\uc758 \ud544\uc694\uc131\uc744 \uc81c\uac70\ud574 \uc8fc\ubbc0\ub85c \uc774 \uc544\ud0a4\ud14d\ucc98\uac00 \uc5ec\uc804\ud788 \uc801\uc6a9\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

    \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 \uad6c\uc131 \uc694\uc18c \uc131\ub2a5 \ucd5c\uc801\ud654\ub97c \uc704\ud558 \ud504\ub85c\uc138\uc2f1 \ud50c\ub7ab\ud3fc\uc758 \uc644\uc804\ud55c \uc81c\uc5b4\uac00 \ud544\uc694\ud55c \uc870\uc9c1\uc5d0\uac8c \uad8c\uc7a5\ub429\ub2c8\ub2e4. \uadf8 \ubfd0 \uc544\ub2c8\ub77c, \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 CI\/CD \ud30c\uc774\ud504\ub77c\uc778\uc744 \ud1b5\ud574 \uc6cc\ud06c\ud50c\ub85c\uc5d0 \uc911\uc559 \uc9d1\uc911\uc2dd \uac70\ubc84\ub10c\uc2a4\ub97c \uc801\uc6a9\ud574\uc57c \ud558\ub294 \uc870\uc9c1\uc5d0\uac8c \uad8c\uc7a5\ub429\ub2c8\ub2e4.<\/p>\n

    \uc11c\ub85c \ub2e4\ub978 \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \uc635\uc158\uc774\ub098 Airflow\ub97c \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \ub808\uc774\uc5b4\ub85c \ub3c4\uc785\ud560 \ub54c\uc758 \uc774\uc810\uc744 \ud3c9\uac00\ud558\ub294 \uac83\uc740 \uc774 \uac8c\uc2dc\ubb3c\uc758 \ubc94\uc704\ub97c \ubc97\uc5b4\ub0a9\ub2c8\ub2e4. \uc544\ud0a4\ud14d\ucc98 \ub3c4\uc785\uc744 \uace0\ub824\ud560 \ub54c\uc5d0\ub294 \uad00\ub828 \ub3c4\uad6c\ub97c \ub3c4\uc785\ud558\uae30 \uc704\ud55c \uc2dc\uac04\uacfc \uae30\uc874 \ubcf4\uc720 \uae30\uc220\ub3c4 \uace0\ub824\ud574\uc57c \ud569\ub2c8\ub2e4. Genie\uc758 \uc624\ud508 \uc18c\uc2a4 \ud2b9\uc131\uc740 \ub2e4\ub978 \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \ub3c4\uad6c\uc640\uc758 \ud1b5\ud569\uc744 \uc9c0\uc6d0\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \uc544\ud0a4\ud14d\ucc98\ub97c \ub2e4\ub978 \uc624\ucf00\uc774\uc2a4\ud14c\uc774\uc158 \ub3c4\uad6c\uc640 \ud1b5\ud569\ud558\uace0\uc790 \ud558\ub294 \uacbd\uc6b0 \uc774\ub7ec\ud55c \uacbd\ub85c\ub97c \ud3c9\uac00\ud558\ub294 \uac83\uc774 \uc635\uc158\uc774 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

    \ub9c8\ubb34\ub9ac<\/h2>\n

    \uc774 \uac8c\uc2dc\ubb3c\uc5d0\uc11c\ub294 Apache Airflow, Genie \ubc0f Amazon EMR\uc744 \uc0ac\uc6a9\ud558\uc5ec \ube45 \ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c\ub97c \uad00\ub9ac\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \uc18c\uac1c\ud588\uc2b5\ub2c8\ub2e4. \uc774 \uac8c\uc2dc\ubb3c\uc740 \uc544\ud0a4\ud14d\ucc98 \uad6c\uc131 \uc694\uc18c, \ud574\ub2f9 \uc544\ud0a4\ud14d\ucc98\uac00 \uc9c0\uc6d0\ud558\ub294 \uc0ac\uc6a9 \uc0ac\ub840 \ubc0f \uc0ac\uc6a9\ud574\uc57c \ud560 \uc0c1\ud669\uc5d0 \ub300\ud55c \uc124\uba85\uc744 \uc81c\uacf5\ud588\uc2b5\ub2c8\ub2e4. \ub2e4\uc74c 2\ubd80 \uac8c\uc2dc\ubb3c<\/a>\uc5d0\uc11c\ub294 \ub370\ubaa8 \ud658\uacbd\uc744 \ubc30\ud3ec\ud558\uace0 Genie\ub97c \uad6c\uc131\ud558\uace0 GenieOperator for Apache Airflow\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc744 \ub2e8\uacc4 \ubcc4\ub85c \uc548\ub0b4\ud569\ub2c8\ub2e4.<\/p>\n

     <\/p>\n

    \"\"<\/strong>Francisco Oliveira\ub294 AWS\uc758 \ube45 \ub370\uc774\ud130 \uc194\ub8e8\uc158 \uc120\uc784 \uc544\ud0a4\ud14d\ud2b8\uc785\ub2c8\ub2e4. \uc624\ud508 \uc18c\uc2a4 \uae30\uc220\uacfc AWS\ub97c \ud1b5\ud574 \ube45 \ub370\uc774\ud130 \uc194\ub8e8\uc158\uc744 \uad6c\ucd95\ud558\ub294 \ub370 \ucd08\uc810\uc744 \ub9de\ucd94\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub294 \uc5ec\uac00 \uc2dc\uac04\uc5d0 \uc0c8\ub85c\uc6b4 \uc2a4\ud3ec\uce20\uc640 \uc5ec\ud589\uc744 \uc990\uae30\uba70 \uad6d\ub9bd \uacf5\uc6d0\uc744 \ud0d0\uc0c9\ud558\ub294 \uac83\uc744 \uc88b\uc544\ud569\ub2c8\ub2e4.<\/p>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n

    \"\"<\/p>\n

    Jelez Raditchkov\ub294 AWS\uc5d0\uc11c NoSQL AWS Professional Services \uc5c5\ubb34\ub97c \uc120\ub3c4\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. NoSQL, Graph \ubc0f Search \uc601\uc5ed\uc5d0 \ub300\ud55c \uc9d1\uc911\ud654\ub41c \uc9c0\uc6d0\uc744 \ud1b5\ud574 \uace0\uac1d\uc774 \uc6d0\ud558\ub294 \ube44\uc988\ub2c8\uc2a4 \uacb0\uacfc\ub97c \uc2e4\ud604\ud560 \uc218 \uc788\ub3c4\ub85d \ub3d5\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774\uc804\uc5d0 \uadf8\ub294 AWS Professional Services\uc758 \uc218\uc11d \ub370\uc774\ud130 \ub808\uc774\ud06c \uc544\ud0a4\ud14d\ud2b8\uc600\uc2b5\ub2c8\ub2e4.<\/p>\n

    AWS NoSQL \uc804\uc6a9 \ub370\uc774\ud130\ubca0\uc774\uc2a4 \uc11c\ube44\uc2a4\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 https:\/\/aws.amazon.com\/nosql\/<\/a>\uc744 \ucc38\uc870\ud558\uc2ed\uc2dc\uc624.<\/p>\n

    Source: Apache Airflow, Genie \ubc0f Amazon EMR\uc744 \ud1b5\ud55c \ube45\ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \u2013 1\ubd80<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    Apache Airflow, Genie \ubc0f Amazon EMR\uc744 \ud1b5\ud55c \ube45\ub370\uc774\ud130 \uc6cc\ud06c\ud50c\ub85c \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \u2013 1\ubd80 AWS\uc5d0\uc11c \ube45 \ub370\uc774\ud130 ETL \uc6cc\ud06c\ud50c\ub85c\ub97c \uc2e4\ud589\ud558\ub294 \ub300\uae30\uc5c5\uc740 \ub9ce\uc740 \ub0b4\ubd80 \ucd5c\uc885 \uc0ac\uc6a9\uc790\ub97c \uc9c0\uc6d0\ud558\ub294 \ub300\uaddc\ubaa8\ub85c \uc6b4\uc601\ud558\uba70 \uc218\ucc9c \uac1c\uc758 \ub3d9\uc2dc \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc2e4\ud589\ud569\ub2c8\ub2e4. \uc774\ub7ec\ud55c \uc0c1\ud669\uacfc \uc0c8\ub85c\uc6b4 \ud504\ub808\uc784\uc6cc\ud06c \ubc0f \ube45\ub370\uc774\ud130 \ud504\ub85c\uc138\uc2f1 \ud504\ub808\uc784\uc6cc\ud06c\uc758 \ucd5c\uc2e0 \ub9b4\ub9ac\uc2a4\uc5d0 \ubcf4\uc870\ub97c \ub9de\ucd94\uae30 \uc704\ud574 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc9c0\uc18d\uc801\uc73c\ub85c \uc5c5\ub370\uc774\ud2b8 \ubc0f \ud655\uc7a5\ud574\uc57c \ud558\ub294 \ud544\uc694\uc131\uc5d0 \ub530\ub77c, \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc758 \uad00\ub9ac\ub97c \uac04\uc18c\ud654\ud560 \ubfd0 \uc544\ub2c8\ub77c \ube45 \ub370\uc774\ud130 \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \ub300\ud55c \uac04\ud3b8\ud55c \uc561\uc138\uc2a4\ub97c \uc9c0\uc6d0\ud558\ub294 \ud6a8\uc728\uc801\uc778 \uc544\ud0a4\ud14d\ucc98 \ubc0f \uc870\uc9c1\uc801 \uad6c\uc870\uac00 \uc694\uad6c\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \uac8c\uc2dc\ubb3c\uc5d0\uc11c\ub294 \uc218\ucc9c \uac1c\uc758 \ub3d9\uc2dc ETL \uc6cc\ud06c\ud50c\ub85c\ub97c \uc9c0\uc6d0\ud558\uace0 \uc774\ub97c \uc2e4\ud604\ud558\ub294 \ub370 \ud544\uc694\ud55c \uc6b4\uc601 \uc791\uc5c5\uc744 \uac04\uc18c\ud654\ud558\uae30 \uc704\ud55c \uc911\uc559 \uc9d1\uc911\uc2dd \ud50c\ub7ab\ud3fc \ud300\uc758 \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc \uc720\uc9c0 \uad00\ub9ac \uc791\uc5c5\uc744 \ub3d5\ub294 \uc544\ud0a4\ud14d\ucc98\ub97c \uc18c\uac1c\ud569\ub2c8\ub2e4. \uc544\ud0a4\ud14d\ucc98 \uad6c\uc131 \uc694\uc18c \ub192\uc740 \uc218\uc900\uc5d0\uc11c \ubcf4\uba74 \uc774 \uc544\ud0a4\ud14d\ucc98\ub294 Amazon EMR\uacfc \ud568\uaed8 \ub450 \uac00\uc9c0\uc758 \uc624\ud508 \uc18c\uc2a4 \uae30\uc220\uc744 \uc0ac\uc6a9\ud558\uc5ec ETL \uc6cc\ud06c\ud50c\ub85c \uc791\uc131, \uc624\ucf00\uc2a4\ud2b8\ub808\uc774\uc158 \ubc0f \uc2e4\ud589\uc744 \uc704\ud55c \ube45 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. Genie\ub294 \ube45 \ub370\uc774\ud130 \uc791\uc5c5 \ub3d9\uc2dc \uc81c\ucd9c, \ub3d9\uc801 \uc791\uc5c5 [ more… ]<\/a><\/p>\n<\/div>","protected":false},"author":1,"featured_media":36779,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[169],"tags":[656],"class_list":["post-36778","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-aws"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/36778","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/comments?post=36778"}],"version-history":[{"count":1,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/36778\/revisions"}],"predecessor-version":[{"id":36780,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/posts\/36778\/revisions\/36780"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/media\/36779"}],"wp:attachment":[{"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/media?parent=36778"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/categories?post=36778"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jirak.net\/wp\/wp-json\/wp\/v2\/tags?post=36778"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}