Amazon SageMaker Notebook에서 (Amazon EMR기반) Apache Spark와의 연동 환경 구축 방법

2018-05-18 KENNETH 0

Amazon SageMaker Notebook에서 (Amazon EMR기반) Apache Spark와의 연동 환경 구축 방법 지난 AWS re:Invent 2017에서 처음 소개된 Amazon SageMaker는 데이터 과학과 머신 러닝 워크플로우를 위한 완전 관리형 서비스를 제공하고 있습니다. 특히 모델을 만드는데 사용되는 Jupyter notebook 인터페이스는 SageMaker에서 대단히 중요한 구성 요소 중 하나입니다. 한편Amazon Elastic MapReduce (Amazon EMR) 상에서 동작하는 Apache Spark 클러스터에 notebook 인스턴스를 연결시켜서 SageMaker의 성능을 훨씬 더 향상시킬 수 있습니다. 참고로 EMR은 대량의 데이터를 처리하기 위한 관리형 프레임워크입니다. 따라서 이와 같은 조합을 통해 대용량 데이터에 대한 모델 구축이 가능해집니다. Apache Spark는 빅 데이터를 빠른 속도로 처리할 수 있게 해주는 오픈소스 기반의 클러스터 컴퓨팅 프레임워크로, MLlib이라는 머신 러닝 작업을 위한 라이브러리도 제공하고 있습니다. 여기서는 SageMaker notebook과 Spark EMR 클러스터 간의 연동을 손쉽게 하기 위해, Livy라는 툴을 사용하는 방법을 설명하겠습니다. 참고로 Livy는 Spark 클라이언트를 사용하지 않고도 어디서든 Spark 클러스터와 상호 연동할 수 있게 해주는 오픈소스 REST 인터페이스입니다. 이 [ more… ]

No Image

Changes in MySQL 8.0.11 (General Availability)

2018-05-18 KENNETH 0

Changes in MySQL 8.0.11 (General Availability) The MySQL Development team announced the General Availability of MySQL 8.0 on April 19th, 2018. Here we follow up with references to worklogs that were added in 8.0.11. Note the jump in version number from 8.0.4 (RC2) to 8.0.11 (GA) explained here.… Source: Changes in MySQL 8.0.11 (General Availability)

Announcing support for Pen with Windows Application Driver v1.1 – Preview available now

2018-05-18 KENNETH 0

Announcing support for Pen with Windows Application Driver v1.1 – Preview available now Windows Application Driver (WinAppDriver) is continuing Microsoft’s investment in UI test automation tools for Windows 10, and now, we’re excited to announce the next release of WinAppDriver—version 1.1! A preview is available today— bringing support for Pen automation. The full v1.1 release is also on the horizon and will feature support for Multi-Touch in addition to Pen. What is WinAppDriver? For those of you who aren’t familiar with WinAppDriver or UI automation, WinAppDriver is an open-standards based UI automation service designed to work with all kinds of Windows 10 applications including WPF, WinForms, legacy Win32, and UWP. By complying with an open-standard, WinAppDriver users will be able to leverage the robust UI automation ecosystem already provided by Appium and Selenium. What’s new in the v1.1 Preview In [ more… ]

No Image

WordPress 4.9.6 Privacy and Maintenance Release

2018-05-18 KENNETH 0

WordPress 4.9.6 Privacy and Maintenance Release WordPress 4.9.6 is now available. This is a privacy and maintenance release. We encourage you to update your sites to take advantage of the new privacy features. Privacy The European Union’s General Data Protection Regulation (GDPR) takes effect on May 25. The GDPR requires companies and site owners to be transparent about how they collect, use, and share personal data. It also gives individuals more access and choice when it comes to how their own personal data is collected, used, and shared. It’s important to understand that while the GDPR is a European regulation, its requirements apply to all sites and online businesses that collect, store, and process personal data about EU residents no matter where the business is located. You can learn more about the GDPR from the European Commission’s Data Protection page. [ more… ]

No Image

Previewing support for same-site cookies in Microsoft Edge

2018-05-18 KENNETH 0

Previewing support for same-site cookies in Microsoft Edge Yesterday’s Windows Insider Preview build (build 17672) introduces support for the SameSite cookies standard in Microsoft Edge, ahead of a planned rollout in Microsoft Edge and Internet Explorer. Same-site cookies enable more protection for users against cross-site request forgery (CSRF) attacks. Historically, sites such as example.com that make “cross-origin” requests to other domains such as microsoft.com have generally caused the browser to send microsoft.com’s cookies as part of the request. Normally, the user benefits by being able to reuse some state (e.g., login state) across sites no matter from where that request originated. Unfortunately, this can be abused, as in CSRF attacks. Same-site cookies are a valuable addition to the defense in depth against CSRF attacks. Sites can now set the SameSite attribute on cookies of their choosing via the Set-Cookie header [ more… ]