Spark1.5.1 Cluster Environment Setup

Basically, I followed this blog to deploy the clusters’ environment. Instead of using spark 1.3.0, we adopt the latest version 1.5.1. But it doesn’t influence the configuration.

The issue maybe you would come across

I met a problem in the process of free-password login with SSH. Two of machines could access each other. Only one machine got trapped that other two machines couldn’t access it. After some investigation, we found it was due to the authority.

The authority of .ssh folder must be confined to 755

The authority of authorized_keys must be confined to 600