- HOW TO INSTALL APACHE SPARK ON WINDOWS 7 HOW TO
- HOW TO INSTALL APACHE SPARK ON WINDOWS 7 DOWNLOAD
To quit Spark, at the command prompt type.Open up the web browser and type localhost:4040 in the address bar and you shall see the Spark shell application UI.Run the command spark-shell and you should see the spark logo with the scala prompt.Navigate into the bin directory like cd bin.For example in my case its present in the D directory Change directory to the location where the spark directory is.Which can be done as follows %JAVA_HOME%\BIN %PYTHON_PATH% %HADOOP_HOME% %SPARK_HOME% %M2_HOME%\BIN %MAVEN_HOME%\BIN (Note: Do not forget to end each entry with a semi-colon)Ĭlick on Ok to close the Environment variable window and then similarly on System properties window. Now, all you have to do is append these four system variables namely JAVA_HOME, PYTHON_PATH, HADOOP_HOME & SPARK_HOME to your Path variable. In my case it is in 'D:\APACHE-MAVEN-3.1.1\BIN' so I have MAVEN_HOME=D:\APACHE-MAVEN-3.1.1\BIN and M2_HOME=D:\APACHE-MAVEN-3.1.1\BIN (binary zip) and unpack it using WinZip or WinRAR. Create a new system variable and name it asĪssign the both these variables the value as the path to your Maven binary location.
HOW TO INSTALL APACHE SPARK ON WINDOWS 7 DOWNLOAD
Download Apache Maven 3.1.1 from here Choose Maven 3.1.1. NOTE: Apache Maven installation is an optional step. I am mentioning it here because I want to install SparkR a R version of Spark. The spark shell only requires the Hadoop path which in this case holds the value to winutils that will let us compile the spark program on a windows environment.Īssign the variable value as the path to your Spark binary location. (Note: There is no need to install Hadoop. Create a new system variable and name it as. Set the variable value as the Python Path on your computer. Similarly, create a new system variable and name it as. (please type the path without the single quote) In my case it is 'C:\Program Files\Java\jdk1.7.0_79\' Next set the variable value as the JDK PATH. (in case JAVA is not installed on your computer then follow these steps). We will create two new system variables, So click on “New” button under System variable You will see the window divided into two parts, the upper part will read User variables for username and the lower part will read System variables. Under Start up & Recovery, Click on the button labelled as “Environment Variable”. Right click on Computer- Left click on Properties. If the Path variable is not properly setup, you will not be able to start the spark shell. Setting up the PATH variable in Windows environment : 
Remember, Spark is a engine built over Hadoop. Actually, the official release of Hadoop 2.6 does not include the required binaries (like winutils.exe) which are required to run Hadoop.
Download and install winutils.exe and place it in any location in the D drive. If you are not a scala user then you also do not need to setup the scala path as the environment variable Download and install Scala version 2.10.4 from here only if you are a Scala user otherwise this step is not required. The benefit of using a pre-built binary is that you will not have to go through the trouble of building the spark binaries from scratch. (You can unzip it to any drive on your computer) Once downloaded I unzipped the *.tar file by using WinRar to the D drive. I chose Spark release 1.2.1, package type Pre-built for Hadoop 2.3 or later from here. Download a pre-built Spark binary for Hadoop. If you are not a python user then you also do not need to setup the python path as the environment variable If you are a Python user then Install Python 2.6+ or above otherwise this step is not required. To install Spark on a windows based environment the following prerequisites should be fulfilled first. And finally, I was able to come up with the following brief steps that lead me to a working instantiation of Apache Spark. HOW TO INSTALL APACHE SPARK ON WINDOWS 7 HOW TO
I invested two days searching the internet trying to find out how to install and configure it on a windows based environment. But I wanted to get a taste of this technology on my personal computer. However, they are using a pre-configured VM setup specific for the MOOC and for the lab exercises. In order to learn how to work on it currently there is a MOOC conducted by UC Berkley here. Apache Spark is a lightening fast cluster computing engine conducive for big data processing.
0 kommentar(er)
