Setup Apache Spark in JetBrains IntelliJ (Scala Version)

Please refer this post to see how to setup JetBrains IntelliJ IDE and Scala on your machine.

1) Download latest version of Apache Spark from http://spark.apache.org/downloads.html

2) Unpack the tarball to /opt/spark folder

3) Launch IntelliJ, create a new Scala project and choose SBT as the Build tool.

4) Once IntelliJ IDE is up and all dependencies have been resolved, you need to import Apache Spark’s jar libraries. Spark is written in Scala and you’ll have use the version of Scala with which your version of Spark was originally built. To figure that, just inspect /opt/spark/jars folder.



So from the above, it seems my version of Spark was built using Scala v2.11.

5) In IntelliJ, click on File > Project Structure, then ‘modules’ on the left and finally select ‘Dependencies’ tab on the right. Click on the green ‘+’ sign, browse to /opt/spark/jars folder and import all the jars under here in your project. I believe only a subset of jars (mainly hadoop, scala and spark) from here will actually be needed for running Spark but I haven’t figured which one is or isn’t needed at the moment.



6) Now switch to the ‘Libraries’ on the left and on the right, replace paths to existing scala compiler and library with jar paths from /opt/spark/jars folder.



7) Now hit OK to go back to main IDE, create a new Scala file under src/main/scala folder and type in the following code in the new file.



8) Save the file, press ALT + SHIFT + F10 to run the script and view the output.

0  

Setup Oracle Java (JDK – 8) on Ubuntu

Open up a Ubuntu terminal and run below statements in sequence to install JDK 8

$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer
$ sudo apt-get install oracle-java8-set-default

0  

Install JetBrains IntelliJ IDE with Scala plugin

I recently formatted my machine and have cleanly installed Ubuntu 17.04. I was planning to install JetBrains IntelliJ with Scala plugin and below steps will show you how to do that.

1) Download IntelliJ Community Edition from https://www.jetbrains.com/idea/download. As of today, I’m downloading version: ideaIC-2017.1.2

2) Extract the tarball somewhere or say under /home/{yourhomefolder}/apps and switch to the ‘bin’ folder in a terminal.

3) Run ./idea.sh script and follow the installation instructions.

4) During install, if you see a window prompting you to ‘download and install Scala’, please do click on that or else we can still do it later. Once installation is completed, you will see the below screen.


5) Click on Configure > Plugins, then click on ‘Browse Repositories’ and finally search for ‘Scala’.

6) Make sure Scala, SBT and SBT Executor plugins are installed. Feel free to install anything else you wish to. Finally click close and you’ll be prompted to restart IntelliJ which will display main IntelliJ launch window again.

7) Click on ‘Create New Project’, select ‘Scala’ and choose ‘SBT’ as the build tool. Give your project a name and follow instructions.

8) Once IDE has loaded and all dependencies have been resolved, click on File > Project Structure > Modules and under the Sources tab, select ‘Sources’ which will tell IntelliJ to mark your entire Project as a source code. Finally click OK. More details – http://stackoverflow.com/questions/42027117/could-not-find-or-load-main-class-in-intellij/

9) Create a new ‘HelloWorld.scala’ file under src/main/scala folder. If you don’t see an option to create a scala file, then right click on your project, click on ‘Add Framework Support’, choose Scala framework and close the window. Now you should be able to create a scala file. More details – http://stackoverflow.com/questions/38006735/new-to-scala-unable-to-create-scala-class-on-intellij

10) Open HelloWorld.scala file, paste below code, press ALT + Shift + F10 to execute the script and you will see output shortly.

object HelloWorld{
def main(args:Array[String]):Unit={
println("Hello, world!")
}
}


0  

Python – Matlab Reference Cheatsheet

http://mathesaurus.sourceforge.net/matlab-python-xref.pdf

0  

Installing Sun JDK 1.6 on Ubuntu

This is an excellent article which tells you how to install Sun JDK 1.6 on Ubuntu 12.10 and above.

http://www.liberiangeek.net/2012/11/install-oracle-java-jrejdk-6-in-ubuntu-12-10-quantal-quetzal/

0