Skip to content

Latest commit

 

History

History
100 lines (63 loc) · 4.3 KB

README.md

File metadata and controls

100 lines (63 loc) · 4.3 KB
Raw

vim-sparkShell

This is a very first, basic step for integrating the spark shell (or pyspark) with vim. So far it allows to start a spark shell in tmux, then vim-tbone or vimux is used to send lines to it. There are also functions to enter and exit paste mode.

It's just the minimum for what I wanted. I don't know how far it will evolve but I thought that maybe somebody might already think it's useful, as I am.

Disclaimer: these are my first vimscript lines and I just discovered tmux...

Dependencies

  • tmux (tested with version 1.9, fails with 1.6)
  • spark-shell, pyspark
  • vim-tbone of Tim Pope or vimux.

It has been tested with macvim, gvim, and vim under cygwin (embedded in tmux), but it might require some tweaking in your vimrc (see below).

Extra tip: I forked the vim-scaladoc of mdreves to include Spark's scala doc in the index (here). And of course, ctags is awesome.

Installation and usage

Install the above dependencies.

With pathogen

~/.vim/bundle
git clone git://github.com/tcarette/vim-sparkShell

You can add the following lines to your vimrc.

  • (Optional) For specifying a directory in which all jars have to be included. e.g.

      let g:jarDir    = $WORKDIR."/lib/"

  • Specifies spark path. e.g.

      let g:sparkShellPath = $HOME."/bin/spark/bin/spark-shell"

  • This option allows to use this vim-sparkShell from within tmux using the vimux plugin. This offers a more portable solution to communicating between vim and the spark shell.

    let g:inTmux = 1

  • To work with pyspark, define

    let g:pysparkMode = 1

The only difference is that it bypass anythong to do with the spark-shell paste environment

  • Useful mappings

The following mappings work for pyspark and spark-shell

	map <startMap>                  :call StartSparkShell("")<CR>
	map <enter paste>               :call SparkShellEnterPasteEnv()<CR>
	map <sendLine(s)Map>            :call SparkShellSendMultiLine() <CR>
	map <sendAll>                   :silent 1,$ call SparkShellSendMultiLine() <CR>

	nmap <exit paste>               :call SparkShellExitPasteEnv()<CR>
	nmap <killMap>                  :call StopSparkShell()<CR>
	nmap <sendWordUnderCursor>      :call SparkShellSendKey("<C-R><C-W>\r")<CR> 

	vmap <sendSelectionPerChar>    y:call SparkShellSendKey(substitute('<C-R>0',"\"","\\\"","")."\r")<CR>
	vmap <sendSelectionPerLine>     :call SparkShellSendMultiLine() <CR>

	nmap <countObjectUnderCursor>   :call SparkShellSendKey("<C-R><C-W>.count()\r")<CR><Esc>
	nmap <seeObjectUnderCursor>     :call SparkShellSendKey("<C-R><C-W>\r")<CR><Esc>
	vmap <seeObjectUnderCursor>    y:call SparkShellSendKey(substitute('<C-R>0',"\"","\\\"","")."\r")<CR>

The following is specific to scala (autocmd FileType scala <...>)

	nmap <take5ObjectUnderCursor>   :call SparkShellSendKey("<C-R><C-W>.take(5).foreach(println)\r")<CR><Esc>

and this one does the same thing in pyspark

	nmap <take5ObjectUnderCursor>   :call SparkShellSendKey("for e in <C-R><C-W>.take(5) : print(e)\r\r")<CR><Esc>

StartSparkShell takes extra options for the spark shell call, e.g.

	let g:sparkOptions = "--master local[2]"

Miscellaneous - Issues