Spark - Propagating Data to the Worker Nodes

A brief tutorial on propagating files and folders from the¬†driver node the worker nodes on Spark. Spark ships with a shell script, copy-dir.sh, to make copying data from the driver to workers very easy. This tutorial is a hello world for using that script. Let's assume a basic setup, 1 driver and 3¬†workers and that […]