For example, Databricks Runtime 5.x has Python 3.5, Databricks Runtime 5.x ML has Python 3.6, and Databricks Runtime 6.x and above Databricks Runtime 6.x ML have Python 3.7. Also, the minor version of your client python installation must be the same as the minor Python version of your Databricks cluster (2.7, 3.5, 3.6, or 3.7).
Make sure that the library version of databricks-connect matches your Databricks cluster’s runtime version. If you had previously installed Spark stand-alone mode, or are re-using an existing environment then first uninstall PySpark by pip uninstall pyspark then reinstall databricks-connect. I referenced many valuable tips below from their original article.
Here I have listed common issues that you may faced during the setup.Ī special thanks to and their comprehensive troubleshooting guide on Databricks Connect installation.
0 kommentar(er)
