Setup pycharm for pyspark in Cloudera
HTML-код
- Опубликовано: 8 сен 2024
- Versions may vary in the latest CDH. But this video gives a rough idea on the Python alt installation (without interrupting the system) and paths configuration for Python and PySpark in PyCharm (or in other IDEs such as Jupyter)
Update:
You will need to add one line after setting up PYSPARK_SUBMIT_ARGS so the spark-csv library can be downloaded without any error.
os.environ['JAVA_TOOL_OPTIONS'] = "-Dhttps.protocols=TLSv1.2"
I m not able to open pycharm website , it keeps loading but nothing is visible in it ,and my internet connection is quite stable