Linux [Ubuntu, other versions are for reference only] Run the Pyspark program in Jupyter Notebook error, query information discovery, current2020.11.18Pyspark still does not support higher version of Python, only supports Python3.7 (some blogs may write 3.6, updated).

solution

• Switch the Python version of Jupyter Notebook in Anaconda, because the process is more complicated, did not succeed
• Repair Anaconda3

Specific process

1. Uninstall the original Anaconda3, the default installation path is ~ / Anaconda3, directly delete

$ rm -rf ~/anaconda3

Then delete the configuration file~/.bashrcCentral and CONDA-related configuration, Note: There is a file at the end of the fileconda initNote, all of which delete it, and there is a path configuration related to the PATH section

There is a way to delete the use of toolsanaconda-cleanHowever, test findings is not used, maybe it is wrong :-D

2. Download ANACONDA3 >>
   Official websiteIf you have an IDE accelerator, speed is moderate
   Tsinghua University Mirror【Recommended Use】

Pay attention to the selected version,2020.02 is still Python3.7, and 2020.07 has been updated to 3.8 (Pyspark is not supported)

3. Installation configuration, referencexmu.db
   Surridden to jump directly into the code

$ CD installation directory
$ bash Anaconda3-xxx.sh

...
 By default, enter/yes
...

Reopen the terminal

# Remove (Base)
$ conda config --set auto_activate_base false
# Configure
$ vi ~/.bashrc

 Modify PATH content:
	export PATH=$PATH
>>
	export PATH=$PATH:/home/Your username/anaconda3/bin

$ source ~/.bashrc

The problem is basically solved, the next is to configure Jupyter Notebook, recommended referencexmu.db

Solve this problem, turn down the Python version in Linux, it is very clear!

[Note: The final Anaconda version of Python in the system should be consistent, and it will report an error in inconsistency! 】

Python version of Linux Reference here >>Linux (ubuntu) Python version switch