# Upgrading to later version of Python ## post by tancs711 on Jun 24, 2023 I need Python 3.9 or higher. The server I launch is 3.8 I ran ```bash sudo apt-get update && sudo apt-get upgrade sudo apt-get install python3.11 ``` But still `python3 --version` still shows 3.8. And my code did not run. ValidationError: 1 validation error for PythonAstREPLTool **root** This tool relies on Python 3.9 or higher (as it uses new functionality in the `ast` module, you have Python version: 3.8.10 (default, May 26 2023, 14:05:08) ```plaintext [GCC 9.4.0] (type=value_error) ``` ## post by addavox on Jun 24, 2023 This article should give you everything you need: [Installing Multiple Alternative Versions of Python on Ubuntu 20.04](https://towardsdatascience.com/installing-multiple-alternative-versions-of-python-on-ubuntu-20-04-237be5177474) Typically, with python projects, you run your project in a virtual environment in which you can specify which interpreter you will use. That is a little bit longer discussion. The article above should get you rolling. ## post by tancs711 on Jun 24, 2023 Thanks [@addavox](https://deeptalk.lambda.ai/u/addavox). Appreciate this. I am new to python environment. ## post by tancs711 on Jun 24, 2023 I tried these and managed to install another version of python. How do I get my jupyter notebook to use this version when I click on the launch link at the instance panel. The following `sys.version` still shows that Jupyter notebook is linked to the 3.8 (although on jupyter terminal it shows 3.11). I also created a new sudo user and created another environment. And installed jupyter lab there. When I tunnel into that, it still launches the 3.8 version. ## post by mpapili on Jun 24, 2023 What you’re looking for is doable in the Web JupyterLab that Lambda sets up for you. Here is a quick demonstration: **Step1:** Setting up a normal instance, nothing special. 1x_A10 if curious, but not important for this demonstration: **Step2:** Installing Python3.11 and setting up our VirtualEnv After either SSH’ing in or using the `Terminal` sessions from the JupyterLab: ```bash # install Python 3.11 to our system sudo apt update sudo apt install python3.11 python3.11-venv ``` now let’s set up the Virtual Environment and activate it normally: ```bash # make the venv python3.11 -m venv mikesVenv # activate source mikesVenv/bin/activate # check which python /home/ubuntu/mikesVenv/bin/python ``` -…obviously, name the `virtualenv` whatever you’d like **Step3:** Install ipykernel and add to Jupyter Make sure your virtualenv is still activated from above and: ```bash # install the ipykernel module pip install ipykernel # Add this environment to Jupyter python -m ipykernel install --user --name=py311 ``` **Step4:** Test it out! Refresh your JupyterLab session for this instance and you should see your new `py311` session ready and available to use if you go to `File` → `New Launcher`: -…and just replicating your test, making sure that we are indeed using our Python3.11 install: And that’s all there is to it! Let me know if this solution works for you. **One last note/disclaimer:** This is a fresh Python environment, so you won’t have things like `Torch` and `Tensorflow` which Lambda sets up for you out the box until you pip install them yourself in the venv. ## post by tancs711 on Jun 24, 2023 [@mpapili](https://deeptalk.lambda.ai/u/mpapili). Good news! I managed to change the kernel to 3.11 Can you let me know why the following did not work - Note is using the 311 kernel - Run `!pip install openai`. - But when I try and `import openai` I receive the following. ```sql -------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[3], line 3 1 get_ipython().system('pip -q install openai') 2 import os ----> 3 import openai 4 5 # This this to the location of the key file 6 with open('keys.txt', 'r') as file: ModuleNotFoundError: No module named 'openai' ``` ## post by tancs711 on Jun 24, 2023 It seems I cannot execute `!pip install openai` from the notebook even when the kernel is 311 and expect the module to be installed to the 311 environment. It seems I need to run `pip install openai` in a terminal with 311 myenv. Is this correct? I am just wondering why previously I was able to `!pip install openai` from the notebook and it would be done to the native 3.8 version. ## post by mpapili on Jun 25, 2023 While I’m not much of a Jupyter user personally, it seems like `get_ipython().system()` does not respect your current virtual environment in the same way that you may expect. It’s not running a session with the virtual environment “activated” in the same way (meaning `$PATH`’s are not set to point to your Python3.11’s “pip” and what you’re finding is the system’s “pip”). **Quick Demonstration:** Notice how my Python3.11 session is seeing the system `pip` while virtual environment’s `pip` (which will install modules in the desired PythonPath) is different. Fortunately, you can still install modules from within your Jupyter session without needing to hop back and forth between the terminal. While the commands issued to the system through `!` aren’t looking at your virtual environment, you can just use the full path to your Python interpreter: ```bash !/home/ubuntu/mikesVenv/bin/python -m pip install openai ``` then… ```swift import openai openai.__file__ # returns '/home/ubuntu/mikesVenv/lib/python3.11/site-packages/openai/__init__.py' ``` as we can see, the install and import works in our Python3.11 session and the module is installed in our desired Virtual Environment without needing to hop back and forth between an SSH or Terminal session and your notebook. Quick screenshot demonstrating this with `requests` and `pandas`: ## post by tancs711 on Jun 25, 2023 [@mpapili](https://deeptalk.lambda.ai/u/mpapili),. Thanks. It’s all working now with your suggestion to prepend the path to the pip. It’s a little cumbersome if we change the environment. But importantly, it’s a solution so I can continue the R&D work. ## post by ctjlewis on Jun 25, 2023 Not technically stable since you have to uninstall system packages but this should help. I needed this for use with [`axolotl`](https://github.com/OpenAccess-AI-Collective/axolotl).