VSCode remote-container: getting started with Python

I have already talked about the remote-ssh VSCode extension that changed my way to code in Python for small electronics projects with a Raspberry Pi. Here we'll take a look at an extension that makes it a lot easier to develop inside a container. We will concentrate on remote-container by using the first steps of a Python/Flask tutorial. This will allow beginners to follow along as well.

Before we start, just out of curiosity, who installs a ton of sh... , crumm, a lot of runtimes and libraries in different versions on his/her main machine? Who installs the latest and greatest to test it out on his/her main system? Aaaah, thought so. We don't want to risk breaking a well tuned system we rely on for our day to day work! So, what do we do?
We quickly spawn a local virtual machine with Vagrant: we can break whatever we want in there. We use a throw away VM in the cloud. We install RVM or NVM to play with different ruby or node versions. We use specific Gemsets for different projects. We use Virtualenv to avoid installing python modules system wide. We craft development docker containers with the tooling we need and switch back and forth between the main system where we code and the container were we run. Some develop inside the container but most don't. We look for ways to keep our main system uncluttered and safe because we need it to be functional every day. We do not want to spend hours figuring out what and/or why something broke.

The remote-container extension makes working with containers much easier. You get all the benefits of VSCode and its extensions, and the ability to use a container to code, test and debug. You get all the benefits from the container: you abstract runtime, libs, binaries, code and configuration from the underlying host.

The first thing I did was use the quick start for python. A dialog pops up asking me if I want to run the project in a container. I do. And voilà. Now it's time to understand what does what a little more.

I chose to use the flask mega tutorial to put my self in the situation where I want to start a new project in a container. We are going to follow the tutorial along, but without a python virtual env. We'll do it in a container.

An empty "flask" folder is the start. I open VSCode with the folder as its context.

Python/bob/flask at ☸️  kubernetes-admin@kubernetes
➜ code .

I add the container configuration for python.

In the .devcontainer, I now have:

• devcontainer.json: the configuration for the extension
  • using a Dockerfile to build our python 3 container
  • listing extensions to be installed in the container (just the python extension for now)
  • setting path for python, pylint and enabling linting
• Dockerfile: the building blocks of the container we will be using
• noop.txt: a dummy file to copy in case we have no requirements.txt

This is exactly what we had in the discovery project I played with a minute ago.
OK, but I want at least two more extensions: Andvanced New File and Better Comments. Looking at my locally installed extensions, I retrieve the name of the extension to put in into the devcontainer.json file.

I want to run the container as a non root user.
I want to expose the 5000 port so I can reach the app inside the container.

This is my devcontainer.json file:

// For format details, see https://aka.ms/vscode-remote/devcontainer.json or the definition README at
// https://github.com/microsoft/vscode-dev-containers/tree/master/containers/python-3
{
	"name": "Python 3",
	"context": "..",
	"dockerFile": "Dockerfile",

	// Uncomment the next line if you want to publish any ports.
	"appPort": 5000,

	// Uncomment the next line to run commands after the container is created.
	// "postCreateCommand": "python --version",

	// Uncomment the next line to use a non-root user. See https://aka.ms/vscode-remote/containers/non-root-user.
	"runArgs": [ "-u", "1000" ],

	"extensions": [
		"ms-python.python",
		"patbenatar.advanced-new-file",
		"aaron-bond.better-comments"
	],
	"settings": {
		"python.pythonPath": "/usr/local/bin/python",
		"python.linting.pylintEnabled": true,
		"python.linting.pylintPath": "/usr/local/bin/pylint",
		"python.linting.enabled": true
	}
}

Obviously, I will need to add flask to the requirements.txt in the root directory.

CMD+SHIFT+P and I select "Reopen folder in container". In the terminal tab, I can see the container being built. I can see the extensions being installed. No errors, full stop.

I select a new terminal and check the user ID. All good. I list the installed modules and Flask is installed.

Well, let's go back to the tutorial and add the few files we need for the start. I skip the virtualenv part, I'm working in the container. I end up with:

• app/__init__.py
• app/routes.py
• microblog.py

I export the environment variable and run the app in the VSCode terminal (thus the container).

vscode@50adb1befb2f:/workspaces/flask$ export FLASK_APP=microblog.py
vscode@50adb1befb2f:/workspaces/flask$ flask run 
 * Serving Flask app "microblog.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on https://127.0.0.1:5000/ (Press CTRL+C to quit)

It seems to work just fine, but this isn't what I need. I want access into the container from my browser. This is running on localhost inside the container. I run the command again, but this time listening for everything.

vscode@50adb1befb2f:/workspaces/flask$ flask run -h 0.0.0.0  
 * Serving Flask app "microblog.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on https://0.0.0.0:5000/ (Press CTRL+C to quit)
172.17.0.1 - - [13/Aug/2019 07:35:08] "GET / HTTP/1.1" 200 -

And sure enough, I get the "Hello, World!" greeting in my browser.

Fine, but I want to be able to debug. I stop the app in the terminal, and switch over to the Debug tab in VSCode where I need to create a debug configuration.

Let's just go the the VSCode flask tutorial, it should work exactly the same inside or outside the container. I skip everything up to Run the app in the debugger. And there I only concentrate on the the debug configuration.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Flask",
            "type": "python",
            "request": "launch",
            "module": "flask",
            "env": {
                "FLASK_APP": "microblog.py",
            },
            "args": [
                "run",
                "--host", "0.0.0.0",
                "--no-debugger",
                "--no-reload"
            ],
            "console": "integratedTerminal"
        }
    ]
}

I click the debug button, ... and we're in business.

I foresee one issue though, I have set the FLASK_APP environment variable in the terminal, manually. I need a different way to do that. I could set it in the Dockerfile. And I did. And it works.

Then, continuing reading the flask mega tutorial I see that there is a way toe set the environment variable with a helper module: python-dotenv.

I add the .flaskenv file at the root of the directory and edit it with the env variable: FLASK_APP=microblog.py. I open the requirements.txt file and add python-dotenv after the flask module. CMD+SHIFT-P and I rebuild the container image.

A few seconds later, I'm back in the new container. I can see the python-dotenv module in the list of installed modules. Rebuilding the container replaces the "pip install" command I would normally do.

On my main system, I quickly check the images and I can see a new image has been created.

➜ docker image ls
REPOSITORY                                                      TAG                 IMAGE ID            CREATED             SIZE
vsc-flask-9ea4b4f3ab70c4443ee4aa077d0cb6be                      latest              8a4bc0086379        35 seconds ago      941MB
                                                                        3050040fca8f        About an hour ago   941MB

I should only have to go to the VSCode debug tab and start a debug session to get my app running with the debugger. It works.

Continuing to chapter 2 of the flask mega tutorial: templates. We can set a breakpoint and inspect the user variable.

I can use VSCode, with all the fancy extensions I want, with the ability to test and debug within a container that completely abstracts runtime, binaries and libs from my main system. I have a shell to the container. According to changes I want to do, like adding modules or extensions for example, I can rebuild the container from within VSCode.

Brilliant!

Using a docker-compose file is an option as well. This is good because most of the time, we need more than one container for an app.

If we follow the flask mega tutorial to the database chapter, we could use another container and change the setup to use a docker-compose file. I'm game. I'll use postgress.

First we need a docker-compose file.

version: "3.7"
services:

  microblog:
    build:
      context: ..
      dockerfile: .devcontainer/Dockerfile
    user: vscode
    ports:
      - "5000:5000"
    networks:
      - frontend
      - backend
    volumes:
      # Mounts the project folder to '/workspace'. The target path inside the container
      # should match what your application expects. In this case, the compose file is
      # in a sub-folder, so we will mount '..'. We'll then reference this as the
      # workspaceFolder in '.devcontainer/devcontainer.json' so VS Code starts here.
      - ..:/workspace

      # [Optional] If you are using SSH keys w/Git, mount your .ssh folder to
      # /root/.ssh-localhost so we can copy its contents
      - ~/.ssh:/home/vscode/.ssh-localhost:ro
    # Overrides default command so things don't shut down after the process ends.
    command: sleep infinity

  db:
    image: postgres:11
    environment:
      - POSTGRES_PASSWORD=mysecretpassword
    volumes:
      - ../db-data:/var/lib/postgresql/data
    ports: 
      - 5432:5432
    networks:
      - backend

networks:
  frontend:
  backend:

volumes:
  db-data:

We will use a volume map to persists the database data, 2 networks for the frontend and the backend (the DB here), a volume map to get our code into the microblog container at /workspace. We want to expose ports for the app and the database (during the development phase). And we will build the development image based on the Dockerfile. I use a volume map for my ssh keys as well here so I can push to a remote git repository (just remove it if you do not use ssh keys for your git authentication) BTW, you will need to define and install a text editor in the container for git.

Now we need to edit the devcontainer.json file and adjust the settings for the docker-compose file. I left the initial setup commented in there to make the changes more apparent.

// For format details, see https://aka.ms/vscode-remote/devcontainer.json or the definition README at
// https://github.com/microsoft/vscode-dev-containers/tree/master/containers/python-3
{
	"name": "Microblog",
	"context": "..",
	// "dockerFile": "Dockerfile",

	// Uncomment the next line if you want to publish any ports.
	// "appPort": 5000,

	// Uncomment the next line to run commands after the container is created.
	// "postCreateCommand": "python --version",

	// Uncomment the next line to use a non-root user. See https://aka.ms/vscode-remote/containers/non-root-user.
	// "runArgs": [ "-u", "1000" ],

	"dockerComposeFile": [
        "docker-compose.yaml"
    ],
	"service": "microblog",
    "workspaceFolder": "/workspace",
    "shutdownAction": "stopCompose",

	// [Optional] If you are using SSH keys w/Git, copy them and set correct permissions
	"postCreateCommand": "mkdir -p ~/.ssh && cp -r ~/.ssh-localhost/id_rsa.pub ~/.ssh && chmod 700 ~/.ssh && chmod 600 ~/.ssh/*",

	"extensions": [
		"ms-python.python",
		"patbenatar.advanced-new-file",
		"aaron-bond.better-comments"
	],
	"settings": {
		"python.pythonPath": "/usr/local/bin/python",
		"python.linting.pylintEnabled": true,
		"python.linting.pylintPath": "/usr/local/bin/pylint",
		"python.linting.enabled": true
	}
}

I limit myself to 1 docker-compose file for now to keep things simple (no need to extend anything at this stage). The service I want to work in is "microblog" (named in the Dockerfile). I want to work in /workspace where my code will be. When I close the connection to the container, I want docker-compose to stop the containers. You could choose "none" here and the containers would continue to run. I copy an ssh key in the postCreateCommand command (remove it if you don't use ssh keys to authenticate to a git repository). One more change in the .flaskenv where we need to modify the FLASK_APP environment variable.

After all these changes, we can "Reopen in the container" which this time will do the docker-compose up and put us in the microblog container. We can work as we did before.

I deviated from the tutorial a bit by going with Postgres, but it's OK. I will add the extra psycopg2 module.

Note that at this stage, VSCode's intellisense is not going to show completion for flask-sqlachemy (issues 4027 and similar issue for Django). Pylint will as well show warnings and errors. But pylint can be somewhat configured to ignore the errors via this package. This will require a pylintrc file or a configuration change in the .devcontainer.json for the VSCode settings.

		"python.linting.pylintArgs": [
			"--disable=all",
			"--enable=F,E,unreachable,duplicate-key,unnecessary-semicolon,global-variable-not-assigned,unused-variable,binary-op-exception,bad-format-string,anomalous-backslash-in-string,bad-open-mode",
			"--load-plugins", "pylint_flask_sqlalchemy"
		],

With that configuration in place we can avoid most of the linter complaints and get back to coding. I now have 2 containers running: the database and the app container I am working in. After the DB migration, the upgrade worked perfectly.

If you haven't followed along and want to give it a shot, here is the github repository.

I can't wait to see how this evolves. Wouldn't it be great if we could use it directly with kubectl and a Kubernetes cluster?

Remote-container is now part of my standard extensions.

Have a go at it, it's great!