# Python

# Version

list all versions of python (system wide)

ls -ls /usr/bin/python*

# Best practises

# Execute only if directly called

When directly called, the internal virtual python __name__ will be __main__.
This way, in some case you will be able to import your script as library (functions) and NOT execute code 😉

def main():
  # statement

if __name__ == '__main__':
  exitcode = 0
  try:
      main()
  except KeyboardInterrupt:
      die(3, 'Exiting on user request')
  sys.exit(exitcode)

1
2
3
4
5
6
7
8
9
10

Exampleopen in new window

# Write errors to stderr

By default python write its errors to stdout. Which is not a wanted behaviour when executed on Linux systems.

sys.stdout.write('Dive in')
sys.stderr.write('Dive in')

1
2

# Symbol

Symbol	Meaning
()	tuple
[]	list, array, tableau
{}	dictionary, hashtable, clé-valeur

# JSON, YAML, dict

There is a bijectionopen in new window between JSON and dict python
There are 2 isomorph objects

JSON <--> dict Python

exactly like YAML and JSON (and dict)

YAML <--> JSON <--> dict Python

{
  'job': {
       'id': job,
       'ref': job_ref,
       'created_at': job_created_at,
       'finished_at': job_finished_at,
       'started_at': job_started_at,
       'status': job_status
   },
  'user': {
      'username': user_username,
      'state': user_state
  }
}

--> 

job:
    id: {{ job }}
    ref: {{ job_ref }}
    created_at: {{ job_created_at }}
    finished_at: {{ job_finished_at }}
    started_at: {{ job_started_at }}
    status: {{ job_status }}
user:
    username: {{ user_username }}
    state: {{ user_state }}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

Yaml are easier to read for a human being but JSON has the same data structure

tutorialopen in new window

# Handling Exceptions

Official docopen in new window

It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using Control-C or whatever the operating system supports); note that a user-generated interruption is signalled by raising the KeyboardInterruptopen in new window exception.

while True:
    try:
        x = int(raw_input("Please enter a number: "))
        break
    except ValueError:
        print "Oops!  That was no valid number.  Try again..."

1
2
3
4
5
6

The try statement works as follows.

First, the try clause (the statement(s) between the try and except keywords) is executed.
If no exception occurs, the except clause is skipped and execution of the try statement is finished.
If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.

# Functions (Well known)

# format

'{} {}'.format('one', 'two')
'{} {}'.format(1, 2)


resource = ('api/v4/projects/{0}/jobs/{1}/artifacts/'
            .format(project, job))

1
2
3
4
5
6

sourceopen in new window

# findall (Regular expressions)

findall() is probably the single most powerful function in the re module. Above we used re.search() to find the first match for a pattern. findall() finds all the matches and returns them as a list of strings, with each string representing one match.

## Suppose we have a text with many email addresses
str = 'purple alice@google.com, blah monkey bob@abc.com blah dishwasher'

## Here re.findall() returns a list of all the found email strings
emails = re.findall(r'[\w\.-]+@[\w\.-]+', str) ## ['alice@google.com', 'bob@abc.com']
for email in emails:
  # do something with each found email string
  print email

1
2
3
4
5
6
7
8

Sourceopen in new window

# Built-in

https://docs.python.org/3/library/functions.html#built-in-functions

For those functions you don't need to install them on your filesystem, there are already existing 😃

## Variables

# Get data from dictionnary

That are the same results

value = mydict['key']

syslog_java_application.get('name')
syslog_java_application['name']

1
2
3
4

# Type inference

All the following variable definitions return False in if statement

{}, [], "", '', False

# Build a package

# Directory structure

Create python package (to be downloaded in site-packages local dir)
Make the following directory structure in your local dev machine


-----------------------------
some_root_dir/
|-- README
|-- setup.py
|-- an_example_pypi_project
|   |-- __init__.py
|   |-- useful_1.py
|   |-- useful_2.py
|-- tests
|-- |-- __init__.py
|-- |-- runall.py
|-- |-- test0.py

----------------------------

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# setup.py

setup.py content (in the dir)

Utility function to read the README file.
Used for the long_description.
It's nice, because now

we have a top level README file
it's easier to type in the README file than to put a raw string in below ...

import os
from setuptools import setup

def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname)).read()

setup(
    name = "an_example_pypi_project",
    version = "0.0.4",
    author = "Andrew Carter",
    author_email = "andrewjcarter@gmail.com",
    description = ("An demonstration of how to create, document, and publish "
               "to the cheese shop a5 pypi.org."),
    license = "BSD",
    keywords = "example documentation tutorial",
    url = "http://packages.python.org/an_example_pypi_project",
    packages=['an_example_pypi_project', 'tests'],
    long_description=read('README'),
    classifiers=[
        "Development Status :: 3 - Alpha",
        "Topic :: Utilities",
        "License :: OSI Approved :: BSD License",
    ],
)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# wheel

create ".whl" with wheel
within the root directory
Your package have been built in /dist/$(package-name)-$(version)-$(py2-compatible)-$(py3-compatible)-any.whl

python setup.py sdist bdist_wheel
example : ./dist/dns_admin-1.0.0-py2-none-any.whl

1
2

# Package installer (Pip)

pip is the package installer for Python. You can use pip to install packages from the Python Package Index and other indexes.

install pip3

apt-get install build-essential python3-dev python3-pip

install a package

pip install virtualenv
pip --proxy http://10.10.10.10:5000 install docker

1
2

install without TLS verif (not recommended)

pip install --trusted-host pypi.python.org \
            --trusted-host github.com \
            https://github.com/Exodus-Privacy/exodus-core/releases/download/v1.0.13/exodus_core-1.0.13.tar.gz

1
2
3

Show information about one or more installed packages

pip3 show $package_name
pip3 show virtualenv

1
2

print all installed package (depends on your environement venv or system-wide)

pip3 freeze

install from local sources (setup.py required)

python setup.py install --record files.txt

print dependencies tree of a specified package

pipdeptree -p uwsgi

global site-packages ("dist-packages") directories

python3 -m site

more concise list

python3 -c "import site; print(site.getsitepackages())"

Note: With virtualenvs getsitepackages is not available, sys.path from above will list the virtualenv s site-packages directory correctly, though.

# Virtual environment

# dependencies

apt install python-pip python3-pip
pip install pipenv

1
2

# Pipenv

Create a new project using Python 3.7, specifically:

pipenv --python 3.7

Remove project virtualenv (inferred from current directory):

pipenv --rm

Install all dependencies for a project (including dev):

pipenv install --dev

Create a lockfile containing pre-releases:

pipenv lock --pre

Show a graph of your installed dependencies:

pipenv graph

Check your installed dependencies for security vulnerabilities:

pipenv check

Install a local setup.py into your virtual environment/Pipfile:

pipenv install -e .

Use a lower-level pip command:

pipenv run pip freeze

Commands

check      Checks for security vulnerabilities and against
           PEP 508 markers provided in Pipfile.
clean      Uninstalls all packages not specified in
           Pipfile.lock.
graph      Displays currently-installed dependency graph
           information.
install    Installs provided packages and adds them to
           Pipfile, or (if no packages are given),
           installs all packages from Pipfile.
lock       Generates Pipfile.lock.
open       View a given module in your editor.
run        Spawns a command installed into the virtualenv.
shell      Spawns a shell within the virtualenv.
sync       Installs all packages specified in
           Pipfile.lock.
uninstall  Un-installs a provided package and removes it
           from Pipfile.
update     Runs lock, then sync.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# SSL / TLS

check the protocols supported by your Python version

vim /tmp/testPythonProtocols.py

import ssl;
for i in dir(ssl): 
  if i.startswith("PROTOCOL"):
    print(i)

1
2
3
4

/tmp/testPythonProtocols.py