Agenda

  • What is (Static|Dynamic|Gradual) typing?
  • Gradual Typing in Python
    • Function annotations
    • Type hints
    • Type hinting syntax proposal

What is (Static|Dynamic|Gradual) typing?

First things first

  • A compiler converts code in text format to another format suitable for execution
    • it may be a binary with machine code for a certain platform
    • or a bytecode that is interpreted by a virtual machine (CPython does this)
  • A static type checker is a program that given source code it can verify that certain contracts hold true through all the program
    • may be part of the compiler
    • dynamic languages defer those checks to runtime

Static typing

  • The values of a program have type information associated to them (often implicit, inferred)
  • The types of the values of a program are checked at compile time
  • If the type checker accepts a program
    • it is guaranteed to satisfy the properties encoded in its types, be values or operations
    • the compiler is able to make optimizations with type information

Static typing: Languages

  • Haskell
  • Scala
  • Java
  • C
  • C++
  • Rust

Static typing: tradeoffs

  • Catches certain kinds of bugs earlier in a convenient, low-cost way
  • Type information can be used by the compiler for generating more efficient executables
  • Interaction with third-party code is checked, not just your code

  • Type checkers are, generally speaking, able to check only fairly simple properties if your program

Dynamic typing

  • The type checking of a program is deferred to run time
  • Compilers
    • compile programs that aren't type safe
    • don't know about the types of the values in the program
  • Many bugs are discovered when running the program

Dynamic typing: tradeoffs

  • Dynamic typing reduces the friction of type-checking when programming
  • It makes it easy to deal with situations where the type of a value depends on information only available at run time
  • Great for interactive development and exploratory programming
  • Flexibility for adapting to changing requirements

  • There is generally little or no effort put on annotating values with their types in dynamic languages, even though the cost of doing it is low

Dynamic typing: tradeoffs

  • The following example is a valid Python program, although it's obviously ill-typed
def yolo():
    return None + 42

yolo()
# TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Dynamic typing: Languages

  • Python
  • Clojure
  • Racket
  • Lua
  • Ruby
  • PHP
  • JavaScript

Gradual typing

  • Developed by Jeremy Siek & Walid Taha in 2006
  • Values may be either typed at compile-time (static) or at run-time (dynamic), hybrid typing model
  • We choose which parts of a program want to type check: "pay-as-you-go"

Gradual typing: Languages

  • Lua
  • Racket
  • Clojure
  • Groovy
  • TypeScript

Gradual typing: a semi-formal definition

  • Gradual typing defines a relationship between types: consistency
    • similar to subtyping (subclassing)
    • not transitive when the Any type is involved (more on that later)
    • not symmetric
  • We say that a type t1 is consistent with a type t2
  • Given two variables x and y, and given the type of x is consistent with the type of y
    • we can assing x to y in a type-safe manner
    • otherwise we get a type error

Gradual typing: a semi-formal definition

  • The consistency relationship is defined by the following rules
    1. A type t1 is consistent with a type t2 if t1 is a subclass of t2. This relation is not symmetric.
    2. The Any type is consistent with every type, but it is not a subclass of every type.
    3. Every type is a subclass of Any, which makes every type consistent with Any.

Gradual typing: Any

  • Any is at the root of the type graph, like object
  • However, object is not consistent with most types
  • Both Any and object mean "any type allowed" when annotating a function argument
    • but only Any can be passed no matter what type is expected
  • In essence, the Any type never makes a type checker complain

Gradual type checker

  • A type checker that checks for type errors in the statically typed part of a gradually typed program
  • The type checker deals with unannotated variables and function parameters by assigning them the Any type
  • Can perform more sophisticated static analysis than the linters we use today

Gradual typing in Python

Function annotations

  • Introduced by PEP 3107
  • Allow arbitrary expressions as annotations
    • For parameters and return values
    • Optional
    • Insignificant runtime cost
    • Introduced with no standard semantics
      • but BDFL has spoken: Type hints!

Parameters and return value

  • Optional expressions that follow the parameter name
  • Precede the (optional) default value
def say_hello(name : str = "Anon") -> None: # <- return value annotation
    #              ^^^^^   ^
    #              |       `- default value
    #              `- parameter annotation
    print("Hello, {}".format(name))

Positional and keyword arguments

  • Can be annotated just like any other parameter
def say_hello_to_everybody(*names : "An arbitrary number of names"):
    list(map(say_hello, names))

lambda

  • Supporting annotations would require modifying their syntax
  • To avoid further incompatibility, annotations on lambdas were discarded
  • In Python generally def is preferred over lambda

Type hints

Type hints

Goals and non-goals

  • Goals
    • Standard syntax for type annotations
    • Easier static analysis and refactoring for Python code
  • Non-goals
    • Runtime type-checking
    • Performance optimizations based on type hints

Hello, type hints!

  • The type checker can check the annotated parts of our programs
def greet(name : str) -> str:
    return "Hello, {}".format(name)

greet("Ada")
# "Hello, Ada"

greet(0xADA)

Type hinting syntax proposal

Primitive values

  • Builtins are represented with their constructors
int     # unbounded integer
complex # complex number
float   # floating point number
bool    # boolean value
str     # unicode string
bytes   # 8-bit string

Explicit types for variables

  • Variable types are inferred by the initializer, although there are ambiguous cases
l = []
  • In the above example we know l will be a list but the type of its elements is unknown
  • We can type hint variables using comments
l = [] # type: List[int]

Collections

  • The typing module contains generic versions of concrete collection types
from typing import List, Dict, Set, Tuple

List[str]            # list of str objects
Dict[str, int]       # dictionary from str to int
Set[str]             # set of str objects
Tuple[int, int, int] # a 3-tuple of ints
Tuple[int, ...]      # a variable length tuple of ints

List

from typing import List

l = [] # type: List[int]

l = [1, 2, 3]

l = [1, 2, "three"]

Dict

from typing import Dict

d = {} # type: Dict[str, str]

d = { "name": "Ada" }

d = { "name": 0xADA }

Set

from typing import Set

s = set() # type: Set[int]

s = {1, 2, 3}

s = {"foo", "bar", "baz"}

Tuple

from typing import Tuple

t = () # type: Tuple[str, int]

t = ("baz", 42)

t = (42, "foo")
# or
t = ("baz", 42, "foo")

Abstract collections

  • It also defines abstract collection types similar to those found in collections.abc for duck typing
  • Abstract collection types distinguish between mutable and immutable
  • Let's see some examples
from typing import Mapping, MutableMapping, Sequence, MutableSequence

Mapping[str, str]        # a mapping from strings to strings
MutableMapping[str, str] # a mutable mapping from strings to strings

Sequence[int]            # a sequence of integers
MutableSequence[int]     # a mutable sequence of integers

User defined types

  • Every class in the Python program is also a type
  • Subclass' instances are also instances of their base class types
class Foo:
    def bar(self) -> str:
        return "baz"

fooer = Foo()

User defined types

  • Access to properties and methods that don't exist yield a type error
  • Limited to properties and methods that are defined at compile-time
    • some forms of metaprogramming are not supported
  • No more runtime errors for typos!

fooer.baz()
#     ^
#     `- typos in method and property names result in type errors

Union types

  • Python functions often accept values of several different types
  • We are able to express those types as type unions
  • A union represents a set of legal types, Union is a type constructor
from typing import Union

def mul(n : int, m : Union[str, int]):
    return n * m

mul(42, 1)
# 1
mul(42, '*')
# "******************************************"

Union types

  • None as an argument is a special case and is replaced by type(None)
  • Unions of unions are flattened
Union[Union[int, str], float] == Union[int, str, float]
  • Unions of a single argument vanish
Union[str] == str
  • Redundant arguments are skipped
Union[str, str, int] == Union[str, int]
  • The argument order is irrelevant

Union types

  • When two arguments have a subtype relationship, the least derived argument is kept
  • If Any is present, the union becomes an Any

Type variables

  • We can use TypeVar for creating named generic types
    • Supports type constraints
  • Type variables can also be used in function annotations
from typing import TypeVar, Sequence, Iterator

T = TypeVar('T')

def dupe(xs : Sequence[T]) -> Iterator[T]:
    for x in xs:
        yield x
        yield x

Optional

  • Represents the possible absence of value
from typing import Optional, TypeVar, Sequence

T = TypeVar('T')

def first(s : Sequence[T]) -> Optional[T]: # equivalent to Union[T, None]
    return s[0] if s else None

x = first(["foo", "bar", "baz"])
x.lower() # must check for None before using

Generic classes

  • Collection classes are an example of generic classes
  • Generic classes have one or more type parameters, which can be arbitrary types
    • Set[int] and Dict[str, int] are concretions of generic classes

Generic classes example (1/3)

  • Given that we have two arbitrary types A and B
from typing import TypeVar, Generic

A = TypeVar('A')
B = TypeVar('B')

Generic classes example (2/3)

  • We can construct a typed generic pair of values
class Pair(Generic[A, B]):
    def __init__(self, x : A, y : B) -> None:
        self._value = (x, y,)

    @property
    def first(self) -> A:
        return self._value[0]

    @property
    def second(self) -> B:
        return self._value[1]

    def swap(self) -> Pair[B, A]:
        return Pair[B, A](self.second, self.first)

Generic classes example (3/3)

  • In the example below, sp has the type Pair[int, str]
a_pair = Pair[str, int]("yada", 42)
another_pair = p.swap()

a_string = "" # implies that x will be of type string
a_string = another_pair.first

Callables

  • The Callable type includes the type of its arguments and return value
Callable[[int, int], int]
  • Keyword argument annotation isn't supported
    • although annotating only the return type is
Callable[..., int]

Callables example (1/2)

from typing import Callable, Iterable, TypeVar

A = TypeVar('A')
B = TypeVar('B', Iterable[A])

def typed_filter(f : Callable[[A], bool], xs : B) -> B:
    return filter(f, xs)

def is_positive(x : int) -> bool:
    return x > 0

Callables example (2/2)

typed_filter(is_positive, {-3, -2, -1, 0, 1, 2, 3})
# {1, 2, 3}

from operator import add

typed_filter(add, {1, 2, 3})

Wrapping up

Things we didn't cover

  • Annotating third-party code with stub files
  • Type casts
  • Overloading
  • IO types
  • Regex types
  • Silencing the type checker

Conclusions

  • Type annotations convey valuable information and make programs easier to understand and mantain
  • Type checkers will be able to detect errors before running your program, thus aiding during development
  • Tooling around Python can get better thanks to type hints
    • documentation
    • tests generation
  • Type hints are arguably Pythonic and not very invasive
    • "Explicit is better than implicit"

Further information

Questions?

Thank you ˊ~ω~ˋ