Open-source libraries used

1 libraries are bundled into this tool's code.

Python Cheat Sheet

A concise reference for modern Python 3 (3.11+) syntax, built-in types, and the most-used standard library modules — covers about 80% of everyday scenarios.

Py

Python Python 3.11 (LTS-style)

CPython · Multi-paradigm (OOP · functional · procedural) · Dynamic · strong typing · duck typing

Recommended Learning Path

Start by running python3 and using the REPL -> master variables, built-in types, and control flow -> go deeper into functions, classes, and exceptions -> use the standard library for file and network I/O -> understand references and mutability (shallow vs deep copy) -> pick up asyncio, processes, and regex as needed -> finally learn venv/pip/pyproject for engineering workflows. Revisit the FAQ section whenever you hit a pitfall.

1.Hello World & Build Environment

Run Python, the interactive REPL, virtual environments, and package management.

Minimal Program

print writes to the console. Python organizes blocks with indentation — no braces or semicolons.

1
2
3
4
5
6
7
8
# hello.py
print('Hello, world!')
# Run: python3 hello.py
# Or interactively: launch python3 to enter the REPL and type directly
print(f'1 + 1 = {1 + 1}')
# Print to stderr:
import sys
print('error', file=sys.stderr)

Ways to Run

python3 runs a script, -c executes a string, -m runs a module. The interactive REPL evaluates immediately.

1
2
3
4
5
6
7
8
9
10
11
# Run a script:
# python3 hello.py
# Execute a string:
# python3 -c "print('hi')"
# Run a module (with __main__):
# python3 -m http.server 8000
# Check the version:
# python3 --version
# Enter the REPL:
# python3
# At the >>> prompt, type code and press Enter to run it

Virtual Environments

venv isolates project dependencies. After activation, pip installs affect only that environment. Always use a virtual environment per project.

1
2
3
4
5
6
7
8
9
10
11
12
# Create a virtual environment:
python3 -m venv .venv
# Activate (Windows):
# .venv\Scripts\activate
# Activate (macOS/Linux):
# source .venv/bin/activate
# Deactivate: deactivate
# Once activated:
# python -m pip install requests
# Check the interpreter:
# which python # Linux/macOS
# The venv name appears as a prompt prefix

pip Package Management

pip installs, uninstalls, and freezes dependencies. Pin with ==, range with >=,<y.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Install:
pip install requests
# Pin a version:
pip install 'requests==2.31.0'
# Version range:
pip install 'requests>=2,<3'
# Upgrade: pip install -U requests
# Uninstall: pip uninstall requests
# Export dependencies:
pip freeze > requirements.txt
# Install from a file:
pip install -r requirements.txt
# List: pip list
# Upgrade pip: python3 -m pip install -U pip

pyproject.toml

The modern project config: dependencies, build backend, and tool config all live in pyproject.toml.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# pyproject.toml (PEP 621):
# [project]
# name = "my-app"
# version = "0.1.0"
# requires-python = ">=3.11"
# dependencies = [
# "requests>=2.31",
# "httpx>=0.24",
# ]
#
# [build-system]
# requires = ["setuptools>=68"]
# build-backend = "setuptools.build_meta"
#
# [tool.ruff]
# line-length = 88
# Install dependencies: pip install -e .

Shebang Scripts

#!/usr/bin/env python3 makes a script executable. On POSIX, chmod +x it and run it directly with ./script.py.

1
2
3
4
5
6
7
8
9
10
11
#!/usr/bin/env python3
# script.py
# Grant execute permission:
# chmod +x script.py
# Run it directly:
# ./script.py
print('可执行脚本')
# No need to hardcode a python path:
# env locates python3 for you
# On Windows a double-clicked .py uses the associated interpreter
# Without a shebang, run it as python3 script.py

REPL Interaction

Run python3 to enter the REPL for interactive debugging. _ holds the last result. Call exit() to leave.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
>>> 1 + 2
3
>>> _
3
>>> def f(x): return x * 2
>>> f(21)
42
# Exit the REPL:
# exit() or Ctrl+D
# Built-in help:
>>> help(str)
>>> help('str'.upper)
# Inspect object attributes:
>>> dir('hello')
# History: browse with the arrow keys

__main__ & Modules

if __name__ == '__main__' distinguishes running-as-a-script from being imported. This is the standard idiom.

1
2
3
4
5
6
7
8
9
10
11
12
# greetings.py
def hello(name='world'):
return f'Hello, {name}!'
if __name__ == '__main__':
# Runs only when executed directly
print(hello())
# Run directly: python3 greetings.py
# Import and use:
# from greetings import hello
# hello('Nick')
# __name__ is '__main__' when run directly, the module name when imported

2.Variables & Type Annotations

Dynamic typing, type annotations, multiple assignment, scope, and the walrus operator.

Assignment & Inference

No variable declarations — assignment creates the binding. Type is inferred dynamically and can change at any time.

1
2
3
4
5
6
7
8
9
10
11
x = 42 # inferred int
y = 3.14 # float
name = 'Rex' # str
ok = True # bool
empty = None # NoneType
# The type can change:
x = '现在是字符串'
# Several variables at once:
a = b = c = 0
# A variable is a name, not a box:
# it binds to an object; reassigning rebinds it

Type Annotations

PEP 484 style annotations with a colon to mark variable types. Annotations don't affect runtime — they're hints and enable checking.

1
2
3
4
5
6
7
8
9
10
11
12
count: int = 0
items: list[str] = []
mapping: dict[str, int] = {}
# PEP 604 union syntax (3.10+):
optional: int | None = None
# Function annotations:
def add(a: int, b: int) -> int:
return a + b
# Annotations are not enforced:
add('a', 'b') # no error at runtime
# Checkers: mypy / pyright
# Generics: list[int], not list[int, ...]

Multiple Assignment

Swap with a, b = b, a. Extended unpacking uses *rest. Counts must match.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
a, b, c = 1, 2, 3
# Swap:
a, b = b, a
# Extended unpacking:
first, *rest = [1, 2, 3, 4]
# first=1, rest=[2,3,4]
*head, last = [1, 2, 3, 4]
# head=[1,2,3], last=4
# Unpack a tuple:
x, y = (10, 20)
# Unpack dict keys:
keys = 'ab'
d = dict(zip(keys, [1, 2]))
# A count mismatch raises ValueError

Constant Conventions

Python has no constant keyword. ALL_CAPS is the convention for module-level constants.

1
2
3
4
5
6
7
8
9
10
11
PI = 3.14159
MAX_RETRIES = 3
DATABASE_URL = 'postgres://...'
# Convention only, still mutable:
# PI = 99 allowed, but do not do it
# Why:
# values that never change stand out at a glance
# configuration stays in one place
# Use Enum instead of magic numbers:
# from enum import Enum
# class Color(Enum): RED = 1

LEGB Scope

Lookup order: Local -> Enclosing -> Global -> Built-in. Assigning in a function requires global or nonlocal.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
counter = 0
def inc():
global counter # read/write module level
counter += 1
def outer():
x = 1
def inner():
nonlocal x # read/write the enclosing function's variable
x += 1
inner()
print(x) # 2
# Reading an outer variable needs no nonlocal
# Globals are defined at the top of the module
# Built-ins (len/print) live in the Built-in scope

Walrus Operator

The := operator assigns inside an expression (Python 3.8+). Avoids repeating a computation.

1
2
3
4
5
6
7
8
9
10
11
12
# Classic form:
line = input('> ')
while line != 'quit':
print(line)
line = input('> ')
# Walrus form:
while (line := input('> ')) != 'quit':
print(line)
# Reuse it inside a comprehension:
# [y for x in data if (y := f(x)) > 0]
# Mind the parentheses: an assignment expression must be wrapped
# Use case: initialise and test in a single condition

None & Booleans

None means "no value". Test truthiness with if x, not if x == True. Empty containers are falsy.

1
2
3
4
5
6
7
8
9
10
11
12
13
value = None
if value is None: # test None with is
print('无值')
# Truthiness:
if [1, 2]: pass # a non-empty list is true
if []: pass # an empty list is false
# Falsy: False/0/''/[]/{}/(,)/None
if not value: # when value is falsy
pass
# Ternary:
status = 'ok' if value else 'empty'
# Null-coalescing alternative:
name = value or '默认值'

del & Garbage Collection

del removes a name or a container element. The name becomes unbound and the object's reference count drops by one.

1
2
3
4
5
6
7
8
9
10
11
x = 42
del x # delete the name
# accessing x again raises NameError
items = [1, 2, 3]
del items[0] # delete an element -> [2, 3]
d = {'a': 1}
del d['a'] # delete a key
# pop deletes a dict key and takes a default:
val = d.pop('a', None)
# Pair it with try to catch KeyError
# del on a local often releases a large object early

3.Data Types

Built-in types: numeric, boolean, sequence, mapping, and set, plus advanced type annotations.

Numeric Types

int has arbitrary precision, float is IEEE-754, complex for complex numbers. bool is a subclass of int. Division has two semantics.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
n = 10 # int
g = 3.14 # float
c = 1 + 2j # complex
big = 2 ** 100 # int, arbitrary precision
# Division:
5 / 2 # 2.5 (true division)
5 // 2 # 2 (floor division)
5 % 2 # 1 (remainder)
# Bases:
0b1010 # binary 10
0o12 # octal 10
0xff # hex 255
# Conversion:
int('42'); float(1); bool(0)

Booleans & None

True/False are capitalized. None is a singleton. and/or short-circuit and return the deciding operand.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
t, f = True, False
# Short-circuit logic:
print(0 and 'x') # 0 (short-circuits to the left value)
print('' or '默认') # '默认'
print(None or []) # []
# Ternary:
age = 17
msg = '成年' if age >= 18 else '未成年'
# Chained comparison:
if 0 < x < 10: pass
# Testing None:
if x is None: pass
# bool() conversion:
bool([]) # False
bool([1]) # True

Sequence Types

str/list/tuple are all sequences: indexed, sliced, and iterable. str/tuple are immutable.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
s = 'abc'
# Indexing:
s[0] # 'a'
s[-1] # 'c'
# Slicing:
s[1:] # 'bc'
s[::-1] # 'cba' reversed
# Common to all sequences:
len(s), min([3,1]), max([3,1])
sum([1,2,3])
list('abc') # ['a','b','c']
# Membership:
'a' in s # True
# Immutable sequences cannot be changed:
# s[0] = 'x' error
# An out-of-range index raises IndexError

Mapping dict

dict stores key/value pairs and preserves insertion order. Keys must be hashable. .get safely fetches a value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
d = {'a': 1, 'b': 2}
# Lookup:
d['a'] # 1, a missing key raises KeyError
d.get('c', 0) # 0, returns the default when missing
# Add / update:
d['c'] = 3
d.setdefault('x', 0)
# Delete:
d.pop('a', None)
# Merge:
d.update({'y': 9})
# Views:
d.keys(); d.values(); d.items()
# Dict comprehension:
{n: n**2 for n in range(3)}
# Keys must be hashable: numbers/strings/tuples work

Set Types

set is an unordered, deduplicating collection; frozenset is its immutable variant. Supports union/intersection/difference.

1
2
3
4
5
6
7
8
9
10
11
12
13
s = {1, 2, 3}
s.add(4)
s.remove(2) # a missing key raises KeyError
s.discard(99) # a missing key is ignored
# Deduplicate:
list(set([1, 1, 2])) # [1, 2]
# Operations:
{1, 2} | {2, 3} # union {1,2,3}
{1, 2} & {2, 3} # intersection {2}
{1, 2} - {2} # difference {1}
# Immutable set:
frozenset([1, 2])
# Use set() for an empty set; {} is an empty dict

Bytes Types

bytes is immutable, bytearray is mutable. Convert between text and bytes with encode/decode.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
b = b'hello' # bytes
ba = bytearray(b'abc')
ba[0] = 120 # mutable
# Encoding:
text = '你好'.encode('utf-8')
# b'\xe4\xbd\xa0\xe5\xa5\xbd'
text.decode('utf-8') # '你好'
# Decoding errors:
text.decode('utf-8', errors='ignore')
# Converting between str and bytes always needs an encoding
# Binary data:
import struct
struct.pack('>i', 42)
# Byte order: > big-endian, < little-endian

Advanced Type Annotations

The typing module: Optional, Union, TypeVar, generic containers. Python 3.10+ supports the | syntax.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from typing import Optional, Union, TypeVar, Callable
# Optional:
x: Optional[int] = None # same as int | None
# Union:
y: Union[int, str] # same as int | str
# Type variable:
T = TypeVar('T')
def first(items: list[T]) -> T:
return items[0]
# Callable:
def apply(f: Callable[[int], int], n: int) -> int:
return f(n)
# Type alias (3.12+):
type Vector = list[float]
# Checkers: mypy / pyright

Duck Typing

"If it walks like a duck…": care about behavior, not type. typing.Protocol defines structural interfaces.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Duck typing: any object with len will do
def size(obj):
return len(obj)
size([1, 2]) # 2
size('abc') # 3
# Protocols (typing.Protocol):
from typing import Protocol
class Sizeable(Protocol):
def __len__(self) -> int: ...
def total_len(objs: list[Sizeable]) -> int:
return sum(len(o) for o in objs)
# Structural subtyping: a matching shape is enough
# No explicit inheritance required

4.References & Mutability

Python's object reference semantics: mutable/immutable, shallow vs deep copy, and shared reference pitfalls.

Reference Semantics

Variables bind to objects, not boxes. Multiple names can point to the same object. Assignment rebinds, it does not copy.

1
2
3
4
5
6
7
8
9
10
11
12
13
a = [1, 2, 3]
b = a # b and a point at the same list
b.append(4)
print(a) # [1, 2, 3, 4]
# Rebinding does not touch the original object:
a = [9]
print(b) # still [1,2,3,4]
# Immutable objects such as ints:
x = 10
y = x
x = 20
print(y) # 10, the value did not change
# Key idea: separate mutating an object from rebinding a name

Mutable vs Immutable

Mutable types can be changed in place: list/dict/set. Immutable: int/str/tuple/frozenset.

1
2
3
4
5
6
7
8
9
10
11
12
13
# Mutable: modified in place
lst = [1, 2]
lst.append(3) # the same object changes
# Immutable: operations build a new object
s = 'ab'
s2 = s.upper() # a new string
# s is unchanged
# Immutable values can be dict keys:
d = {('a', 1): 'x'}
# A list cannot be a key:
# {[1]: 'x'} error
# A tuple holding mutable items is unhashable too
# Immutable objects are safe to share

Shallow Copy

copy.copy creates a shallow copy: a new outer container, but inner elements are still shared. Slicing a list is also shallow.

1
2
3
4
5
6
7
8
9
10
11
12
13
import copy
original = [[1, 2], [3, 4]]
shallow = copy.copy(original)
# The outer object differs:
shallow is original # False
# The inner objects are shared:
shallow[0] is original[0] # True
# A list slice behaves the same:
sub = original[:]
# Changing an inner item affects both:
shallow[0].append(99)
print(original) # [[1,2,99], [3,4]]
# A shallow copy suits one-level structures

Deep Copy

copy.deepcopy copies recursively. Edits to nested mutable structures don't leak across the copy. Watch the cost.

1
2
3
4
5
6
7
8
9
10
11
12
import copy
original = [[1, 2], [3, 4]]
deep = copy.deepcopy(original)
deep[0].append(99)
print(original) # [[1, 2], [3, 4]]
print(deep) # [[1, 2, 99], [3, 4]]
# A deep copy recurses:
# every nested object is brand new
# Cost grows with depth
# Cyclic references are handled
# Self-referencing objects: copy.deepcopy(x)
# Use with care on big data; rethink the structure

Shared Reference Pitfalls

When references share a mutable object, a mutation is visible everywhere. Argument passing is also by reference.

1
2
3
4
5
6
7
8
9
10
11
# Mutating a list passed into a function:
def add_item(items):
items.append('x') # affects the caller!
my_list = []
add_item(my_list)
print(my_list) # ['x']
# Pass a copy to avoid in-place changes:
add_item(my_list.copy())
# Or return a new list from the function
# See the FAQ for the default-argument trap
# Ask: does the function promise a read-only parameter?

is vs ==

== compares values; is compares identity (same object). Use is None to compare with None.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
a = [1, 2]
b = [1, 2]
a == b # True, equal values
a is b # False, different objects
# Small-int interning:
x = 256
y = 256
x is y # True (-5..256 are cached)
# Large ints:
x, y = 10**6, 10**6
x is y # not guaranteed, do not rely on it
# Correct usage:
if x is None: pass
# Compare values with ==:
if x == 42: pass

Default Argument Pitfall

A mutable default is evaluated once at definition and shared across calls. Use None as a sentinel and create it inside the function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# BAD: a shared default list
def add(x, acc=[]):
acc.append(x)
return acc
add(1) # [1]
add(2) # [1, 2] !
# GOOD: None + create it inside the function
def add(x, acc=None):
if acc is None:
acc = []
acc.append(x)
return acc
# A default value is evaluated once:
# created at definition time, reused ever after

Interning & Caching

Small ints and short strings are interned for memory savings. Don't rely on is to compare contents.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Small-int cache range:
# -5 to 256 are singletons
x, y = 256, 256
x is y # True
# Outside that range:
a, b = 257, 257
a is b # unspecified (implementation detail)
# String interning:
'hello' is 'hello' # True (compile-time constant)
# Built at runtime it differs:
'he' + 'llo' is 'hello' # may be False
# Conclusion:
# always compare content with ==
# use is only for None and singletons

5.Control Flow

Conditional branches, loops, pattern matching, and comprehensions.

if / elif / else

Indentation delimits blocks. Use elif for multiple conditions. The condition can be any boolean expression.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
score = 85
if score >= 90:
grade = 'A'
elif score >= 80:
grade = 'B'
elif score >= 70:
grade = 'C'
else:
grade = 'D'
# One-liner (not for complex logic):
if x > 0: print('正数')
# Multiple conditions:
if a and not b: pass
if x in (1, 2, 3): pass

Pattern Matching

match-case is structured pattern matching (Python 3.10+). Match and destructure by shape.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def handle(command):
match command.split():
case ['quit']:
print('退出')
case ['open', path]:
print(f'打开 {path}')
case ['open', path, *rest]:
print(f'打开 {path} 附加 {rest}')
case _:
print('未知命令')
# Guards:
def f(point):
match point:
case (0, 0): print('原点')
case (x, y) if x == y: print('对角')
case (x, y): print(f'({x}, {y})')
# The _ branch is the catch-all

for Loop

for iterates any iterable. enumerate adds the index, zip runs in parallel, dict.items() yields key/value pairs.

1
2
3
4
5
6
7
8
9
10
11
12
for i in range(5):
print(i) # 0..4
for idx, item in enumerate(['a', 'b']):
print(idx, item)
for k, v in {'a': 1}.items():
print(k, v)
for a, b in zip([1, 2], ['x', 'y']):
print(a, b) # (1,'x') (2,'y')
# range step:
range(0, 10, 2) # 0 2 4 6 8
# Reverse: reversed(lst)
# Reverse with index: range(len(x)-1, -1, -1)

while Loop

while loops while the condition is true. Use break to exit early — and avoid infinite loops.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
n = 0
while n < 5:
print(n)
n += 1
# Infinite loop + break:
while True:
line = input('> ')
if line == 'quit':
break
print(line)
# while-else:
# else runs when the loop ends without break
attempts = 0
while attempts < 3:
if try_connect():
break
attempts += 1
else:
print('连接失败')

break & continue

break exits the whole loop; continue skips to the next iteration. The else clause runs when no break occurred.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
for i in range(10):
if i % 2 == 0:
continue # skip even numbers
if i > 7:
break # stop early
print(i) # 1 3 5 7
# for-else:
for n in range(2, 10):
for d in range(2, n):
if n % d == 0:
break
else:
print(n, '是质数')
# else runs when the loop finishes normally
# Note: the else belongs to for, not to if

Comprehensions

List/dict/set comprehensions build collections in a single line — filter with conditions, transform with expressions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# List comprehension:
squares = [n ** 2 for n in range(5)]
# [0, 1, 4, 9, 16]
# With a condition:
evens = [n for n in range(10) if n % 2 == 0]
# Dict comprehension:
{n: n * 2 for n in range(3)}
# {0:0, 1:2, 2:4}
# Set comprehension:
{n % 3 for n in range(10)}
# Nested:
flat = [y for xs in matrix for y in xs]
# Generator expression (lazy):
total = sum(n * n for n in range(100))

Truthiness

Use if x for truthiness. 0, '', [], and None are falsy. Avoid comparing explicitly to True/False.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Testing empty containers:
items = []
if not items: # GOOD: empty is falsy
print('空列表')
# BAD: do not write this
if len(items) == 0: pass
# Testing None:
if value is not None:
print(value)
# Testing numbers:
if count: pass # 0 is falsy
# Strings:
if name: pass # '' is falsy
# Explicit bool:
if bool(x): pass
# Priority: readability first, keep conditions short

Iteration Idioms

itertools provides efficient iteration primitives: groupby, chain, product, and more.

1
2
3
4
5
6
7
8
9
10
11
12
from itertools import chain, groupby, product
# Concatenate:
list(chain([1, 2], [3, 4])) # [1,2,3,4]
# Cartesian product:
list(product('ab', [1, 2]))
# [('a',1),('a',2),('b',1),('b',2)]
# Group (input must be sorted):
rows = sorted(rows, key=lambda r: r['type'])
for key, group in groupby(rows, key=lambda r: r['type']):
print(key, list(group))
# Infinite iterator: itertools.count()
# Permutations/combinations: itertools.permutations / combinations

6.Functions

Function definitions, parameter passing, decorators, generators, and closures.

Function Definition

def defines a function with an indented body. return yields a value; omitting it returns None.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def greet(name):
return f'Hello, {name}!'
print(greet('Nick'))
# No return value:
def log(msg):
print(msg) # returns None
# Multiple return values (a tuple):
def min_max(nums):
return min(nums), max(nums)
lo, hi = min_max([3, 1, 4])
# Type annotations:
def add(a: int, b: int) -> int:
return a + b
# Docstring:
def f():
"""Describe what it does"""
pass

Parameter Types

Positional, keyword, and default arguments. A call can mix positional and keyword arguments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def show(a, b, c):
print(a, b, c)
# Positional call:
show(1, 2, 3)
# Keyword call:
show(c=3, a=1, b=2)
# Mixed (positional first):
show(1, c=3, b=2)
# Default arguments:
def greet(name, greeting='Hi'):
return f'{greeting}, {name}!'
# Defaults must come last:
# def f(a=1, b) error
# Keyword-only parameters:
def f(*, strict=False): pass
# Positional-only parameters (3.8+):
def f(a, /): pass

*args & **kwargs

*args collects extra positional args into a tuple; **kwargs collects extra keyword args into a dict.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def log_all(*args, **kwargs):
print('位置:', args)
print('关键字:', kwargs)
log_all(1, 2, x=3, y=4)
# Forwarding a call:
def wrapper(*args, **kwargs):
return original(*args, **kwargs)
# Unpacking into a call:
items = [1, 2, 3]
print(*items) # 1 2 3
config = {'a': 1}
f(**config)
# Order: positional -> *args -> keyword -> **kwargs
# Do not overuse; prefer an object when there are many parameters

Lambda

lambda is a single-expression anonymous function, often used as a key for sort/map/filter.

1
2
3
4
5
6
7
8
9
10
11
square = lambda x: x * x
square(5) # 25
# Sort key:
users.sort(key=lambda u: u['age'])
# Higher-order functions:
list(map(lambda x: x * 2, [1, 2]))
list(filter(lambda x: x > 1, [1, 2]))
# Immediately invoked:
(lambda x: x + 1)(41) # 42
# Use def for complex logic; keep lambda to simple expressions
# A lambda cannot contain statements (if/return)

Closures

An inner function captures variables from the enclosing scope. Returning such a function is the closure pattern used in factories.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def make_multiplier(n):
def multiply(x):
return x * n # captures n
return multiply
double = make_multiplier(2)
triple = make_multiplier(3)
double(5) # 10
triple(5) # 15
# A closure that modifies an outer variable:
def counter():
count = 0
def inc():
nonlocal count
count += 1
return count
return inc
# Each closure keeps its own n/count

Decorators

@decorator wraps a function to enhance its behavior without changing its source — useful for timing and logging.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import functools, time
def timing(func):
@functools.wraps(func) # keep the metadata
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
print(f'{func.__name__}: {time.perf_counter()-start:.4f}s')
return result
return wrapper
@timing
def slow():
time.sleep(0.1)
# Decorator with arguments:
def repeat(times):
def deco(func):
@functools.wraps(func)
def wrapper(*a, **kw):
for _ in range(times): func(*a, **kw)
return wrapper
return deco
# Built-in decorators: @staticmethod/@classmethod/@property

Generators

yield produces values lazily. Generators stream values one at a time — memory-friendly for large sequences.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
def count_up(n):
i = 0
while i < n:
yield i # pause and hand back a value
i += 1
for x in count_up(3):
print(x) # 0 1 2
# Generator expression:
squares = (n * n for n in range(3))
# Infinite sequence:
def fib():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
# Lazy: not everything is held in memory
# Single-use: rebuild it once exhausted
# Pipelines: chain generators to process a stream

Functional Tools

map/filter/reduce/sorted for functional style. functools adds partial and other tools.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
from functools import partial, reduce
# map transforms:
doubled = list(map(lambda x: x * 2, [1, 2, 3]))
# filter selects:
evens = list(filter(lambda x: x % 2 == 0, range(6)))
# reduce aggregates:
total = reduce(lambda a, b: a + b, [1, 2, 3])
# Partial application: pin an argument
def power(base, exp): return base ** exp
square = partial(power, exp=2)
square(5) # 25
# Generators work with map/filter:
list(map(str, range(3))) # ['0','1','2']
# Modern style prefers comprehensions for readability

Recursion

A function that calls itself — always with a base case. CPython caps depth near 1000 (tweakable via sys.setrecursionlimit).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def factorial(n):
if n <= 1:
return 1
return n * factorial(n - 1)
factorial(5) # 120
# Recursion depth limit:
import sys
sys.getrecursionlimit() # 1000 by default
# Deep recursion: use iteration or an explicit stack
def fib_iter(n):
a, b = 0, 1
for _ in range(n):
a, b = b, a + b
return a
# Tree traversal / divide and conquer suit recursion
# Memoisation: @functools.lru_cache

7.Strings

Immutable str, f-string formatting, common methods, and encoding.

String Basics

Strings are created with single, double, or triple quotes. Escape sequences apply. str is immutable — operations return new strings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
s1 = '单引号'
s2 = "双引号"
s3 = '''多行
字符串'''
# Raw string (paths/regex):
r'C:\\path'
# Escapes:
'\n' # newline
'\t' # tab
'\\' # backslash
# Strings are sequences:
s = 'hello'
s[0] # 'h'
s[1:4] # 'ell'
# Immutable:
# s[0] = 'H' error
# Concatenation:
first + last
' '.join(['a', 'b'])

Common Methods

upper/lower for case, strip for whitespace, split/join for splitting and joining, startswith for prefix checks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
s = ' Hello World '
s.upper() # ' HELLO WORLD '
s.lower()
s.strip() # strip whitespace
s.startswith('He') # True
s.endswith('ld') # True
s.replace('World', 'Python')
# Search:
s.find('World') # index or -1
s.index('World') # ValueError if not found
s.count('l')
# Split / join:
'1,2,3'.split(',') # ['1','2','3']
'-'.join(['a', 'b']) # 'a-b'
# Tests:
'abc'.isalpha(); '123'.isdigit()

f-string Formatting

f"{var}" interpolates expressions. Specs control alignment, precision, and thousands separators. Python 3.12+ allows reusing quotes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
name = 'Nick'
age = 30
f'{name} 今年 {age} 岁'
# Expressions:
f'{age + 1}'
# Alignment width:
f'{name:>10}' # right-align, width 10
f'{name:<10}' # left-align
# Number formatting:
f'{3.14159:.2f}' # '3.14'
f'{1000000:,}' # '1,000,000'
f'{0.5:.0%}' # '50%'
# Base:
f'{255:x}' # 'ff'
# Date:
import datetime
f'{datetime.date.today():%Y-%m-%d}'
# Dict lookup: f'{d["key"]}'

format & % Formatting

str.format uses positional/named placeholders. The % operator is the legacy style. Prefer f-strings for new code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# format method:
'{0} {1}'.format('a', 'b')
'{name} 今年 {age}'.format(name='Nick', age=30)
# Alignment:
'{:>10}'.format('hi') # right-align
'{:.2f}'.format(3.14159)
# Old-style % formatting:
'%s 今年 %d' % ('Nick', 30)
'%.2f' % 3.14159
'%x' % 255
# Three styles compared:
# f-string: f'{x:.2f}' (3.6+, recommended)
# format: '{:.2f}'.format(x)
# Old-style: '%.2f' % x
# Use f-string in new code

Split & Join

split on a separator, rsplit from the right, partition into a 3-tuple, join for efficient concatenation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
'a,b,c'.split(',') # ['a','b','c']
'a,b,c'.split(',', 1) # ['a', 'b,c']
'a b c'.split() # whitespace auto-split
'a-b-c'.rsplit('-', 1) # ['a-b', 'c']
'key=value'.partition('=') # ('key','=','value')
# Join (performance):
parts = ['a', 'b', 'c']
', '.join(parts)
# Per-character split:
list('abc')
# Multiple delimiters:
import re
re.split(r'[;,|]', 'a;b,c|d')
# Avoid + in loops (O(n^2))

Strip & Pad

strip/lstrip/rstrip trim whitespace. zfill zero-pads, center aligns, removeprefix/removesuffix strip known strings (3.9+).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
s = ' hello '
s.strip() # 'hello'
s.lstrip() # 'hello '
s.rstrip() # ' hello'
' x \n'.strip() # 'x'
# Specific characters:
'xxhelloxx'.strip('x') # 'hello'
# Padding:
'42'.zfill(5) # '00042'
'hi'.center(7, '-') # '--hi---'
# Remove prefix/suffix (3.9+):
'/usr/bin'.removeprefix('/usr/') # 'bin'
'/usr/bin'.removesuffix('/bin') # '/usr'
# Newline cleanup:
text.rstrip('\n')

Unicode & Encoding

Python str is a sequence of Unicode code points. encode to bytes, decode back. len counts code points.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
s = '你好👋'
len(s) # 3 (emoji is 1 codepoint)
# Encode:
b = s.encode('utf-8')
# Decode:
b.decode('utf-8')
# Codepoint:
ord('中') # 20013
chr(20013) # '中'
# Normalize:
import unicodedata
unicodedata.normalize('NFKC', s)
# Case:
s.casefold() # stricter lowercase
# Encoding error handling:
b.decode('utf-8', errors='replace')
# Iterate codepoints: for ch in s is already codepoint-level

Search & Replace

find/index locate substrings, replace substitutes, translate maps a translation table, count tallies occurrences.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
s = 'the quick brown fox'
s.find('quick') # 4
s.find('z') # -1
s.index('quick') # 4 (raises if missing)
s.rfind('o') # search from right
s.count('o') # 4
# Replace:
s.replace('fox', 'dog')
s.replace('o', '0', 1) # replace only once
# Replace many (translate):
table = str.maketrans({'a': 'A', 'e': 'E'})
'apple'.translate(table) # 'ApplE'
# Case-insensitive replace:
import re
re.sub('(?i)the', 'THE', s)
# Extract substring: s[4:9]

Text Wrapping

textwrap formats text: wrap to a width, dedent common leading whitespace, fill to a paragraph.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import textwrap
long_text = '这是一个比较长的文本,需要按照指定宽度断行显示。'
# Wrap by width:
lines = textwrap.wrap(long_text, width=20)
# Fill into a paragraph:
textwrap.fill(long_text, width=20)
# Remove common indent:
block = '''\
line1
line2'''
textwrap.dedent(block)
# Indent:
textwrap.indent('a\nb', ' ')
# Use case: CLI help text layout
# Console output alignment

8.Collections & Containers

list/dict/set/tuple operations, the collections module, and sorting.

List Operations

append to the tail, insert anywhere, remove by value, pop the tail or an index, index to find, count to tally.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
nums = [3, 1, 2]
nums.append(4) # [3,1,2,4]
nums.insert(0, 0) # [0,3,1,2,4]
nums.remove(1) # remove first 1
last = nums.pop() # pop tail
first = nums.pop(0) # pop at index
nums.index(2) # index
nums.count(1)
nums.reverse() # reverse in place
nums.clear() # clear
# Copy:
nums[:] or nums.copy()
# Extend:
nums.extend([5, 6])
nums += [7, 8]

Dict Operations

Fetch with .get, merge with update or |, iterate items, sort by key/value, and drill into nested dicts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
d = {'a': 1, 'b': 2}
d['c'] = 3 # add/update
d.get('x', 0) # default 0
val = d.setdefault('y', 0)
# Merge (3.9+):
merged = d | {'z': 9}
d.update({'z': 9})
# Iterate:
for k, v in d.items(): pass
# Sort by key:
sorted(d.items())
# Sort by value:
sorted(d.items(), key=lambda kv: kv[1])
# Nested:
d['user']['name']
# Safe chain access:
d.get('user', {}).get('name')
# defaultdict: collections.defaultdict

Set Operations

Union |, intersection &, difference -, symmetric difference ^. Membership test is O(1). Great for dedup.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
a = {1, 2, 3}
b = {2, 3, 4}
a | b # {1,2,3,4} union
a & b # {2,3} intersection
a - b # {1} difference
b - a # {4}
a ^ b # {1,4} symmetric difference
# Test:
2 in a # True O(1)
# Subset:
a <= b; a < b
# Dedupe preserving order:
list(dict.fromkeys([3, 1, 3, 2]))
# Find duplicates:
[x for x in lst if lst.count(x) > 1]
# Immutable: frozenset

Tuple Usage

tuple is an immutable sequence — handy for multiple return values, records, and dict keys. namedtuple is a typed variant.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
t = (1, 2, 3)
# Unpack:
a, b, c = t
# Single element needs comma:
single = (1,)
# Record use:
point = (10, 20)
x, y = point
# Named tuple:
from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])
p = Point(1, 2)
p.x; p[0] # two access styles
# Named tuple with type annotations:
# from typing import NamedTuple
# Immutable: t[0] = 9 raises
# As dict key (hashable)

collections Utilities

Counter for tallies, deque for double-ended queues, defaultdict for missing-key defaults, OrderedDict for ordered mappings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from collections import Counter, deque, defaultdict
# Count:
Counter('abracadabra').most_common(2)
# [('a', 5), ('r', 2)]
# Double-ended queue:
q = deque([1, 2])
q.appendleft(0); q.pop(); q.popleft()
# Fixed length (drop oldest):
last5 = deque(maxlen=5)
# defaultdict:
dd = defaultdict(list)
dd['k'].append(1) # auto-create empty list
# Ordered dict:
# Python 3.7+ dict is ordered by default
# Counter arithmetic:
Counter('aab') + Counter('abb')
# Counter('aab') - Counter('b')

Sorting

sorted returns a new list; list.sort sorts in place. Provide a key callable. Set reverse=True to flip the order.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
nums = [3, 1, 2]
sorted(nums) # [1,2,3] new list
nums.sort() # in-place sort
# Reverse:
sorted(nums, reverse=True)
# Sort by key:
words = ['banana', 'apple', 'cherry']
sorted(words, key=len)
# List of dicts:
users.sort(key=lambda u: u['age'])
# Multi-key sort:
sorted(users, key=lambda u: (u['age'], u['name']))
# Stable sort:
sorted(users, key=lambda u: u['age'])
# Descending key:
sorted(users, key=lambda u: -u['age'])

Slicing

seq[start:stop:step] slices. Negative indices count from the end. Reverse with step -1, copy with [:].

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
s = list(range(10))
s[2:5] # [2, 3, 4]
s[:3] # from start
s[7:] # to end
s[-3:] # last 3
s[::2] # every other
s[::-1] # reverse
s[1:8:2] # step
# Assign:
s[1:3] = [9] # replace slice
# Delete:
del s[1:3]
# Copy:
copy = s[:]
# Same for strings:
'hello'[::-1] # 'olleh'
# Note: slicing out of range does not raise

Heaps & Queues

heapq implements heaps; queue provides thread-safe queues — including priority queues.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import heapq
nums = [3, 1, 4, 1, 5]
heapq.heapify(nums) # build min-heap
heapq.heappop(nums) # pop smallest
heapq.heappush(nums, 0)
# Get largest/smallest n:
heapq.nlargest(2, [3, 1, 4])
heapq.nsmallest(2, [3, 1, 4])
# Priority queue:
import queue
pq = queue.PriorityQueue()
pq.put((1, '低'))
pq.put((0, '高'))
pq.get() # (0, '高')
# Thread-safe: queue.Queue()
# Task queue: queue.Queue(maxsize)

9.Memory & Performance

Garbage collection, reference counting, memory tools, and performance tuning.

Garbage Collection

Reference counting handles most reclamation; a generational GC breaks reference cycles. No manual free needed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Refcount hits zero -> collected
x = [1, 2]
y = x
del x # count -1, still alive
del y # count hits zero, collected
# Cyclic references:
import gc
gc.collect() # manually triggered
# Object finalizer hook:
class Temp:
def __del__(self):
print('回收')
# Generally no need to call gc manually
# Refcount can't handle cycles; generational GC catches them

memoryview & Zero-Copy

memoryview exposes a buffer without copying — efficient for large binary data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
data = bytearray(b'hello world')
view = memoryview(data)
view[0] # 104
view[6:11] # b'world'
# Slice is a view, not a copy:
sub = view[6:11]
sub[0] = 87 # modifications affect original
# Format:
mem = memoryview(b'\x00\x01')
mem.cast('H') # read as 16-bit unsigned
# Use cases:
# 1. Chunked large-file processing
# 2. Protocol parsing without copies
# 3. struct combined views
# Faster than slice-copy in perf-sensitive paths

Weak References

weakref.ref does not keep its referent alive. Useful for caches of large objects and for breaking reference cycles.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import weakref
class Node:
pass
node = Node()
ref = weakref.ref(node)
print(ref()) # the object
node = None # drop strong reference
gc.collect()
print(ref()) # None (already collected)
# WeakKeyDictionary/WeakValueDictionary:
# key/value weakly referenced, removed when collected
cache = weakref.WeakValueDictionary()
# Prevent cycles:
# replace one edge with a weak reference
# Use cases: cache, observer, singleton

__slots__ for Memory

__slots__ declares fixed attributes and removes per-instance __dict__ — saving memory and speeding access for millions of instances.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# By default each instance has __dict__:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
# With __slots__:
class Point:
__slots__ = ('x', 'y')
def __init__(self, x, y):
self.x = x
self.y = y
p = Point(1, 2)
# p.z = 3 Error: undeclared attribute
# Saves: one dict per instance
# Significant for millions of small objects
# Tradeoffs: can't add attrs dynamically, no __dict__

Large-Data Processing

Process large data in chunks, iterate lazily, and never load it all at once. Stream files line by line.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Stream a large file line-by-line:
with open('big.log') as f:
for line in f: # lazy per-line
process(line)
# Use generators for large lists:
def read_all_lines(path):
with open(path) as f:
for line in f:
yield line
# Process in batches:
def chunks(iterable, size):
from itertools import islice
it = iter(iterable)
while batch := list(islice(it, size)):
yield batch
for batch in chunks(range(100), 10):
process_batch(batch)
# Avoid list(map(...)) materializing everything
# numpy handles large numeric data more efficiently

Profiling

cProfile profiles by function and call count; timeit runs micro-benchmarks. Profile before optimizing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import timeit, cProfile
# Microbenchmark:
timeit.timeit("'x' * 100", number=10000)
# Or CLI:
# python3 -m timeit "'-'.join(str(n) for n in range(100))"
# Full profiling:
# python3 -m cProfile -s cumulative script.py
# In-code profiling:
cProfile.run('my_func()', sort='cumulative')
# Timing decorator:
import time
def timed(f):
def w(*a, **k):
t = time.perf_counter()
r = f(*a, **k)
print(f.__name__, time.perf_counter() - t)
return r
return w
# Rule: profile first, then optimize

Performance Tips

Locals beat globals, comprehensions beat loops, set lookups are O(1). Don't recompute in tight loops.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Local-variable caching:
import math
def f(n):
m = math.sqrt # local reference
return m(n)
# Set membership is O(1):
allowed = set(['a', 'b'])
if x in allowed: pass
# Comprehension vs loop:
# comprehensions are usually faster and cleaner
squares = [n * n for n in range(1000)]
# Avoid repeated computation:
# hoist invariants out of loops
# join for string concatenation
# numpy for large numeric data
# C extension / numba for hot paths

Common Memory Errors

MemoryError for out-of-memory, RecursionError for stack blow-ups, and reference cycles — diagnostic tools included.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Stack overflow:
def infinite():
return infinite()
# RecursionError: maximum recursion depth
# Out of memory:
# MemoryError (loading too much at once)
# Diagnostics:
import tracemalloc
tracemalloc.start()
# ... code runs ...
snapshot = tracemalloc.take_snapshot()
top = snapshot.statistics('lineno')
print(top[:5])
# Object reference graph:
import objgraph # third-party
# objgraph.show_refs([obj])
# Fix:
# chunking, releasing refs, generators

10.Classes & Objects

class definition, inheritance, dunder methods, dataclass, and enum.

Class Basics

class defines a type. __init__ initializes instances. self refers to the instance; instance methods bind to it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
def greet(self):
return f'Hi, {self.name}'
p = Person('Nick', 30)
p.greet()
# Class attribute (shared):
class Counter:
total = 0 # class-level
def __init__(self):
Counter.total += 1
# Instance attribute (per instance):
p.name
# Dynamic attribute:
p.email = '[email protected]'
# Type check: isinstance(p, Person)

Attribute Access

Instance attrs shadow class attrs. setattr/getattr/hasattr do dynamic access. property turns a method into a computed attribute.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class User:
default_role = 'user' # class attribute
def __init__(self, name):
self.name = name
# Lookup: instance -> class -> base
u = User('a')
u.default_role # 'user'
# Dynamic access:
hasattr(u, 'name') # True
getattr(u, 'name', None)
setattr(u, 'role', 'admin')
# property:
class Circle:
def __init__(self, r):
self._r = r
@property
def area(self):
return 3.14 * self._r ** 2
c = Circle(2)
c.area # method accessed like attribute
# property setter for validation

Three Kinds of Methods

Instance methods take self, class methods take cls, static methods take neither. Decorate with @staticmethod/@classmethod.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Tool:
count = 0
def __init__(self, name):
self.name = name
Tool.count += 1
def instance_method(self):
return self.name # access instance
@classmethod
def class_method(cls):
return cls.count # access class
@staticmethod
def helper(x):
return x * 2 # no self/cls
# Call:
t = Tool('锤子')
t.instance_method()
Tool.class_method()
Tool.helper(4)
# classmethod for factories / counters
# staticmethod for utility functions

Inheritance

A subclass inherits methods from its parent. super() reaches the parent. Override methods as needed. Multiple inheritance follows MRO.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Animal:
def __init__(self, name):
self.name = name
def speak(self):
return f'{self.name} 发出声音'
class Dog(Animal):
def speak(self):
return f'{self.name} 汪汪'
def fetch(self):
return f'{self.name} 捡球'
d = Dog('Rex')
d.speak() # override
# super() calls parent:
class Cat(Animal):
def __init__(self, name, color):
super().__init__(name)
self.color = color
# isinstance and issubclass
# Multiple inheritance uses MRO order
# Duck typing: prefer protocols over inheritance

Dunder Methods

Dunder methods customize built-in behavior: __repr__ for display, __eq__ for comparison, __len__ for length, __add__ for +.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Vector:
def __init__(self, x, y):
self.x, self.y = x, y
def __repr__(self):
return f'Vector({self.x}, {self.y})'
def __add__(self, other):
return Vector(self.x + other.x, self.y + other.y)
def __eq__(self, other):
return (self.x, self.y) == (other.x, other.y)
def __len__(self):
return 2
v = Vector(1, 2)
str(v); v + v; v == Vector(1, 2)
# Common ones:
# __str__ for end users
# __getitem__ subscript access
# __call__ callable objects
# __bool__ truthiness
# __enter__/__exit__ context manager

dataclass

@dataclass auto-generates __init__, __repr__, __eq__, and more — a declarative way to write data classes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
from dataclasses import dataclass, field
@dataclass
class Point:
x: int
y: int
label: str = 'origin' # default
tags: list = field(default_factory=list)
p = Point(1, 2)
p2 = Point(1, 2)
p == p2 # True (auto __eq__)
repr(p) # Point(x=1, y=2, label='origin')
# Frozen (immutable):
@dataclass(frozen=True)
class Config:
debug: bool = False
# Orderable:
@dataclass(order=True)
class Item:
price: int
# Convert to dict:
from dataclasses import asdict, astuple
asdict(p)
# Inherited fields handled too

Enum

Enum defines a set of named constants. Supports str enums, auto numbering, and iteration. Replaces magic numbers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from enum import Enum, auto
class Color(Enum):
RED = 1
GREEN = 2
BLUE = 3
class Status(Enum):
ACTIVE = auto() # auto-numbered
INACTIVE = auto()
# Access:
Color.RED
Color(1) # lookup by value
Color.RED.value # 1
Color.RED.name # 'RED'
# Iterate:
for c in Color: pass
# String enum:
class Mode(str, Enum):
FAST = 'fast'
SLOW = 'slow'
# Compare:
Color.RED is Color.RED # True (singleton)
# With description:
class Kind(Enum):
A = ('a', '描述')
def __init__(self, code, desc):
self.code = code
self.desc = desc

Context Managers

with manages resources. Implement __enter__/__exit__, or use @contextlib.contextmanager to write one as a generator.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class Timer:
def __enter__(self):
import time
self.start = time.perf_counter()
return self
def __exit__(self, exc_type, exc, tb):
import time
print(f'耗时 {time.perf_counter()-self.start:.4f}s')
return False # False: don't swallow exception
with Timer():
pass
# Simplified:
from contextlib import contextmanager
@contextmanager
def timer():
import time
t = time.perf_counter()
yield
print(f'耗时 {time.perf_counter()-t:.4f}s')
with timer():
pass
# with open(...) is a built-in context manager
# contextlib.suppress / ExitStack

11.Exception Handling

try-except, exception hierarchy, custom exceptions, and the with statement for resource management.

try / except

Catch and handle errors. except filters by type. Use as e to grab the exception object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
try:
num = int('abc')
except ValueError:
print('无法转换')
# Catch and access exception:
except ValueError as e:
print(e)
# Multiple types:
except (ValueError, TypeError):
pass
# Order: more specific first
try:
result = risky()
except (ValueError, KeyError) as e:
print(f'可预期错误: {e}')
except Exception as e:
print(f'未知错误: {e}')
# Uncaught exceptions propagate until crash

else & finally

else runs only when no exception was raised. finally runs no matter what — perfect for cleanup.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
try:
data = parse(input_data)
except ValueError as e:
print(f'解析失败: {e}')
else:
print('解析成功') # runs only if no exception
finally:
cleanup() # always runs
# Combined use:
# try: code that may raise
# else: logic only when successful
# finally: cleanup (close/release)
# finally doesn't suppress propagation
# else or finally can be omitted

Exception Hierarchy

Exception is the base for most errors. Common ones: ValueError, TypeError, KeyError, IndexError.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
try:
d['missing'] # KeyError
except KeyError:
pass
try:
[1][5] # IndexError
except IndexError:
pass
try:
int('x') # ValueError
except ValueError:
pass
try:
1 / 0 # ZeroDivisionError
except ZeroDivisionError:
pass
# Hierarchy:
# BaseException > Exception > specific errors
# KeyboardInterrupt/SystemExit are not Exception
# except Exception catches business errors
# Avoid bare except: (swallows even exits)

Custom Exceptions

Subclass Exception to define domain errors. Add fields and messages. Callers catch by type.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class ValidationError(Exception):
pass
class NotFoundError(Exception):
def __init__(self, entity, key):
self.entity = entity
self.key = key
super().__init__(f'{entity} 不存在: {key}')
# Raise:
def get_user(user_id):
if user_id == 0:
raise NotFoundError('用户', user_id)
return {'id': user_id}
# Catch:
try:
get_user(0)
except NotFoundError as e:
print(e.entity, e.key)
# Common base class:
class AppError(Exception): pass
# SpecificError(AppError)

raise & Chaining

raise throws deliberately. raise ... from ... chains the cause. A bare raise inside except re-throws the current exception.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
def validate(age):
if age < 0:
raise ValueError('年龄不能为负')
# Reraise:
try:
risky()
except SomeError:
raise # reraise as-is
# Exception chaining (preserve cause):
try:
parse_file(path)
except FileNotFoundError as e:
raise RuntimeError(f'读取失败: {path}') from e
# Suppress chain:
raise NewError('x') from None
# Traceback (most recent call last):
# cause first, current after
# Preserve __cause__ for debugging

assert

assert checks invariants during development. Running with -O strips asserts — never use them for input validation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def process(data):
assert data is not None, 'data 不能为空'
assert isinstance(data, list), '期望列表'
return len(data)
# Failed assert raises AssertionError
# Disable asserts:
# python3 -O script.py
# Use for:
# 1. Internal invariants
# 2. Debug-time argument checks
# 3. Type preconditions
# Don't use assert for:
# user input validation
# security boundary checks
# critical business logic

Exceptions & Logging

Use logging to capture exceptions with stack traces. logger.exception in an except block records the full traceback.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import logging
logging.basicConfig(level=logging.INFO)
try:
x = 1 / 0
except ZeroDivisionError:
logging.exception('除法出错') # with stack trace
# Or:
logger.error('出错', exc_info=True)
# Levels:
logging.debug/info/warning/error
# Config:
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
)
# Structured:
logger.info('请求', extra={'id': req_id})
# Don't use print for logging (no level, no timestamp)

Error Handling Patterns

EAFP ("easier to ask forgiveness than permission") vs LBYL ("look before you leap"). Prefer guard clauses for early returns.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# EAFP: try first, catch on error
def parse_eafp(s):
try:
return int(s)
except ValueError:
return None
# LBYL: check before acting
def parse_lbyl(s):
if not s.isdigit():
return None
return int(s)
# Broad vs specific catch:
# Specific: except ValueError
# Fallback: except Exception
# Guard clauses for early return:
def handle(user):
if user is None:
return
if not user.active:
return
process(user)
# Reduces nesting, flatter logic

12.Input / Output

print/input, file I/O, pathlib, JSON/CSV, and standard streams.

Output & Input

print accepts many args with sep/end control. input reads one line as str. Format outputs with f-strings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
print('a', 'b') # a b
print('a', 'b', sep=', ') # a, b
print('end', end='!\n')
# Print to a file:
with open('out.txt', 'w') as f:
print('text', file=f)
# Input:
name = input('你的名字? ') # always str
# Numeric input:
age = int(input('年龄? '))
# Multiple inputs at once:
a, b = input().split()
# Output variables:
print(f'Hi {name}')
# Console progress:
print('\r进度 50%', end='')
# Flush buffer: print(..., flush=True)

File Read/Write

open modes: r read, w write, a append, b binary. read/readline/readlines for reading.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Read:
f = open('data.txt', encoding='utf-8')
text = f.read() # all
line = f.readline() # one line
lines = f.readlines() # list of lines
f.close()
# Write:
f = open('out.txt', 'w', encoding='utf-8')
f.write('内容\n')
f.writelines(['a', 'b'])
f.close()
# Modes:
# 'r' read, 'w' overwrite write, 'a' append
# 'rb'/'wb' binary
# 'r+' read/write
# Always use with to auto-close

with & Files

with closes the file automatically — even on exceptions. The idiomatic way to read a file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Recommended:
with open('data.txt', encoding='utf-8') as f:
text = f.read() # auto-closes after use
# Line-by-line:
with open('data.txt', encoding='utf-8') as f:
for line in f:
print(line.rstrip())
# Write:
with open('out.txt', 'w', encoding='utf-8') as f:
f.write('hello')
# Append:
with open('log.txt', 'a') as f:
f.write('更多')
# Error handling:
try:
with open('missing.txt') as f:
pass
except FileNotFoundError:
print('文件不存在')
# Always pass encoding for cross-platform consistency

pathlib Paths

Path provides cross-platform paths with /, exists, mkdir, read_text, etc. The modern replacement for os.path.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from pathlib import Path
p = Path('data') / 'sub' / 'file.txt'
p.exists() # exists?
p.is_file(); p.is_dir()
p.mkdir(parents=True, exist_ok=True)
# Read/write:
text = Path('data.txt').read_text(encoding='utf-8')
Path('out.txt').write_text('hi', encoding='utf-8')
# Iterate:
for f in Path('.').glob('*.py'):
print(f.name)
# rglob recursive:
Path('.').rglob('*.json')
# Attributes:
p.name; p.suffix; p.stem
p.parent; p.absolute()
# Common dirs:
Path.home()
Path.cwd()
# Cleaner syntax, cross-platform

JSON Read/Write

json.loads/dumps serialize. Convert dict/list to/from JSON. Pass ensure_ascii=False to keep non-ASCII characters readable.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import json
data = {'name': 'Nick', 'tags': ['a', 'b']}
# Serialize:
text = json.dumps(data, ensure_ascii=False)
# Pretty:
json.dumps(data, indent=2, ensure_ascii=False)
# Deserialize:
obj = json.loads(text)
# File I/O:
with open('data.json', 'w', encoding='utf-8') as f:
json.dump(data, f, ensure_ascii=False, indent=2)
with open('data.json', encoding='utf-8') as f:
obj = json.load(f)
# Error:
try:
json.loads('invalid')
except json.JSONDecodeError as e:
print(e)
# Types: JSON object <-> dict

CSV Read/Write

The csv module handles comma-separated values. writer writes, reader and DictReader read.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import csv
rows = [['name', 'age'], ['Nick', 30], ['Anna', 25]]
# Write:
with open('data.csv', 'w', newline='', encoding='utf-8') as f:
writer = csv.writer(f)
writer.writerows(rows)
# Read:
with open('data.csv', newline='', encoding='utf-8') as f:
reader = csv.reader(f)
for row in reader:
print(row)
# Dict read/write:
with open('data.csv', encoding='utf-8') as f:
for row in csv.DictReader(f):
print(row['name'])
# Write dicts:
with open('o.csv', 'w', newline='') as f:
w = csv.DictWriter(f, fieldnames=['name'])
w.writeheader()
w.writerow({'name': 'x'})
# Watch quoting / newline handling

Standard Streams

sys.stdin/stdout/stderr handle pipe data. Read line by line from stdin.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import sys
# Read one line:
line = sys.stdin.readline()
# Read all lines (piped):
for line in sys.stdin:
sys.stdout.write(line.upper())
# Error stream:
print('error', file=sys.stderr)
# Pipe usage:
# cat data.txt | python3 filter.py
# Redirect:
# python3 filter.py > out.txt
# Encoding:
# PYTHONIOENCODING=utf-8 controls stream encoding
# input() is equivalent to sys.stdin.readline()
# Progress to stderr to avoid polluting stdout

Binary I/O

Binary mode reads and writes bytes. struct packs and unpacks. seek/tell provide random access.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Read/write bytes:
with open('img.bin', 'rb') as f:
data = f.read()
with open('out.bin', 'wb') as f:
f.write(b'\x00\x01\x02')
# Fixed-size chunks:
with open('file.bin', 'rb') as f:
while chunk := f.read(1024):
process(chunk)
# struct pack:
import struct
packed = struct.pack('>I', 1024) # 4 bytes
value = struct.unpack('>I', packed)[0]
# Random access:
with open('db.bin', 'rb') as f:
f.seek(100) # seek to offset
data = f.read(10)
# Append: 'ab' mode

13.Common Pitfalls

The most common pitfalls in everyday Python development and the correct idioms.

Mutable Default Argument

A mutable default is created once and shared across calls. Use None as a sentinel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# BAD: shared default list
def add(item, basket=[]):
basket.append(item)
return basket
add('a') # ['a']
add('b') # ['a', 'b'] !
# GOOD: None + create inside
def add(item, basket=None):
if basket is None:
basket = []
basket.append(item)
return basket
# Why: default is evaluated once at def time
# Impact: hits list/dict/set

is vs == Confusion

is compares identity, == compares values. Small ints / short strings may be interned — never use is for content.

1
2
3
4
5
6
7
8
9
10
11
12
13
# BAD: use is to compare values
a = [1, 2]
b = [1, 2]
a is b # False, but values match
# GOOD: == compares values
a == b # True
# Use is for None:
x = None
x is None # True
# Beware interning:
# 257 is 257 may be True in CPython
# but that's an implementation detail, don't rely on it
# Rule: == for values, is for None/singletons

Loop Closure Pitfall

A lambda in a loop captures the loop variable by reference (late binding). Bind the current value if you need it now.

1
2
3
4
5
6
7
8
9
10
11
# BAD: they all print 9
funcs = [lambda: i for i in range(10)]
funcs[0]() # 9
# GOOD: bind the current value
funcs = [lambda i=i: i for i in range(10)]
funcs[0]() # 0
# Or capture it through a factory:
funcs = [(lambda x: lambda: x)(i) for i in range(10)]
# Why: a closure captures the name, not the value
# Fix: default arguments are evaluated at definition time
# Note: the for variable still exists after the loop ends

String Concatenation

Loop + concat is O(n^2). Collect into a list and join — or build with io.StringIO for huge inputs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# BAD: concatenating in a loop
s = ''
for i in range(10000):
s += str(i) # builds a new string every time
# GOOD: collect, then join
parts = []
for i in range(10000):
parts.append(str(i))
s = ''.join(parts)
# Or with a generator:
s = ''.join(str(i) for i in range(10000))
# A couple of + joins are fine:
name = first + ' ' + last
# Rule: use join when concatenating in a loop
# f-strings suit fragments you already know

Mutating a List While Iterating

Mutating a list while iterating it skips/repeats elements. Iterate over a copy, or build a new list.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# BAD: removing while iterating skips items
items = [1, 2, 3, 4]
for x in items:
if x % 2 == 0:
items.remove(x) # the indices shift
# GOOD: build a new list
items = [x for x in items if x % 2 == 1]
# Or iterate over a copy:
for x in items[:]:
if x % 2 == 0:
items.remove(x)
# Changing dict keys while iterating is just as dangerous
# Principle: the iterator is invalidated -> iterate a copy
# Or build a new container instead

Shadowing Built-ins

Naming a variable after a built-in shadows it. Avoid overwriting list, str, input, sum, etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# BAD: shadowing a built-in
def total(data):
sum = 0 # hides the built-in sum
for x in data:
sum += x
return sum
# Calling sum() later fails
# GOOD: pick a different name
total = 0
def total(data):
return sum(data)
# Common traps:
# list = [] shadows list()
# input = ... shadows input()
# same story for type/str/dict
# Comprehension variables do not leak (own scope since 3.x)

Swallowing Exceptions

A bare except catches everything — including KeyboardInterrupt. Narrow it to specific types, or at least log it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# BAD: a bare except silently swallows everything
import os
try:
os.remove('f.txt')
except:
pass # you now know nothing
try:
value = int(x)
except Exception:
value = 0 # hides type errors
try:
risky()
except Exception:
print('出错了') # not even clear what failed
# GOOD: catch precisely + log it
import logging
try:
value = int(x)
except (ValueError, TypeError) as e:
logging.warning('转换失败 %s', e)
value = 0
# Never swallow KeyboardInterrupt/SystemExit

Misusing Globals

Assigning to a global inside a function creates a local. Add global to rebind it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# BAD: looks like a global update but raises
count = 0
def inc():
count += 1 # UnboundLocalError
# Why: += is an assignment, so a local count is created
# GOOD: declare it global
def inc():
global count
count += 1
# Reading a global needs no declaration:
def show():
print(count)
# Do not overuse globals:
# pass arguments and return values
# or wrap the state in a class
# keep module-level config read-only

Copy & Sort

list.sort sorts in place; sorted returns a new list. Shared references can lead to accidental mutation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# BAD: sort mutates in place
original = [3, 1, 2]
sorted_list = original.sort() # returns None
# and the original ends up sorted too
# GOOD: sorted returns a new list
original = [3, 1, 2]
sorted_list = sorted(original)
# Or copy first, then sort:
c = original[:]; c.sort()
# Shared references:
b = original
b.append(9) # original changes as well
# Copy when you need independence:
new_list = original[:]
# Mutating a parameter affects the caller:
# pass a copy instead

14.Concurrency & Async

threading, asyncio, the GIL, and choosing the right concurrency model.

Thread Basics

threading.Thread creates a thread. start launches it, join waits, daemon=True makes it a background thread.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import threading
def work(n):
print(f'线程 {n} 工作')
threads = []
for i in range(3):
t = threading.Thread(target=work, args=(i,))
threads.append(t)
t.start()
for t in threads:
t.join() # wait for it to finish
# Thread object with arguments:
threading.Thread(target=work, args=(1,), daemon=True)
# A daemon thread exits with the main thread
# Keep the thread count modest to avoid context-switch overhead
# Good enough for simple concurrency

Thread Class

Subclass Thread and override run. Use locks to share data safely — a clean way to package worker threads.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import threading
class Worker(threading.Thread):
def __init__(self, name):
super().__init__(name=name)
self.result = None
def run(self):
self.result = heavy_compute()
w = Worker('job-1')
w.start()
w.join()
print(w.result)
# Override run, never start
# Return data through instance attributes
# Exceptions in a child thread are silent:
# wrap the body of run in try-except and log
# Prefer ThreadPoolExecutor when you need return values

Locks & Synchronization

Lock guards shared state. with lock acquires and releases automatically. RLock is reentrant.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import threading
lock = threading.Lock()
counter = 0
def increment():
global counter
for _ in range(1000):
with lock: # acquires and releases for you
counter += 1
threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()
print(counter) # 10000 (the lock protects it)
# RLock: the same thread may acquire it again
# Condition variable: Condition
# Event: Event
# Semaphore: caps the level of concurrency
# Deadlock: inconsistent lock order, waiting while holding a lock
# Prefer a Queue to hand data over without locks

GIL & Parallelism

CPython's GIL prevents CPU-bound threading from going parallel. Threads still help when the workload is I/O-bound.

1
2
3
4
5
6
7
8
9
10
11
12
13
# GIL: the global interpreter lock
# Only one thread executes bytecode at any moment
# CPU-bound work gains nothing from threads:
# a compute task can even get slower
# I/O-bound work does benefit:
# the GIL is released while waiting on network/disk
# Real parallelism for CPU-bound work:
# 1. multiprocessing, separate processes
# 2. C extensions that release the GIL (numpy and friends)
# 3. asyncio, single-threaded async
# Deciding:
# bottleneck is I/O -> threads/asyncio
# bottleneck is CPU -> processes

Multiprocessing

multiprocessing parallelizes CPU-bound work with a process pool. ProcessPoolExecutor is the high-level wrapper.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from multiprocessing import Pool
def square(n):
return n * n
with Pool(4) as pool:
results = pool.map(square, range(10))
# ProcessPoolExecutor:
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers=4) as ex:
results = list(ex.map(square, range(10)))
# Separate memory: processes share nothing
# Communication: Queue/Pipe
# Startup costs more than a thread
# Watch the serialisation cost with big data
# Pair it with if __name__ == '__main__'

asyncio Basics

async def defines a coroutine, await suspends it, asyncio.run drives the event loop.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import asyncio
async def say_hi():
await asyncio.sleep(1)
print('hi')
asyncio.run(say_hi())
# await suspends the coroutine and yields control
# Running concurrently:
async def main():
task = asyncio.create_task(say_hi())
await task
asyncio.run(main())
# The event loop:
# schedules coroutines inside a single thread
# Blocking code stalls the loop:
# time.sleep -> await asyncio.sleep
# swap blocking libraries for async ones: httpx/aiohttp
# 3.11+ TaskGroup manages tasks

async / await

async declares a coroutine. await waits on a result. gather runs coroutines in parallel. asyncio.timeout sets deadlines.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import asyncio
async def fetch(url):
await asyncio.sleep(0.1)
return f'data from {url}'
# Sequential:
async def main():
a = await fetch('a')
b = await fetch('b')
# Parallel:
results = await asyncio.gather(
fetch('a'), fetch('b'), fetch('c'))
# Timeout:
try:
result = await asyncio.wait_for(fetch('a'), timeout=1)
except asyncio.TimeoutError:
print('超时')
# Creating tasks in bulk:
tasks = [fetch(u) for u in urls]
await asyncio.gather(*tasks)
# Results keep the input order; exceptions propagate

Async I/O

aiohttp / httpx make async HTTP requests. For very high-throughput I/O, async wins handily.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# httpx async client:
# pip install httpx
# import httpx
async def main():
async with httpx.AsyncClient() as client:
r = await client.get('https://example.com')
return r.status_code
# Mixing in synchronous code:
async def worker():
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, blocking_fn)
# Async files:
import aiofiles
# async with aiofiles.open('f', 'r') as f:
# text = await f.read()
# Throughput improves markedly with heavy concurrent I/O
# Databases: asyncpg/aiomysql

Executors

ThreadPoolExecutor pools threads; ProcessPoolExecutor pools processes. Both expose submit() and map().

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from concurrent.futures import ThreadPoolExecutor, as_completed
def work(n):
return n * 2
with ThreadPoolExecutor(max_workers=4) as ex:
futures = [ex.submit(work, i) for i in range(10)]
for f in as_completed(futures):
print(f.result()) # completion order
# map keeps the input order:
with ThreadPoolExecutor(4) as ex:
results = list(ex.map(work, range(10)))
# Exceptions:
# f.result() re-raises the original exception
# shutdown(wait=True) waits for the pool
# A thread pool suits I/O-bound work
# A process pool suits CPU-bound work
# 3.9+ cancellation: f.cancel()

15.Networking & Modules

Module system, HTTP requests, URL handling, and servers.

Modules & Imports

import loads a module. from x import y pulls specific names. A package is a directory with an __init__.py.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Importing:
import os
import os.path
import json as js # alias
from datetime import datetime, timedelta
from collections import * # not recommended
# Relative imports (inside a package):
# from . import utils
# from ..other import x
# How importing works:
# sys.path is the search path
# cached: a module is loaded only once
# __main__ is the entry point
# Your own modules:
# utils.py in the same directory
# import utils
# Lazy import: import inside a function to cut startup time

HTTP Requests

The requests library: GET/POST, query params, headers, JSON bodies, response status.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import requests
# GET:
r = requests.get('https://api.example.com/users',
params={'page': 2}, timeout=5)
# Status:
r.status_code == 200
r.ok
# Data:
r.json() # dict
r.text # text
r.content # bytes
# POST:
resp = requests.post('https://api.example.com/login',
json={'user': 'nick'},
headers={'Authorization': 'Bearer x'},
timeout=10)
# Errors:
try:
r.raise_for_status()
except requests.HTTPError:
print('请求失败')
# Session (keeps cookies):
s = requests.Session()

urllib Standard Library

urllib.request makes HTTP requests without third-party libs. urllib.parse splits URLs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import urllib.request
with urllib.request.urlopen('https://example.com') as resp:
data = resp.read()
print(resp.status)
# Building a request with parameters:
req = urllib.request.Request('https://api.com',
headers={'User-Agent': 'my-app'},
data=b'body')
with urllib.request.urlopen(req) as resp:
print(resp.read().decode('utf-8'))
# URL parsing:
from urllib.parse import urlparse, urlencode, quote
urlparse('https://a.com/p?q=1#f')
# ParseResult(scheme='https', netloc='a.com', path='/p', query='q=1')
urlencode({'a': 1, 'b': 'x y'}) # 'a=1&b=x+y'
quote('中文') # percent-encoding
# Reach for requests first; urllib is the stdlib fallback

Simple HTTP Server

http.server serves static files or quick prototypes. Subclass BaseHTTPRequestHandler to customize.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Static file server:
# python3 -m http.server 8000
# Browse http://localhost:8000
# Bind an address:
# python3 -m http.server 8000 --bind 0.0.0.0
# Custom handler:
from http.server import HTTPServer, BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = b'{"ok": true}'
self.send_response(200)
self.send_header('Content-Type', 'application/json')
self.send_header('Content-Length', str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, fmt, *args):
pass # stay quiet
HTTPServer(('', 8000), Handler).serve_forever()
# Use a framework in production: FastAPI/Flask

Socket Networking

socket provides raw TCP/UDP. Hand-roll protocols or inspect traffic here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import socket
# TCP client:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect(('example.com', 80))
s.sendall(b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n')
data = s.recv(4096)
print(data[:200])
# TCP server:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server:
server.bind(('0.0.0.0', 9000))
server.listen(5)
conn, addr = server.accept()
with conn:
data = conn.recv(1024)
conn.sendall(b'pong')
# UDP: SOCK_DGRAM with sendto/recvfrom
# Timeout: s.settimeout(3)
# Prefer a framework for complex protocols

URLs & Parameters

urlparse parses URLs, urlencode builds query strings, quote percent-encodes, urljoin resolves relative paths.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from urllib.parse import urlparse, urlencode, quote, urljoin
# Parse:
p = urlparse('https://[email protected]:8080/path?q=1#sec')
p.scheme; p.netloc; p.path
p.query; p.fragment
# Build a query string:
params = urlencode({'q': 'python 教程', 'page': 2})
# 'q=python+%E6%95%99%E7%A8%8B&page=2'
url = f'https://api.com/search?{params}'
# Join:
urljoin('https://a.com/docs/', '../about')
# Decode:
from urllib.parse import unquote
unquote('%E4%B8%AD') # '中'
# Path safety:
quote('a/b', safe='') # escapes the slash
# Parse the parameters:
from urllib.parse import parse_qs
parse_qs('a=1&a=2&b=3') # {'a': ['1','2'], 'b': ['3']}

Config & Environment

os.environ exposes environment variables; python-dotenv loads a .env file. Keep config layered.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import os
# Read an environment variable:
port = int(os.environ.get('PORT', '3000'))
secret = os.environ['SECRET_KEY'] # KeyError when missing
# Set one:
os.environ['DEBUG'] = '1'
# Test one:
if os.environ.get('DEBUG') == '1':
print('调试模式')
# dotenv:
# pip install python-dotenv
from dotenv import load_dotenv
load_dotenv() # loads .env
# The .env file:
# PORT=3000
# SECRET_KEY=xxx
# Why:
# deployment config stays out of the code
# secrets never reach version control
# Validate config: fail loudly when something is missing
# Layers: default/dev/prod

API Call Patterns

Wrap requests with retries and rate limiting. Validate responses with pydantic for typed data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import requests
from time import sleep
def api_get(url, retries=3):
for attempt in range(retries):
try:
r = requests.get(url, timeout=10)
if r.status_code == 429: # rate limited
sleep(2 ** attempt) # back off
continue
r.raise_for_status()
return r.json()
except requests.RequestException:
if attempt == retries - 1:
raise
sleep(2 ** attempt)
# Shared headers:
HEADERS = {'User-Agent': 'my-app/1.0'}
# Response validation (pydantic):
# pip install pydantic
# from pydantic import BaseModel
# class User(BaseModel):
# name: str
# age: int
# u = User.model_validate(r.json())
# Caching: lru_cache / diskcache

16.Date & Time

datetime, formatting, timedeltas, and time zones.

datetime Basics

datetime construction, date/time subclasses, attribute access, and comparison.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from datetime import datetime, date, time
# Current time:
now = datetime.now() # local
now_utc = datetime.utcnow() # deprecated, use timezone
# Construct:
d = datetime(2026, 8, 2, 12, 30, 0)
# Access:
d.year; d.month; d.day
d.hour; d.minute; d.second
# Weekday:
d.weekday() # 0=Monday
d.isoweekday() # 1=Monday
# Date or time only:
date(2026, 8, 2)
time(12, 30)
# Compare:
d > datetime(2026, 1, 1)
# An invalid month raises ValueError

Format & Parse

strftime formats, strptime parses. Codes: %Y year, %m month, %d day, %H hour.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from datetime import datetime
now = datetime.now()
# Formatting:
now.strftime('%Y-%m-%d') # '2026-08-02'
now.strftime('%Y-%m-%d %H:%M:%S')
now.strftime('%H:%M') # hour and minute
now.strftime('%A') # weekday name
# Parsing a string:
d = datetime.strptime('2026-08-02', '%Y-%m-%d')
d = datetime.fromisoformat('2026-08-02T12:00:00')
# Common directives:
# %Y 4-digit year, %y 2-digit year
# %m month, %d day, %H 24-hour, %I 12-hour
# %M minute, %S second, %f microsecond
# Localised weekday: map it yourself
# Timezone safety: attach the tz before formatting

timedelta

timedelta expresses duration. Add/subtract days and hours. Compute differences between dates.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from datetime import datetime, timedelta
now = datetime.now()
# Plus 7 days:
now + timedelta(days=7)
# Plus 2 hours:
now + timedelta(hours=2)
# Minus 30 minutes:
now - timedelta(minutes=30)
# Difference between two moments:
d1 = datetime(2026, 8, 2)
d2 = datetime(2026, 1, 1)
delta = d1 - d2
delta.days # 213
delta.total_seconds() # seconds
# Available units:
# weeks/days/hours/minutes/seconds
# Deltas compare directly:
if delta > timedelta(days=100):
print('超过百天')
# Month differences need your own maths:
# timedelta has no month unit

Time Zones

timezone handles fixed offsets; zoneinfo (3.9+) loads IANA names. Store in UTC, render in local.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from datetime import datetime, timezone, timedelta
# Fixed offset:
utc8 = timezone(timedelta(hours=8))
now_utc8 = datetime.now(utc8)
# Named timezone (3.9+):
from zoneinfo import ZoneInfo
shanghai = ZoneInfo('Asia/Shanghai')
now_sh = datetime.now(shanghai)
# Conversion:
utc_time = datetime.now(timezone.utc)
local_time = utc_time.astimezone(shanghai)
# Aware vs naive:
# only a value carrying tzinfo is aware
# comparisons need the same zone, or both aware
# In practice:
# store UTC in the database
# convert to the user's zone for display
# the tzdata package ships the zone database

Timestamps

Timestamps are Unix seconds. Convert with timestamp()/fromtimestamp(). Ideal for storage and comparison.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import time
from datetime import datetime, timezone
# Unix timestamp (seconds):
now_ts = time.time()
datetime.now().timestamp()
# datetime to timestamp:
dt = datetime(2026, 8, 2, tzinfo=timezone.utc)
dt.timestamp()
# Timestamp to datetime:
datetime.fromtimestamp(now_ts) # local
datetime.fromtimestamp(now_ts, timezone.utc) # UTC
# Milliseconds:
int(time.time() * 1000)
# Comparing timestamps:
if now_ts > last_ts: pass
# Storage:
# an integer count of unix seconds is the common standard
# Display: convert the timestamp and format it locally
# Precision: the float carries fractional seconds

Timing & Sleeping

time.sleep pauses, time.perf_counter measures. For cron-style jobs use the schedule library.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import time
# Wait:
time.sleep(1.5) # seconds
# Timing:
start = time.perf_counter()
# run the code...
elapsed = time.perf_counter() - start
print(f'{elapsed:.4f}s')
# Precision:
time.time() # wall clock, can be adjusted
time.perf_counter() # high-resolution and monotonic
# Timed loop:
def run_every(interval, fn):
while True:
fn()
time.sleep(interval)
# Scheduling libraries:
# pip install schedule
# import schedule
# schedule.every(10).minutes.do(job)
# while True: schedule.run_pending()
# Blocking vs async waiting:
# inside an event loop use asyncio.sleep

Date Utilities

Weekday detection, month start/end, relative dates, date ranges. The calendar module helps.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from datetime import datetime, timedelta
import calendar
d = datetime.now()
# Weekday test:
d.weekday() < 5 # a working day
# First day of the month:
d.replace(day=1)
# Last day of the month:
last_day = calendar.monthrange(d.year, d.month)[1]
d.replace(day=last_day)
# A sequence of dates:
start = datetime(2026, 8, 1)
for i in range(7):
day = start + timedelta(days=i)
print(day.strftime('%m-%d %a'))
# Natural-language parsing:
# pip install python-dateutil
# from dateutil.parser import parse
# parse('2026-08-02 10:30')
# Month arithmetic:
# dateutil.relativedelta
# from dateutil.relativedelta import relativedelta
# d + relativedelta(months=1)

Time Interval Checks

Check whether a time falls in a window, detect overlapping intervals, compute remaining time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from datetime import datetime, timedelta
# Range test:
now = datetime.now()
start = datetime(2026, 1, 1)
end = datetime(2027, 1, 1)
if start <= now < end:
print('在窗口内')
# Overlapping ranges:
def overlaps(a1, a2, b1, b2):
return a1 < b2 and b1 < a2
# Time remaining:
dealine = now + timedelta(hours=2)
remaining = deadline - now
if remaining > timedelta(minutes=30):
print('时间充足')
# A scheduled window:
def in_working_hours(dt):
return 9 <= dt.hour < 18
# Boundaries:
# half-open ranges [start, end) avoid double counting
# normalise when a range crosses midnight
# Rate limiting: compare against the previous timestamp

17.Processes & System

subprocess, the sys/os modules, command-line arguments, and signals.

subprocess

subprocess.run launches external commands. Capture output, check the return code, and set a timeout.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import subprocess
# Run and capture the output:
result = subprocess.run(
['ls', '-l'],
capture_output=True, text=True)
print(result.returncode) # 0 means success
print(result.stdout)
print(result.stderr)
# Check for failure:
result.check_returncode() # non-zero raises CalledProcessError
# Timeout:
subprocess.run(['sleep', '10'], timeout=2)
# On timeout: TimeoutExpired
# Safety:
# pass an argument list, do not build a shell string
# shell=True carries an injection risk
# Live output: read line by line from subprocess.Popen

sys Module

sys.argv holds CLI arguments, sys.exit sets the exit code, sys.path the module search path.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import sys
# Command-line arguments:
sys.argv # ['script.py', 'arg1', 'arg2']
# Exit:
sys.exit(0) # success
sys.exit(1) # failure
# Module search path:
sys.path # a list
sys.path.append('/custom') # add an entry
# Version:
sys.version_info # (3, 11, ...)
sys.version
# Streams:
sys.stdin; sys.stdout; sys.stderr
# Platform:
sys.platform # 'win32' / 'darwin' / 'linux'
# Encoding:
sys.getdefaultencoding() # 'utf-8'
# Recursion limit:
sys.getrecursionlimit()
sys.setrecursionlimit(5000)

os Module

os covers environment, paths, and directory operations. os.getcwd for the current directory, os.mkdir to create one.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import os
# Environment variables:
os.environ['HOME']
# Directories:
os.getcwd() # current directory
os.chdir('/tmp') # change directory
os.mkdir('dir') # create a directory
os.makedirs('a/b/c', exist_ok=True)
# Files:
os.remove('file')
os.rename('a', 'b')
# Paths (os.path):
os.path.join('a', 'b') # cross-platform join
os.path.exists('f')
os.path.isdir('d')
os.path.dirname('/a/b/c') # '/a/b'
os.path.basename('/a/b/c') # 'c'
os.path.splitext('f.txt') # ('f', '.txt')
# Walking a tree:
for root, dirs, files in os.walk('.'):
print(root, files)
# Prefer pathlib over os.path

argparse

argparse parses arguments, generates help, supports defaults. The standard way to write CLI tools.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import argparse
parser = argparse.ArgumentParser(description='示例工具')
parser.add_argument('input', help='输入文件')
parser.add_argument('-v', '--verbose', action='store_true')
parser.add_argument('-n', '--count', type=int, default=1)
args = parser.parse_args()
print(args.input, args.verbose, args.count)
# Usage:
# python3 tool.py data.txt -v -n 3
# Help:
# python3 tool.py --help
# Options:
parser.add_argument('--out', default='out.txt')
# Mutually exclusive arguments:
# parser.add_mutually_exclusive_group()
# Subcommands:
# subparsers = parser.add_subparsers()
# Type validation: type=int converts automatically

Exit Codes

0 means success, non-zero failure. CI/scripts rely on exit codes. An unhandled exception exits with 1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import sys
# Normal exit:
sys.exit(0)
# Failure:
sys.exit(1)
sys.exit('错误信息') # prints to stderr and exits with 1
# An uncaught exception exits with 1 automatically
# Custom codes:
sys.exit(2) # conventionally a usage error
# Inspecting the return code:
# shell: echo $? (POSIX)
# python3 script.py; echo $?
# From the caller:
import subprocess
r = subprocess.run(['python3', 'x.py'])
if r.returncode != 0:
print('失败')
# In CI any non-zero code fails the build

Files & Processes

Large file handling, file locks, temp files, atomic writes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Temporary file:
import tempfile, os
with tempfile.NamedTemporaryFile(mode='w', delete=True) as f:
f.write('data')
name = f.name
# Temporary directory:
with tempfile.TemporaryDirectory() as d:
print(d) # cleaned up automatically
# Atomic write (write a temp file, then rename):
import os
def atomic_write(path, content):
tmp = path + '.tmp'
with open(tmp, 'w') as f:
f.write(content)
os.replace(tmp, path) # atomic replacement
# File locks:
# pip install filelock
# from filelock import FileLock
# with FileLock('app.lock'):
# mutually exclusive work
# Large files: read line by line or in fixed blocks

Signal Handling

signal handles Ctrl+C and SIGTERM for graceful shutdown. POSIX mainly — Windows has limited support.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import signal, sys, time
# Graceful shutdown flag:
running = True
def handle_stop(signum, frame):
global running
print(f'收到信号 {signum},正在退出...')
running = False
signal.signal(signal.SIGINT, handle_stop) # Ctrl+C
signal.signal(signal.SIGTERM, handle_stop) # kill
# Main loop:
while running:
work()
time.sleep(1)
print('清理完成')
# Ignore a signal:
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
# Timeout signal (POSIX):
# signal.alarm(5) raises SIGALRM after 5 seconds
# Windows supports only some signals
# Graceful shutdown: save state, close connections

Daemons & Backgrounding

Run in the background, rotate logs, supervise with supervisor or systemd.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Run in the background:
# python3 script.py &
# Redirect the output:
# python3 script.py > out.log 2>&1 &
# nohup ignores hangups:
# nohup python3 script.py &
# Log rotation:
import logging
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
'app.log', maxBytes=1_000_000, backupCount=5)
logging.basicConfig(handlers=[handler])
logging.info('运行中')
# Supervision in production:
# a systemd unit / supervisor restart policy
# Enforcing a single instance:
# check a pid lock file
# Resource limits: ulimit

18.Regular Expressions & Text

The re module: matching, searching, replacing, groups, and compilation.

match & search

re.match anchors at the start; re.search scans the whole string. Returns a Match or None.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import re
# search: anywhere in the string
m = re.search(r'\d+', '订单号 123')
if m:
print(m.group()) # '123'
# match: only from the start
m2 = re.match(r'\d+', '123abc')
# m2.group() == '123'
# No match returns None:
if re.search(r'xyz', 'abc'): pass
# Position info:
m.start(); m.end(); m.span()
# Word boundaries:
re.search(r'\bcat\b', 'a cat eats')
# Metacharacters must be escaped:
r'\.' matches a literal dot

findall & finditer

findall returns all matches as a list. finditer yields them one at a time (memory-friendly for large inputs).

1
2
3
4
5
6
7
8
9
10
11
12
13
import re
# findall: every match
nums = re.findall(r'\d+', 'a1b22c333')
# ['1', '22', '333']
# With groups it returns tuples:
re.findall(r'(\d+)-(\d+)', '1-2,3-4')
# [('1','2'), ('3','4')]
# finditer: iterate over Match objects
for m in re.finditer(r'\d+', 'a1b22'):
print(m.group(), m.span())
# Use finditer when you need positions
# finditer saves memory on large text
# No match gives an empty list / empty iterator

sub Replacement

re.sub performs regex replacement. Backreferences \1 … \9 in the replacement; pass a function for dynamic logic.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import re
# Simple replacement:
re.sub(r'\s+', ' ', 'a b c') # 'a b c'
# Group references:
re.sub(r'(\d{4})-(\d{2})-(\d{2})', r'\3/\2/\1',
'2026-08-02')
# '02/08/2026'
# Replacement function:
re.sub(r'\d+', lambda m: str(int(m.group()) * 2),
'1 and 2')
# '2 and 4'
# Limit the count:
re.sub(r'a', 'x', 'aaa', count=1) # 'xaa'
# Case-insensitive:
re.sub(r'hello', 'hi', 'HELLO', flags=re.I)

split

re.split cuts on a regex. Supports multiple delimiters, retains captures, and caps splits.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import re
# Split on comma/semicolon/pipe:
re.split(r'[,;|]', 'a,b;c|d')
# ['a', 'b', 'c', 'd']
# Split on whitespace (runs included):
re.split(r'\s+', 'a b c')
# A capture group keeps the separator:
re.split(r'(,)', 'a,b,c')
# ['a', ',', 'b', ',', 'c']
# Limit:
re.split(r',', 'a,b,c', maxsplit=1)
# ['a', 'b,c']
# Split into lines:
re.split(r'\r?\n', text)
# By paragraph:
re.split(r'\n\s*\n', text)
# Use re.split when str.split cannot handle several separators

Groups

() captures; (?P<name>) names a group; (?:) groups without capturing; | alternation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import re
# Groups:
m = re.search(r'(\w+)@(\w+)\.(\w+)', '[email protected]')
m.group(0) # '[email protected]', the whole match
m.group(1) # 'a'
m.group(2) # 'b'
m.groups() # ('a','b','com')
# Named groups:
m = re.search(r'(?P<user>\w+)@(?P<domain>\w+)', '[email protected]')
m.group('user') # 'a'
m.groupdict() # {'user': 'a', 'domain': 'b'}
# Non-capturing:
re.search(r'(?:ab)+', 'abab')
# Alternation:
re.search(r'cat|dog', 'a dog')
# Group reference in a replacement: \1
# Nested groups are numbered by opening parenthesis

Flags

re.I case-insensitive, re.M multiline anchors, re.S dot matches newlines, re.X verbose.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import re
# re.I ignores case:
re.search(r'hello', 'HELLO', re.I)
# re.M makes the anchors multiline:
re.search(r'^line', 'a\nline b', re.M)
# re.S lets the dot match newlines:
re.search(r'a.b', 'a\nb', re.S)
# re.X verbose (comments and whitespace):
pattern = re.compile(r'''
(\d{4}) # year
[-/]
(\d{2}) # month
''', re.X)
# Combining flags:
re.search(r'x', 'X', re.I | re.M)
# Compile once and reuse:
pat = re.compile(r'\d+', re.IGNORECASE)
pat.search('a1')
# Precompiling is faster and pins the options

Common Patterns

Common regex snippets for email, phone, URL, IP, Chinese, etc. Validate business formats.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import re
# Email:
EMAIL = r'^[^\s@]+@[^\s@]+\.[^\s@]+$'
# Chinese mainland mobile number:
MOBILE = r'^1[3-9]\d{9}$'
# URL:
URL = r'^https?://[^\s]+$'
# IPv4:
IP = r'^(\d{1,3}\.){3}\d{1,3}$'
# Chinese characters:
CHINESE = re.compile(r'[\u4e00-\u9fff]+')
# ID card number (18 digits):
ID = r'^\d{17}[0-9Xx]$'
# Date:
DATE = r'^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])$'
# Validation helper:
def is_email(s): return bool(re.match(EMAIL, s))
# Extract Chinese text:
CHINESE.findall('hello 世界 你好')
# Colour:
COLOR = r'^#[0-9a-fA-F]{6}$'

Regex Performance

Precompile patterns, avoid catastrophic backtracking (nested quantifiers), and bound widths. Process big text in chunks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import re
# Precompile:
pattern = re.compile(r'\d+') # at module level
pattern.findall(text)
# Avoid catastrophic backtracking:
# BAD: nested quantifiers
# re.match(r'(a+)+$', 'a' * 30 + '!') extremely slow
# GOOD: simplify it
re.match(r'a+$', 'aaa')
# Bound the width:
re.match(r'[a-z]{1,20}', 'x' * 100)
# Use a character class instead of alternation:
# BAD: [a|b|c]
# GOOD: [abc]
# Large text:
# process it in sections rather than all at once
# Prefer str methods for simple parsing:
# s.startswith / s.split
# Let regex handle structural matching

19.Build & Engineering

Dependency management, virtual environments, testing, linting, and CI.

requirements.txt

Pin versions with pip freeze -> requirements.txt. pip install -r installs. Lock for reproducibility.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Export the current environment:
pip freeze > requirements.txt
# Example file content:
# requests==2.31.0
# httpx>=0.24.0
# # a comment explaining the purpose
# Install:
pip install -r requirements.txt
# Keep dev dependencies separate:
# requirements-dev.txt holds the test tooling
# Version pinning strategy:
# == pins exactly
# >= allows upgrades
# Pin exact versions for production
# Modern alternative:
# declare in pyproject.toml + a lock file
# uv installs extremely fast

venv in Practice

Best practices: one venv per project, never commit it, recreate on broken dependency trees.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Create:
python3 -m venv .venv
# Activate (POSIX):
source .venv/bin/activate
# Activate (Windows):
# .venv\Scripts\activate
# Check the current interpreter:
which python
# Leave it: deactivate
# Delete and rebuild (when dependencies break):
# rm -rf .venv && python3 -m venv .venv
# Add .venv/ to .gitignore
# Install dependencies into the venv:
# never pip install into the system environment
# Point your editor at the venv interpreter

Modern Package Managers

uv for blazing-fast installs; poetry for dependency + packaging. Lock files make builds reproducible.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# uv (recommended, extremely fast):
# pip install uv
uv venv # create a virtual environment
uv add requests # add a dependency
uv sync # install everything in sync
# As a pip replacement:
uv pip install -r requirements.txt
# poetry:
# poetry new mypkg
# poetry add requests
# poetry install
# pyproject.toml + poetry.lock
# The consistent workflow:
# declare dependencies -> lock file -> CI installs from the lock
# The lock file gives everyone the same versions
# Upgrade: uv lock --upgrade

Testing with pytest

Name tests test_* and use assert. fixtures share setup, parametrize runs over inputs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import pytest
from app import add
def test_add():
assert add(1, 2) == 3
def test_add_negative():
assert add(-1, 1) == 0
# Parametrised:
@pytest.mark.parametrize('a,b,expected', [
(1, 2, 3),
(0, 0, 0),
])
def test_add_param(a, b, expected):
assert add(a, b) == expected
# fixture:
def setup_db():
db = create_db()
yield db
db.close()
def test_using_fixture(setup_db):
assert setup_db.query()
# Asserting an exception:
with pytest.raises(ValueError):
int('x')
# Run: pytest or python3 -m pytest
# Coverage: pytest --cov

unittest Standard Library

The stdlib testing framework. Subclass TestCase, use assertXxx methods, override setUp/tearDown.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import unittest
from app import add
class TestAdd(unittest.TestCase):
def setUp(self):
self.data = [1, 2]
def test_add(self):
self.assertEqual(add(1, 2), 3)
def test_types(self):
self.assertIsInstance(add(1, 2), int)
def test_raises(self):
with self.assertRaises(TypeError):
add('a', 1)
if __name__ == '__main__':
unittest.main()
# Run:
# python3 -m unittest test_app
# Discovery mode:
# python3 -m unittest discover
# Assertion methods:
# assertEqual/assertTrue/assertIn
# assertAlmostEqual for floats
# mock: unittest.mock fakes dependencies

Lint & Formatting

ruff for fast lint + format, black for style, mypy for types. Pin the rules in pyproject.toml.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# ruff (recommended, extremely fast):
# pip install ruff
ruff check . # check
ruff format . # format
ruff check --fix . # fix automatically
# black:
# pip install black
black --line-length 88 .
# mypy type checking:
# pip install mypy
mypy src/
# Configure it in pyproject.toml:
# [tool.ruff]
# line-length = 88
# [tool.mypy]
# strict = true
# Editor integration:
# format on save
# Before committing: a pre-commit hook

Packaging & Publishing

Configure packaging in pyproject.toml, build sdist + wheel, publish to PyPI with twine.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# pyproject.toml:
# [project]
# name = "my-pkg"
# version = "0.1.0"
# description = "..."
# requires-python = ">=3.10"
# dependencies = ["requests"]
#
# [build-system]
# requires = ["setuptools>=68"]
# build-backend = "setuptools.build_meta"
#
# [tool.setuptools.packages.find]
# include = ["mypkg*"]
# Build:
# pip install build
# python3 -m build
# produces dist/*.whl and tar.gz
# Publish:
# pip install twine
# twine upload dist/*
# Test index: test.pypi.org
# Semantic versioning: 1.2.0
# Try the install before publishing: pip install .

CI & Deployment

Pipeline stages: lint, type-check, test, build. Configure in GitHub Actions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# GitHub Actions:
# .github/workflows/ci.yml
# name: CI
# on: [push, pull_request]
# jobs:
# test:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5
# with: python-version: '3.11'
# - run: pip install -e .[dev]
# - run: ruff check .
# - run: mypy src/
# - run: pytest --cov
# Cache dependencies: actions/cache
# Multi-version matrix: python-version: [3.10, 3.11, 3.12]
# A failure blocks the merge
# Deployment: upload to PyPI on release
# Environment variables: keep secrets in GitHub Secrets

Official Links

Direct links to the official docs and resources.

Version 2.1.1