Justin’s Python Performance Guide

This is an exhaustive catalog of optimizations I’ve learned from years of making my Python projects perform faster. I try to keep it up to date with recent Python interpreter releases.

The Python Speed and Python Performance Tips sections of the official Python wiki might also be worth checking out.

- Justin Turner Arthur

Contents:

• Macro-optimization
  • Do things concurrently
  • Run long-running Python with pypy
  • If speed is the top priority, don’t use Python
• Micro-optimization
  • Use Class slots if you’re constantly constructing objects
  • Use operators instead of functions where possible
  • Use literals instead of functions where possible
  • Use a faster JSON builder/parser
  • Use sets or frozensets for frequent “in” queries
  • Avoid sets or frozensets for frequent inserts
  • Avoid frequent object attribute lookup
  • Don’t use ExitStack or AsyncExitStack for visual organization
  • Pre-compile frequently used structs and regular expressions
  • Avoid runtime imports
  • If you have the RAM, avoid generator comprehensions that get exhausted
  • Function of loop instead instead of loop of function (cpython)
  • Faster set and frozenset constructors in older interpreters
  • The faster no-op and constant return function
  • Don’t use queue.Queue, asyncio.Queue, or trio.MemoryChannel if you never wait
• Optimizations for Popular Libraries
  • aiohttp
  • boto3
  • requests

• Search Page