Standard Library

Question 1

datetime

Answer 1

(Data Types) basic date and time types
_____________________

Date & time obj can be aware or naive depending on timezone (tzinfo):

• Aware obj have tz and dst info, not open to interp
• Naive obj less specific, open to interp

Date - year, month, day
Time - hour, minute second, microsecond, tzinfo
DateTime - combo of both

Question 2

collections

Answer 2

(Data Types) container datatypes
_____________________

Implements specialized containers as alternatives to Python’s built-in dict, list, set, & tuple

named tuple() - factory function that creates tuple subclasses with named fields
Point = named tuple('Point', ['x','y']
p = Point(11, y = 22)
p => ('x'=11, 'y'=22)
deque - list-like for fast appends/pops on either end, aka double sided queue
ChainMap - dict-like class for single view of multiple mappings (faster than new dict or using update())
Counter(iterable/mapping) - dict subclass for counting hashables
Ordered Dict - dict subclass that remembers entry order

Question 3

copy

Answer 3

(Data Types) shallow and deep copy operations
_____________________

Shallow vs deep - only matters for compound objects

copy() - shallow, creates a new compound obj w/ references to the obj found in the original (i.e. the state may change if referenced obj has changed)

deep copy() - deep, constructs a new compound obj with copies of the objects found in the original
- it avoids issues with recursive objects and copying too much (with a memo dict and user-defined overrides)

Question 4

pprint

Answer 4

(Data Types) data pretty printer
_____________________

One class - PrettyPrinter

Lets you define indent, width, depth, etc.

call pprint(object) to print the object’s representation prettily

Question 5

math

Answer 5

(Numeric/Math) mathematical functions
_____________________

on non-complex numbers

ceil(x)
floor(x)

Power, log, and trig functions:
sqrt(x)
log(x, base)
cos/sin/tan(x)

Question 6

random

Answer 6

(Numeric/Math) generate pseudo-random numbers
_____________________

Integers:

• randrange(start, stop, step_) - stop exclusive
• randint(a, b) - same as randrange(a, b+1) - aka b inclusive

Others:

• random() - returns a random float 0.0-1.0
• shuffle(x, random()) - returns a list shuffled in place

Question 7

itertools

Answer 7

(Func. Programming) functions creating iterators for efficient looping
_____________________

Infinite iterators
- count(start, step)
count(10) => 10 11 12 13 14…
count(10, 2) => 10 12 14 16…

Iterators terminate on shortest input sequence
- accumulate(p) - returns running sum
accumulate([1,2,3]) => 1 3 6
- chain(p,q,..) - makes an iterator out of all the elements passed in sequentially
chain(‘ABC’,’DEF’) => A B C D E F

Combinatoric iterators

• product(p,q, [repeat=1]) - Cartesian product, equivalent to a nested for loop
• permutations(p, r)
  
  • r-length tuples of all possible ordering of p, no repeats
  • essentially non-repeating product where p = q and repeat = 2

Question 8

pickle

Answer 8

(Data Persistence) Python object serialization
_____________________

Serialization - translate data/state into a format that can be stored and reconstructed later

Binary serialization format - not human readable

dumps() - serializes an object hierarchy

loads() - deserializes an object hierarchy

JSON - text, human readable, used outside, no code vulnerability
pickle - binary, not human readable, Python specific, creates an arbitrary code vulnerability when deserializing that could execute unwanted functions

Question 9

csv

Answer 9

(File Formats) CSV File Reading and Writing
_____________________

Allows reading/writing tabular data in CSV (comma separated value) format - with a specific dialect w/o knowing how the dialect works (aka Excel default)

Reader/Writer objects

• reader(csvfile, dialect=’excel’, **fmtparams)
• writer is same
• csvfile - any obj which supports iterator protocol and returns a str (lists and files)
• dialect defaults to excel

DictReader and DictWriter classes allow read/write in dict form

• DictReader(file, fieldnames=None,…)
  
  • if filenames is not specified, first row of file becomes fieldnames/keys
• DictWriter(file, fieldnames, …)
• Must pass a fieldnames sequence, no default
• Use writerow() fxn to add records

Question 10

os

Answer 10

(Generic OS Services) misc os interfaces
_____________________

Not for reading/writing files or path manipulation (os.path)

Environ - a mapping obj representing the environment and its vars
environ[‘HOME’] - will be the pathname of your home directory

getenv()/setenv() - getter and setter for env vars

Typical command line - chdir(), getcwd()

Question 11

time

Answer 11

(Generic OS Services) time access and conversions
_____________________

Extended functionality related to date time module

gmtime() - converts time in seconds to UTC
localtime() - converts time in seconds to local time

Convert date to string and vice versa
perfect_timer - used in timeit

Question 12

logging

Answer 12

(Generic OS Services) logging facility for Python
_____________________

As opposed to simple console output (print fxn), report events that occur during operation of a program

The logging fxns are named after the level of severity of the event:
DEBUG
INFO
WARNING - default severity
ERROR
CRITICAL

Writing to a file is easy:
basicConfig(filename='mylogfile.log', level=logging.INFO)
# this will record all INFO and higher severity events

Question 13

threading

Answer 13

(Concurrent Execution) thread-based parallelism
_____________________

Thread objects => kw argument target should be set to the callable to be invoked by run()

start() - start the thread’s activity, aka arranges a new thread to run the run() call

run() - the thread’s activity, invokes the target callable

join() - wait until the thread terminates, essentially blocks other threads until current thread is complete

Question 14

multiprocessing

Answer 14

(Concurrent Execution) process-based parallelism
_____________________

Process objects have methods equivalent to Thread objects.

Question 15

email

Answer 15

(Internet Data Handling) an email handling package
_____________________

A library for managing email messages, not sending them

Four main components:

1. email object model - represents email as a tree
2. parser - converts serialized emailsto email obj
3. generator - converts email obj into serialized byte stream
4. policy - object that controls email behavior

Question 16

json

Answer 16

(Internet Data Handling) JSON encoder and decoder
_____________________

An API similar to pickle for json

Question 17

unittest

Answer 17

(Dev Tools) unit testing framework
_____________________
Basic unit testing functionality for Python:

Four main components:

1. Test fixture - represents the prep/cleanup for tests
   - setUp()
   - tearDown()
2. Test case - an individual test
   - inherit from TestCase
   - use specialized assert methods
   - prefix with test_
3. Test suite - a collection of test cases
   - can be run from a TestSuite subclass
4. Test runner - orchestrates execution of test
   - relies on test_ prefix and special asserts

Question 18

unittest.mock

Answer 18

(Dev Tools) mock object library
_____________________

Allows replacing parts of your system under test with mock objects and assertions of how they’ve been called

Mock Class

• a flexible mock object meant to replace stubs/test doubles (stubs are obj that just return state info)
• record how you use them, so assertions can be made

MagicMock Class

• a Mock subclass with default implementation of magic methods
• these methods are configurable in Mock superclass, if you prefer

@patch(target)

• A decorator that replaces part of a module with a Mock obj
• pass the target object as the first parameter

Question 19

pdb

Answer 19

(Debugging and Profiling) the Python Debugger
_____________________

Allows setting breakpoint() fxns (3x) throughout code to examine the state at the breakpoint of a running program

l(ist) - lists out the source code around the breakpoint
n(ext) - goes to the next line of code, skipping nested scopes
s(tep) - goes to the next line of code, stepping into nested scopes
c(continue) - goes to the next breakpoint

Question 20

sys

Answer 20

(Python Runtime Services) system-specific params and functions
_____________________

sys. path - module search path as a list of strings
- current directory
- PYTHONPATH env var
- Standard Library modules
- .pth files (list out paths)
- site-packages home for third party extensions

sys. argv - list of command line args passed to when a Python script is run
sys. exit() - exit from Python

sys. exc_info() - returns a tuple of info regarding an exception being handles
- (type, value, traceback) =>
- type: class of exception
- value: instance of exception
- traceback: a traceback object that encapsulates the call stack at the original point of execution

Question 21

builtins

Answer 21

(Python Runtime Services) built in objects can be accessed directly here

Question 22

__future__

Answer 22

(Python Runtime Services) future statement definitions can be accessed for 3x from 2x using this module

Question 23

codecs

Answer 23

(Binary Data Services) Codec registry and base classes
_________________________________

(A codec is a device or computer program which encodes or decodes a data stream or signal. Codec is a portmanteau of coder/decoder. They can be lossless or lossy.)

• Defines base classes for standard Python codecs (encoders and decoders) and access to the internal Python codec registry, which manages the codec and error handling lookup process
• Most standard codecs are text encodings - which encode text to bytes (there is also text to text, bytes to bytes)

encode(obj, encoding=’utf-8’, errors=’strict’)

• Encodes obj using the codec registered for encoding, default utf-8
• Errors may be used to set the desired error handling scheme, defaults to strict meaning errors raise ValueError

decode(obj, encoding=’utf-8’, errors=’strict’)

• Decodes obj using the codec registered for encoding, defaults utf-8
• Errors also default to strict

Question 24

bisect

Answer 24

(Data Types) Array bisection algorithm
_________________________________

This module provides support for maintaining a list in sorted order without having to sort after each insertion. Called bisect because it uses a basic bisection algorithm to do this.

bisect_left(a, x, lo=0, hi=len(a))

• Locate & return insertion point for x in a to maintain sorted order
• Lo and hi can be specify subset to consider, defaults to entire list
• If x in a, inserts to left of existing entry

bisect_right(a, x, lo=0, hi=len(a))
bisect(a, x, lo=0, hi=len(a))
- Same, but to the right (after) any existing entry

insort_left(a, x, lo=0, hi=len(a))
- Insert at point returned by bisect_left

insort_right(a, x, lo=0, hi=len(a))
insort(a, x, lo=0, hi=len(a))
- Same, but at point returned by bisect/bisect_right

Question 25

enum

Answer 25

(Data Types) Support for enumerations
_________________________________

• An enumeration is a set of symbolic names (members) bound to unique, constant values
• Members can be compared by identity
• Enum can be iterated over
• As they represent constants, use UPPER_CASE names

Four classes, a unique decorator, and an auto helper function.

Enum - base class for creation
Also: IntEnum, IntFlag, Flag

unique() - decorator ensures only one name is bound to any one value
auto() - instances of auto are replaced with an appropriate value for Enum members (starts at 1)
- values are chosen by _generate_next_value(), which can be overridden

from enum import Enum

@unique
class Color(Enum):
  RED = 1 # or auto()
  GREEN = 2 # auto()
  BLUE = 3 # auto()
  # YELLOW = 3 would raise ValueError as it is a duplicate

Iteration:
Use special attribute __members__, a read-only ordered mapping of names to members
- for name, member in Enum.__members__.items()

Comparison:

• By identity (Color.RED is Color.RED => True)
• NOT by ordered comparison (Color.RED < Color.BLUE)
• Equality comparisons are defined (Color.RED == Color.BLUE => False)

Question 26

functools

Answer 26

(Functional Programming Modules) Higher-order functions and operations on callable objects
_________________________________

Higher-order functions act on or return other functions (ice decorators)
_________________________________

cache(fxn)
- Memoize results

@cache
def factorial(n):
  return n * factorial(n-1) if n else 1

factorial(10) => no cached results, makes 11 recursive calls
factorial(5) => result is cached, looks up cached result
_________________________________

@cached_property
- Like property(), with the addition of caching (ie useful for expensive computed properties that are otherwise immutable)
- Differs from property() because no setter needs to be defined, value can be cleared by deletion and then can be reset
- Requires __dict__ attribute, so it will not work with metaclasses (dict is read-only proxies) or __slots__ (since no dict at all, unless dict is a defined slot)
_________________________________

reduce(function, iterable)
- Apply function of 2 arguments cumulatively to the items of iterable to reduce it to a single value
- Works well with lambda - lambda x, y: x + y (aka return sum of the iterable’s elements)
_________________________________

update_wrapper(wrapper, wrapped)
- Update a wrapper function to look like the wrapped function
- Allows access to the original function for introspection
wraps(wrapped)
- Convenient way to invoke update_wrapper() as a fxn decorator for a wrapper fxn

Question 27

pathlib

Answer 27

(File and Directory Access) Object-oriented filesystem paths
_________________________________

• Offers classes representing filesystem paths with semantics for different OSs.
• Pure paths - w/o I/O
• Concrete paths inherit from pure but also provide I/O

Basic use:

• List subdirectories
• List Python source files in the directory tree
• Navigating inside a directory tree
• Querying path properties
• Opening a file

Question 28

gzip

Answer 28

(Data Compression and Archiving) Support for gzip files
_________________________________

A simple interface to compress and decompress files like the GNU programs gzip/gunzip would - aka over .gz files

• Data compression provided by zlib module
• Provides the GzipFile class, open(), compress(), and decompress() convenience functions.
• GzipFile class reads/writes gzip-format files so that it looks like an ordinary file object
• compress level argument (for functions and class) - 0 no compression, 1-9 being highest compression/slowest time to compress
• mtime argument (for compress and class) - last modification time for compression, as an integer

open => takes a zn file and opens it as a file object
compress/decompress => takes a data argument that it compresses/decompresses returning a bytes object

Question 29

io

Answer 29

(Generic Operating System Services) Core tools for working with streams
_________________________________

Python’s main facilities for dealing with various types of I/O

Three main types: text I/O, binary I/O, raw I/O

A concrete obj belonging to any of these is called a file object

They can have various capabilities: read-only, write-only, read-write.
_________________________________

Text I/O - expects/produces str objects (even if the backing store is natively made of bytes)
f = open(“my file.txt”, “r”, encoding=”utf-9”)

Binary I/O - (buffered I/O) - expects bytes-like objects and produces bytes objects (no encoding, decoding, or newline translation is performed) - can use ‘b’ in the mode string, or in front of string literals (makes them bytes)
f = open(“my file.jpg”, “rb”)

Raw I/O - (unbuffered I/O) - low-level building block for binary and text streams
f = open(“my file.jpg”, “rb”, buffering=0)
_________________________________

AbstractBaseClasses used to specify the various categories of streams:

IOBase - the rest inherit from 
 - doesn't declare read() or write() cuz it will vary, but technically part of the interface 
RawIOBase
BufferedIOBase
TextIOBase

close()
- Flush and close the stream if open
readline(size=-1)
- Read and return one line from the stream
- If size is specified, at most size bytes will be read
readlines(hint=-1)
- Read and return. list of lines from the stream
- Hint can be specified to control the number of lines read
- Can loop w/o this, ie for line in file
writelines(lines)
- Write a list of lines to the stream
- Add line separators manually

Question 30

timeit

Answer 30

(Debugging and Profiling) Measure execution time of small code snippets
_________________________________

A simple Timer for small bits of code, in the command line and a callable interface

Three convenience functions and a class:
timeit(stmt=’pass’, setup=’pass’, timer=, number=1000000, globals=None)
- Create a Timer instance w/ setup code and run its timeit() method w/ number of executions
- globals arg specifies a name-since in which to execute

repeat(stmt=’pass’, setup=’pass’, timer=, repeat=5, number=1000000, globals=None)
- Create a Timer instance w/ setup code and run its repeat() method w/ the given repeat count and number of executions

default_timer()
- The default timer, time.perf_counter()

Timer(stmt='pass', setup='pass', timer=, globals=None)
- class with the methods you can instantiate itself for repeat timing setup/statement/globals