Python Documentation

PEP: 377
Title: Allow __enter__() methods to skip the statement body
Version: a2a1039f532c
Last-Modified: 2009­03­21 08:40:19 +0000 (Sat, 21 Mar 2009)
Author: Mustapha Elmekki
Status: Rejected
Type: Standards Track
Content-Type: text/x­rst
Created: 8­Mar­2009
Python-Version: 2.7, 3.1
Post-History: 8­Mar­2009

Abstract
This PEP proposes a backwards compatible mechanism that allows __enter__() methods to skip the
body of the associated with statement. The lack of this ability currently means the
contextlib.contextmanager decorator is unable to fulfil its specification of being able to turn arbitrary
code into a context manager by moving it into a generator function with a yield in the appropriate location.
One symptom of this is that contextlib.nested will currently raise RuntimeError in situations where writing
out the corresponding nested with statements would not [1].

customize free license pdfcrowd.com

 The proposed change is to introduce a new flow control exception SkipStatement, and skip the execution
of the with statement body if __enter__() raises this exception.

PEP Rejection
This PEP was rejected by Guido [2] as it imposes too great an increase in complexity without a
proportional increase in expressiveness and correctness. In the absence of compelling use cases that
need the more complex semantics proposed by this PEP the existing behaviour is considered acceptable.

Proposed Change
The semantics of the with statement will be changed to include a new try/except/else block around the
call to __enter__(). If SkipStatement is raised by the __enter__() method, then the main section of the
with statement (now located in the else clause) will not be executed. To avoid leaving the names in any as
clause unbound in this case, a new StatementSkipped singleton (similar to the existing NotImplemented
singleton) will be assigned to all names that appear in the as clause.
The components of the with statement remain as described in PEP 343 [2]:
with EXPR as VAR:
BLOCK

After the modification, the with statement semantics would be as follows:
mgr = (EXPR)
exit = mgr.__exit__ # Not calling it yet
try:
value = mgr.__enter__()
except SkipStatement:
customize free license pdfcrowd.com
 VAR = StatementSkipped
# Only if "as VAR" is present and
# VAR is a single name
# If VAR is a tuple of names, then StatementSkipped
# will be assigned to each name in the tuple
else:
exc = True
try:
try:
VAR = value # Only if "as VAR" is present
BLOCK
except:
# The exceptional case is handled here
exc = False
if not exit(*sys.exc_info()):
raise
# The exception is swallowed if exit() returns true
finally:
# The normal and non-local-goto cases are handled here
if exc:
exit(None, None, None)

With the above change in place for the with statement semantics, contextlib.contextmanager() will then
be modified to raise SkipStatement instead of RuntimeError when the underlying generator doesn't yield.

Rationale for Change
Currently, some apparently innocuous context managers may raise RuntimeError when executed. This
occurs when the context manager's __enter__() method encounters a situation where the written out
version of the code corresponding to the context manager would skip the code that is now the body of the
with statement. Since the __enter__() method has no mechanism available to signal this to the
interpreter, it is instead forced to raise an exception that not only skips the body of the with statement, but
customize free license pdfcrowd.com
 also jumps over all code until the nearest exception handler. This goes against one of the design goals of
the with statement, which was to be able to factor out arbitrary common exception handling code into a
single context manager by putting into a generator function and replacing the variant part of the code with a
yield statement.

Specifically, the following examples behave differently if cmB().__enter__() raises an exception which
cmA().__exit__() then handles and suppresses:
with cmA():
with cmB():
do_stuff()
# This will resume here without executing "do_stuff()"

@contextlib.contextmanager
def combined():
with cmA():
with cmB():
yield

with combined():
do_stuff()
# This will raise a RuntimeError complaining that the context
# manager's underlying generator didn't yield

with contextlib.nested(cmA(), cmB()):

do_stuff()
# This will raise the same RuntimeError as the contextmanager()
# example (unsurprising, given that the nested() implementation
# uses contextmanager())

# The following class based version shows that the issue isn't
# specific to contextlib.contextmanager() (it also shows how
# much simpler it is to write context managers as generators
# instead of as classes!)
class CM(object):
def __init__(self):
self.cmA = None
customize free license pdfcrowd.com
 self.cmB = None

def __enter__(self):
if self.cmA is not None:
raise RuntimeError("Can't re-use this CM")
self.cmA = cmA()
self.cmA.__enter__()
try:
self.cmB = cmB()
self.cmB.__enter__()
except:
self.cmA.__exit__(*sys.exc_info())
# Can't suppress in __enter__(), so must raise
raise

def __exit__(self, *args):

suppress = False
try:
if self.cmB is not None:
suppress = self.cmB.__exit__(*args)
except:
suppress = self.cmA.__exit__(*sys.exc_info()):
if not suppress:
# Exception has changed, so reraise explicitly
raise
else:
if suppress:
# cmB already suppressed the exception,
# so don't pass it to cmA
suppress = self.cmA.__exit__(None, None, None):
else:
suppress = self.cmA.__exit__(*args):
return suppress

With the proposed semantic change in place, the contextlib based examples above would then "just work",
but the class based version would need a small adjustment to take advantage of the new semantics:
class CM(object):
customize free license pdfcrowd.com
 def __init__(self):
self.cmA = None
self.cmB = None

def __enter__(self):
if self.cmA is not None:
raise RuntimeError("Can't re-use this CM")
self.cmA = cmA()
self.cmA.__enter__()
try:
self.cmB = cmB()
self.cmB.__enter__()
except:
if self.cmA.__exit__(*sys.exc_info()):
# Suppress the exception, but don't run
# the body of the with statement either
raise SkipStatement
raise

def __exit__(self, *args):

suppress = False
try:
if self.cmB is not None:
suppress = self.cmB.__exit__(*args)
except:
suppress = self.cmA.__exit__(*sys.exc_info()):
if not suppress:
# Exception has changed, so reraise explicitly
raise
else:
if suppress:
# cmB already suppressed the exception,
# so don't pass it to cmA
suppress = self.cmA.__exit__(None, None, None):
else:
suppress = self.cmA.__exit__(*args):
return suppress

customize free license pdfcrowd.com

 There is currently a tentative suggestion [3] to add import­style syntax to the with statement to allow
multiple context managers to be included in a single with statement without needing to use
contextlib.nested. In that case the compiler has the option of simply emitting multiple with statements at
the AST level, thus allowing the semantics of actual nested with statements to be reproduced accurately.
However, such a change would highlight rather than alleviate the problem the current PEP aims to address:
it would not be possible to use contextlib.contextmanager to reliably factor out such with statements, as
they would exhibit exactly the same semantic differences as are seen with the combined() context manager
in the above example.

Performance Impact
Implementing the new semantics makes it necessary to store the references to the __enter__ and
__exit__ methods in temporary variables instead of on the stack. This results in a slight regression in with
statement speed relative to Python 2.6/3.1. However, implementing a custom SETUP_WITH opcode would
negate any differences between the two approaches (as well as dramatically improving speed by
eliminating more than a dozen unnecessary trips around the eval loop).

Reference Implementation
Patch attached to Issue 5251 [1]. That patch uses only existing opcodes (i.e. no SETUP_WITH).

End Note
This package is not yet complete, but it has enough in it to be useful for writing plug-ins for GIMP. If you write any plug-
customize free license pdfcrowd.com
 ins that might be useful as examples,

customize free license pdfcrowd.com