cpython-lru_cache-weakref: Objects/codeobject.c comparison

comparison Objects/codeobject.c @ 54297:dddf424bd669 legacy-trunk

Issue #6042: lnotab-based tracing is very complicated and isn't documented very well. There were at least 3 comment blocks purporting to document co_lnotab, and none did a very good job. This patch unifies them into Objects/lnotab_notes.txt which tries to completely capture the current state of affairs. I also discovered that we've attached 2 layers of patches to the basic tracing scheme. The first layer avoids jumping to instructions that don't start a line, to avoid problems in if statements and while loops. The second layer discovered that jumps backward do need to trace at instructions that don't start a line, so it added extra lnotab entries for 'while' and 'for' loops, and added a special case for backward jumps within the same line. I replaced these patches by just treating forward and backward jumps differently.

author	Jeffrey Yasskin <jyasskin@gmail.com>
date	Sat, 23 May 2009 23:23:01 +0000
parents	a171048daf76
children	22f589a8e3ab

comparison

equal deleted inserted replaced

-:d9ce3e740676
+:dddf424bd669
 	0,				/* tp_init */
 	0,				/* tp_alloc */
 	code_new,			/* tp_new */
 };
-/* All about c_lnotab.
+/* Use co_lnotab to compute the line number from a bytecode index, addrq.  See
+lnotab_notes.txt for the details of the lnotab representation.
-c_lnotab is an array of unsigned bytes disguised as a Python string.  In -O
-mode, SET_LINENO opcodes aren't generated, and bytecode offsets are mapped
-to source code line #s (when needed for tracebacks) via c_lnotab instead.
-The array is conceptually a list of
-(bytecode offset increment, line number increment)
-pairs.  The details are important and delicate, best illustrated by example:
-byte code offset    source code line number
-0		    1
-6		    2
-50		    7
-350                 307
-361                 308
-The first trick is that these numbers aren't stored, only the increments
-from one row to the next (this doesn't really work, but it's a start):
-0, 1,  6, 1,  44, 5,  300, 300,  11, 1
-The second trick is that an unsigned byte can't hold negative values, or
-values larger than 255, so (a) there's a deep assumption that byte code
-offsets and their corresponding line #s both increase monotonically, and (b)
-if at least one column jumps by more than 255 from one row to the next, more
-than one pair is written to the table. In case #b, there's no way to know
-from looking at the table later how many were written.  That's the delicate
-part.  A user of c_lnotab desiring to find the source line number
-corresponding to a bytecode address A should do something like this
-lineno = addr = 0
-for addr_incr, line_incr in c_lnotab:
-addr += addr_incr
-if addr > A:
-return lineno
-lineno += line_incr
-In order for this to work, when the addr field increments by more than 255,
-the line # increment in each pair generated must be 0 until the remaining addr
-increment is < 256.  So, in the example above, com_set_lineno should not (as
-was actually done until 2.2) expand 300, 300 to 255, 255,  45, 45, but to
-255, 0,  45, 255,  0, 45.
 */
 int
 PyCode_Addr2Line(PyCodeObject *co, int addrq)
 {
 		line += *p++;
 	}
 	return line;
 }
-/*
+/* Update *bounds to describe the first and one-past-the-last instructions in
-Check whether the current instruction is at the start of a line.
+the same line as lasti.  Return the number of that line. */
+int
-*/
+_PyCode_CheckLineNumber(PyCodeObject* co, int lasti, PyAddrPair *bounds)
-	/* The theory of SET_LINENO-less tracing.
-	   In a nutshell, we use the co_lnotab field of the code object
-	   to tell when execution has moved onto a different line.
-	   As mentioned above, the basic idea is so set things up so
-	   that
-	         *instr_lb <= frame->f_lasti < *instr_ub
-	   is true so long as execution does not change lines.
-	   This is all fairly simple.  Digging the information out of
-	   co_lnotab takes some work, but is conceptually clear.
-	   Somewhat harder to explain is why we don't *always* call the
-	   line trace function when the above test fails.
-	   Consider this code:
-	   1: def f(a):
-	   2:     if a:
-	   3:        print 1
-	   4:     else:
-	   5:        print 2
-	   which compiles to this:
-	   2           0 LOAD_FAST                0 (a)
-		       3 JUMP_IF_FALSE            9 (to 15)
-		       6 POP_TOP
-	   3           7 LOAD_CONST               1 (1)
-		      10 PRINT_ITEM
-		      11 PRINT_NEWLINE
-		      12 JUMP_FORWARD             6 (to 21)
-		 >>   15 POP_TOP
-	   5          16 LOAD_CONST               2 (2)
-		      19 PRINT_ITEM
-		      20 PRINT_NEWLINE
-		 >>   21 LOAD_CONST               0 (None)
-		      24 RETURN_VALUE
-	   If 'a' is false, execution will jump to instruction at offset
-	   15 and the co_lnotab will claim that execution has moved to
-	   line 3.  This is at best misleading.  In this case we could
-	   associate the POP_TOP with line 4, but that doesn't make
-	   sense in all cases (I think).
-	   What we do is only call the line trace function if the co_lnotab
-	   indicates we have jumped to the *start* of a line, i.e. if the
-	   current instruction offset matches the offset given for the
-	   start of a line by the co_lnotab.
-	   This also takes care of the situation where 'a' is true.
-	   Execution will jump from instruction offset 12 to offset 21.
-	   Then the co_lnotab would imply that execution has moved to line
-	   5, which is again misleading.
-	   Why do we set f_lineno when tracing?  Well, consider the code
-	   above when 'a' is true.  If stepping through this with 'n' in
-	   pdb, you would stop at line 1 with a "call" type event, then
-	   line events on lines 2 and 3, then a "return" type event -- but
-	   you would be shown line 5 during this event.  This is a change
-	   from the behaviour in 2.2 and before, and I've found it
-	   confusing in practice.  By setting and using f_lineno when
-	   tracing, one can report a line number different from that
-	   suggested by f_lasti on this one occasion where it's desirable.
-	*/
-int
-PyCode_CheckLineNumber(PyCodeObject* co, int lasti, PyAddrPair *bounds)
 {
 int size, addr, line;
 unsigned char* p;
 p = (unsigned char*)PyString_AS_STRING(co->co_lnotab);
 /* possible optimization: if f->f_lasti == instr_ub
 (likely to be a common case) then we already know
 instr_lb -- if we stored the matching value of p
 somwhere we could skip the first while loop. */
-/* see comments in compile.c for the description of
+/* See lnotab_notes.txt for the description of
 co_lnotab.  A point to remember: increments to p
-should come in pairs -- although we don't care about
+come in (addr, line) pairs. */
-the line increments here, treating them as byte
-increments gets confusing, to say the least. */
 bounds->ap_lower = 0;
 while (size > 0) {
 if (addr + *p > lasti)
 break;
 bounds->ap_lower = addr;
 line += *p++;
 --size;
 }
-/* If lasti and addr don't match exactly, we don't want to
-change the lineno slot on the frame or execute a trace
-function.  Return -1 instead.
-*/
-if (addr != lasti)
-line = -1;
 if (size > 0) {
 while (--size >= 0) {
 addr += *p++;
 if (*p++)
 break;

Mercurial > cpython-lru_cache-weakref

comparison Objects/codeobject.c @ 54297:dddf424bd669 legacy-trunk