Mercurial > cpython-lru_cache-weakref

annotate Lib/pipes.py @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents 6d0da4a4d677
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
1 """Conversion pipeline templates.
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
3 The problem:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
4 ------------
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
5
18778
23ae47e00119 Use string.ascii_letters instead of string.letters (SF bug #226706).
Fred Drake <fdrake@acm.org>
parents: 17622
diff changeset
6 Suppose you have some data that you want to convert to another format,
23ae47e00119 Use string.ascii_letters instead of string.letters (SF bug #226706).
Fred Drake <fdrake@acm.org>
parents: 17622
diff changeset
7 such as from GIF image format to PPM image format. Maybe the
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
8 conversion involves several steps (e.g. piping it through compress or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
9 uuencode). Some of the conversion steps may require that their input
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
10 is a disk file, others may be able to read standard input; similar for
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
11 their output. The input to the entire conversion may also be read
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
12 from a disk file or from an open file, and similar for its output.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
13
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
14 The module lets you construct a pipeline template by sticking one or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
15 more conversion steps together. It will take care of creating and
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
16 removing temporary files if they are necessary to hold intermediate
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
17 data. You can then use the template to do conversions from many
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
18 different sources to many different destinations. The temporary
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
19 file names used are different each time the template is used.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
20
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
21 The templates are objects so you can create templates for many
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
22 different conversion steps and store them in a dictionary, for
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
23 instance.
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
24
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
25
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
26 Directions:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
27 -----------
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
28
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
29 To create a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
30 t = Template()
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
31
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
32 To add a conversion step to a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
33 t.append(command, kind)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
34 where kind is a string of two characters: the first is '-' if the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
35 command reads its standard input or 'f' if it requires a file; the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
36 second likewise for the output. The command must be valid /bin/sh
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
37 syntax. If input or output files are required, they are passed as
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
38 $IN and $OUT; otherwise, it must be possible to use the command in
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
39 a pipeline.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
40
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
41 To add a conversion step at the beginning:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
42 t.prepend(command, kind)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
43
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
44 To convert a file to another file using a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
45 sts = t.copy(infile, outfile)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
46 If infile or outfile are the empty string, standard input is read or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
47 standard output is written, respectively. The return value is the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
48 exit status of the conversion pipeline.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
49
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
50 To open a file for reading or writing through a conversion pipeline:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
51 fp = t.open(file, mode)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
52 where mode is 'r' to read the file, or 'w' to write it -- just like
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
53 for the built-in function open() or for os.popen().
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
54
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
55 To create a new template object initialized to a given one:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
56 t2 = t.clone()
16474
4d406c26c16e a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15870
diff changeset
57 """ # '
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
58
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
59
6519
24da035ceb6e Convert all remaining *simple* cases of regex usage to re usage.
Guido van Rossum <guido@python.org>
parents: 1435
diff changeset
60 import re
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 import os
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 import tempfile
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 import string
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
64
16474
4d406c26c16e a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15870
diff changeset
65 __all__ = ["Template"]
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
66
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
67 # Conversion step kinds
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
69 FILEIN_FILEOUT = 'ff' # Must read & write real files
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
70 STDIN_FILEOUT = '-f' # Must write a real file
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
71 FILEIN_STDOUT = 'f-' # Must read a real file
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
72 STDIN_STDOUT = '--' # Normal pipeline element
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
73 SOURCE = '.-' # Must be first, writes stdout
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
74 SINK = '-.' # Must be last, reads stdin
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
75
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 stepkinds = [FILEIN_FILEOUT, STDIN_FILEOUT, FILEIN_STDOUT, STDIN_STDOUT, \
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
77 SOURCE, SINK]
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
78
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
79
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
80 class Template:
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
81 """Class representing a pipeline template."""
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
82
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
83 def __init__(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
84 """Template() returns a fresh pipeline template."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
85 self.debugging = 0
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
86 self.reset()
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
87
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
88 def __repr__(self):
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
89 """t.__repr__() implements repr(t)."""
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
90 return '<Template instance, steps=%r>' % (self.steps,)
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
91
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
92 def reset(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
93 """t.reset() restores a pipeline template to its initial state."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
94 self.steps = []
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
95
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
96 def clone(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
97 """t.clone() returns a new pipeline template with identical
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
98 initial state as the current one."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
99 t = Template()
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
100 t.steps = self.steps[:]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
101 t.debugging = self.debugging
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
102 return t
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
103
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
104 def debug(self, flag):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
105 """t.debug(flag) turns debugging on or off."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
106 self.debugging = flag
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
107
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
108 def append(self, cmd, kind):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
109 """t.append(cmd, kind) adds a new step at the end."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
110 if type(cmd) is not type(''):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
111 raise TypeError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
112 'Template.append: cmd must be a string'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
113 if kind not in stepkinds:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
114 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
115 'Template.append: bad kind %r' % (kind,)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
116 if kind == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
117 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
118 'Template.append: SOURCE can only be prepended'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
119 if self.steps and self.steps[-1][1] == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
120 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
121 'Template.append: already ends with SINK'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
122 if kind[0] == 'f' and not re.search(r'\$IN\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
123 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
124 'Template.append: missing $IN in cmd'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
125 if kind[1] == 'f' and not re.search(r'\$OUT\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
126 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
127 'Template.append: missing $OUT in cmd'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
128 self.steps.append((cmd, kind))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
129
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
130 def prepend(self, cmd, kind):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
131 """t.prepend(cmd, kind) adds a new step at the front."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
132 if type(cmd) is not type(''):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
133 raise TypeError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
134 'Template.prepend: cmd must be a string'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
135 if kind not in stepkinds:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
136 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
137 'Template.prepend: bad kind %r' % (kind,)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
138 if kind == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
139 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
140 'Template.prepend: SINK can only be appended'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
141 if self.steps and self.steps[0][1] == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
142 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
143 'Template.prepend: already begins with SOURCE'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
144 if kind[0] == 'f' and not re.search(r'\$IN\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
145 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
146 'Template.prepend: missing $IN in cmd'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
147 if kind[1] == 'f' and not re.search(r'\$OUT\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
148 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
149 'Template.prepend: missing $OUT in cmd'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
150 self.steps.insert(0, (cmd, kind))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
151
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
152 def open(self, file, rw):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
153 """t.open(file, rw) returns a pipe or file object open for
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
154 reading or writing; the file is the other end of the pipeline."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
155 if rw == 'r':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
156 return self.open_r(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
157 if rw == 'w':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
158 return self.open_w(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
159 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
160 'Template.open: rw must be \'r\' or \'w\', not %r' % (rw,)
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
161
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
162 def open_r(self, file):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
163 """t.open_r(file) and t.open_w(file) implement
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
164 t.open(file, 'r') and t.open(file, 'w') respectively."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
165 if not self.steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
166 return open(file, 'r')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
167 if self.steps[-1][1] == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
168 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
169 'Template.open_r: pipeline ends width SINK'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
170 cmd = self.makepipeline(file, '')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
171 return os.popen(cmd, 'r')
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
172
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
173 def open_w(self, file):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
174 if not self.steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
175 return open(file, 'w')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
176 if self.steps[0][1] == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
177 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
178 'Template.open_w: pipeline begins with SOURCE'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
179 cmd = self.makepipeline('', file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
180 return os.popen(cmd, 'w')
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
181
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
182 def copy(self, infile, outfile):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
183 return os.system(self.makepipeline(infile, outfile))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
184
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
185 def makepipeline(self, infile, outfile):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
186 cmd = makepipeline(infile, self.steps, outfile)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
187 if self.debugging:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
188 print cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
189 cmd = 'set -x; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
190 return cmd
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
191
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
192
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
193 def makepipeline(infile, steps, outfile):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
194 # Build a list with for each command:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
195 # [input filename or '', command string, kind, output filename or '']
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
196
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
197 list = []
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
198 for cmd, kind in steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
199 list.append(['', cmd, kind, ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
200 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
201 # Make sure there is at least one step
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
202 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
203 if not list:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
204 list.append(['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
205 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
206 # Take care of the input and output ends
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
207 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
208 [cmd, kind] = list[0][1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
209 if kind[0] == 'f' and not infile:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
210 list.insert(0, ['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
211 list[0][0] = infile
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
212 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
213 [cmd, kind] = list[-1][1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
214 if kind[1] == 'f' and not outfile:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
215 list.append(['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
216 list[-1][-1] = outfile
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
217 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
218 # Invent temporary files to connect stages that need files
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
219 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
220 garbage = []
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
221 for i in range(1, len(list)):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
222 lkind = list[i-1][2]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
223 rkind = list[i][2]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
224 if lkind[1] == 'f' or rkind[0] == 'f':
24710
11c701038805 Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Guido van Rossum <guido@python.org>
parents: 18936
diff changeset
225 (fd, temp) = tempfile.mkstemp()
11c701038805 Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Guido van Rossum <guido@python.org>
parents: 18936
diff changeset
226 os.close(fd)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
227 garbage.append(temp)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
228 list[i-1][-1] = list[i][0] = temp
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
229 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
230 for item in list:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
231 [inf, cmd, kind, outf] = item
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
232 if kind[1] == 'f':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
233 cmd = 'OUT=' + quote(outf) + '; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
234 if kind[0] == 'f':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
235 cmd = 'IN=' + quote(inf) + '; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
236 if kind[0] == '-' and inf:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
237 cmd = cmd + ' <' + quote(inf)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
238 if kind[1] == '-' and outf:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
239 cmd = cmd + ' >' + quote(outf)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
240 item[1] = cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
241 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
242 cmdlist = list[0][1]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
243 for item in list[1:]:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
244 [cmd, kind] = item[1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
245 if item[0] == '':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
246 if 'f' in kind:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
247 cmd = '{ ' + cmd + '; }'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
248 cmdlist = cmdlist + ' |\n' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
249 else:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
250 cmdlist = cmdlist + '\n' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
251 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
252 if garbage:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
253 rmcmd = 'rm -f'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
254 for file in garbage:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
255 rmcmd = rmcmd + ' ' + quote(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
256 trapcmd = 'trap ' + quote(rmcmd + '; exit') + ' 1 2 3 13 14 15'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
257 cmdlist = trapcmd + '\n' + cmdlist + '\n' + rmcmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
258 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
259 return cmdlist
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
260
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
261
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
262 # Reliably quote a string as a single argument for /bin/sh
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
263
60914
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
264 # Safe unquoted
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
265 _safechars = frozenset(string.ascii_letters + string.digits + '@%_-+=:,./')
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
266
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
267 def quote(file):
60914
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
268 """Return a shell-escaped version of the file string."""
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
269 for c in file:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
270 if c not in _safechars:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
271 break
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
272 else:
59033
047f19977c42 * fix issue#7476
Jack Diederich <jackdied@gmail.com>
parents: 43042
diff changeset
273 if not file:
047f19977c42 * fix issue#7476
Jack Diederich <jackdied@gmail.com>
parents: 43042
diff changeset
274 return "''"
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
275 return file
60914
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
276 # use single quotes, and put single quotes into double quotes
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
277 # the string $'b is then quoted as '$'"'"'b'
587df0d74785 #7507: quote "!" in pipes.quote(); it is a special character for some shells.
Georg Brandl <georg@python.org>
parents: 59033
diff changeset
278 return "'" + file.replace("'", "'\"'\"'") + "'"