Mercurial > cpython-lru_cache-weakref

annotate Lib/pipes.py @ 54294:dba767bc3155 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
remove mention of old ctypes version
author Benjamin Peterson <benjamin@python.org>
date Sat, 23 May 2009 20:59:09 +0000
parents 8b0b1146d908
children 047f19977c42
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
1 """Conversion pipeline templates.
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
3 The problem:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
4 ------------
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
5
18778
23ae47e00119 Use string.ascii_letters instead of string.letters (SF bug #226706).
Fred Drake <fdrake@acm.org>
parents: 17622
diff changeset
6 Suppose you have some data that you want to convert to another format,
23ae47e00119 Use string.ascii_letters instead of string.letters (SF bug #226706).
Fred Drake <fdrake@acm.org>
parents: 17622
diff changeset
7 such as from GIF image format to PPM image format. Maybe the
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
8 conversion involves several steps (e.g. piping it through compress or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
9 uuencode). Some of the conversion steps may require that their input
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
10 is a disk file, others may be able to read standard input; similar for
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
11 their output. The input to the entire conversion may also be read
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
12 from a disk file or from an open file, and similar for its output.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
13
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
14 The module lets you construct a pipeline template by sticking one or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
15 more conversion steps together. It will take care of creating and
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
16 removing temporary files if they are necessary to hold intermediate
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
17 data. You can then use the template to do conversions from many
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
18 different sources to many different destinations. The temporary
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
19 file names used are different each time the template is used.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
20
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
21 The templates are objects so you can create templates for many
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
22 different conversion steps and store them in a dictionary, for
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
23 instance.
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
24
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
25
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
26 Directions:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
27 -----------
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
28
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
29 To create a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
30 t = Template()
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
31
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
32 To add a conversion step to a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
33 t.append(command, kind)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
34 where kind is a string of two characters: the first is '-' if the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
35 command reads its standard input or 'f' if it requires a file; the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
36 second likewise for the output. The command must be valid /bin/sh
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
37 syntax. If input or output files are required, they are passed as
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
38 $IN and $OUT; otherwise, it must be possible to use the command in
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
39 a pipeline.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
40
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
41 To add a conversion step at the beginning:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
42 t.prepend(command, kind)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
43
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
44 To convert a file to another file using a template:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
45 sts = t.copy(infile, outfile)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
46 If infile or outfile are the empty string, standard input is read or
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
47 standard output is written, respectively. The return value is the
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
48 exit status of the conversion pipeline.
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
49
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
50 To open a file for reading or writing through a conversion pipeline:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
51 fp = t.open(file, mode)
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
52 where mode is 'r' to read the file, or 'w' to write it -- just like
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
53 for the built-in function open() or for os.popen().
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
54
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
55 To create a new template object initialized to a given one:
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
56 t2 = t.clone()
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
57
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
58 For an example, see the function test() at the end of the file.
16474
4d406c26c16e a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15870
diff changeset
59 """ # '
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
60
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
61
6519
24da035ceb6e Convert all remaining *simple* cases of regex usage to re usage.
Guido van Rossum <guido@python.org>
parents: 1435
diff changeset
62 import re
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 import os
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
64 import tempfile
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
65 import string
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
66
16474
4d406c26c16e a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15870
diff changeset
67 __all__ = ["Template"]
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
69 # Conversion step kinds
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
71 FILEIN_FILEOUT = 'ff' # Must read & write real files
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
72 STDIN_FILEOUT = '-f' # Must write a real file
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
73 FILEIN_STDOUT = 'f-' # Must read a real file
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
74 STDIN_STDOUT = '--' # Normal pipeline element
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
75 SOURCE = '.-' # Must be first, writes stdout
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
76 SINK = '-.' # Must be last, reads stdin
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
77
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 stepkinds = [FILEIN_FILEOUT, STDIN_FILEOUT, FILEIN_STDOUT, STDIN_STDOUT, \
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
79 SOURCE, SINK]
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
80
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
81
11706
b6f52c1ef6da More trivial comment -> docstring transformations by Ka-Ping Yee,
Guido van Rossum <guido@python.org>
parents: 10825
diff changeset
82 class Template:
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
83 """Class representing a pipeline template."""
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
84
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
85 def __init__(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
86 """Template() returns a fresh pipeline template."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
87 self.debugging = 0
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
88 self.reset()
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
90 def __repr__(self):
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
91 """t.__repr__() implements repr(t)."""
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
92 return '<Template instance, steps=%r>' % (self.steps,)
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
93
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
94 def reset(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
95 """t.reset() restores a pipeline template to its initial state."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
96 self.steps = []
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
97
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
98 def clone(self):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
99 """t.clone() returns a new pipeline template with identical
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
100 initial state as the current one."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
101 t = Template()
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
102 t.steps = self.steps[:]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
103 t.debugging = self.debugging
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
104 return t
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
105
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
106 def debug(self, flag):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
107 """t.debug(flag) turns debugging on or off."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
108 self.debugging = flag
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
109
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
110 def append(self, cmd, kind):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
111 """t.append(cmd, kind) adds a new step at the end."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
112 if type(cmd) is not type(''):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
113 raise TypeError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
114 'Template.append: cmd must be a string'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
115 if kind not in stepkinds:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
116 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
117 'Template.append: bad kind %r' % (kind,)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
118 if kind == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
119 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
120 'Template.append: SOURCE can only be prepended'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
121 if self.steps and self.steps[-1][1] == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
122 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
123 'Template.append: already ends with SINK'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
124 if kind[0] == 'f' and not re.search(r'\$IN\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
125 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
126 'Template.append: missing $IN in cmd'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
127 if kind[1] == 'f' and not re.search(r'\$OUT\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
128 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
129 'Template.append: missing $OUT in cmd'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
130 self.steps.append((cmd, kind))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
131
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
132 def prepend(self, cmd, kind):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
133 """t.prepend(cmd, kind) adds a new step at the front."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
134 if type(cmd) is not type(''):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
135 raise TypeError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
136 'Template.prepend: cmd must be a string'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
137 if kind not in stepkinds:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
138 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
139 'Template.prepend: bad kind %r' % (kind,)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
140 if kind == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
141 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
142 'Template.prepend: SINK can only be appended'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
143 if self.steps and self.steps[0][1] == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
144 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
145 'Template.prepend: already begins with SOURCE'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
146 if kind[0] == 'f' and not re.search(r'\$IN\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
147 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
148 'Template.prepend: missing $IN in cmd'
17622
44a0a9a277ef SF bug 418615: regular expression bug in pipes.py.
Tim Peters <tim.peters@gmail.com>
parents: 16474
diff changeset
149 if kind[1] == 'f' and not re.search(r'\$OUT\b', cmd):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
150 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
151 'Template.prepend: missing $OUT in cmd'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
152 self.steps.insert(0, (cmd, kind))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
153
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
154 def open(self, file, rw):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
155 """t.open(file, rw) returns a pipe or file object open for
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
156 reading or writing; the file is the other end of the pipeline."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
157 if rw == 'r':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
158 return self.open_r(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
159 if rw == 'w':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
160 return self.open_w(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
161 raise ValueError, \
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 24710
diff changeset
162 'Template.open: rw must be \'r\' or \'w\', not %r' % (rw,)
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
163
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
164 def open_r(self, file):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
165 """t.open_r(file) and t.open_w(file) implement
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
166 t.open(file, 'r') and t.open(file, 'w') respectively."""
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
167 if not self.steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
168 return open(file, 'r')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
169 if self.steps[-1][1] == SINK:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
170 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
171 'Template.open_r: pipeline ends width SINK'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
172 cmd = self.makepipeline(file, '')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
173 return os.popen(cmd, 'r')
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
174
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
175 def open_w(self, file):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
176 if not self.steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
177 return open(file, 'w')
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
178 if self.steps[0][1] == SOURCE:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
179 raise ValueError, \
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
180 'Template.open_w: pipeline begins with SOURCE'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
181 cmd = self.makepipeline('', file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
182 return os.popen(cmd, 'w')
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
183
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
184 def copy(self, infile, outfile):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
185 return os.system(self.makepipeline(infile, outfile))
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
186
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
187 def makepipeline(self, infile, outfile):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
188 cmd = makepipeline(infile, self.steps, outfile)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
189 if self.debugging:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
190 print cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
191 cmd = 'set -x; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
192 return cmd
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
193
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
194
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
195 def makepipeline(infile, steps, outfile):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
196 # Build a list with for each command:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
197 # [input filename or '', command string, kind, output filename or '']
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
198
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
199 list = []
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
200 for cmd, kind in steps:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
201 list.append(['', cmd, kind, ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
202 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
203 # Make sure there is at least one step
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
204 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
205 if not list:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
206 list.append(['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
207 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
208 # Take care of the input and output ends
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
209 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
210 [cmd, kind] = list[0][1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
211 if kind[0] == 'f' and not infile:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
212 list.insert(0, ['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
213 list[0][0] = infile
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
214 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
215 [cmd, kind] = list[-1][1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
216 if kind[1] == 'f' and not outfile:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
217 list.append(['', 'cat', '--', ''])
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
218 list[-1][-1] = outfile
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
219 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
220 # Invent temporary files to connect stages that need files
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
221 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
222 garbage = []
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
223 for i in range(1, len(list)):
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
224 lkind = list[i-1][2]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
225 rkind = list[i][2]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
226 if lkind[1] == 'f' or rkind[0] == 'f':
24710
11c701038805 Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Guido van Rossum <guido@python.org>
parents: 18936
diff changeset
227 (fd, temp) = tempfile.mkstemp()
11c701038805 Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Guido van Rossum <guido@python.org>
parents: 18936
diff changeset
228 os.close(fd)
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
229 garbage.append(temp)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
230 list[i-1][-1] = list[i][0] = temp
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
231 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
232 for item in list:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
233 [inf, cmd, kind, outf] = item
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
234 if kind[1] == 'f':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
235 cmd = 'OUT=' + quote(outf) + '; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
236 if kind[0] == 'f':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
237 cmd = 'IN=' + quote(inf) + '; ' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
238 if kind[0] == '-' and inf:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
239 cmd = cmd + ' <' + quote(inf)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
240 if kind[1] == '-' and outf:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
241 cmd = cmd + ' >' + quote(outf)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
242 item[1] = cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
243 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
244 cmdlist = list[0][1]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
245 for item in list[1:]:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
246 [cmd, kind] = item[1:3]
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
247 if item[0] == '':
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
248 if 'f' in kind:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
249 cmd = '{ ' + cmd + '; }'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
250 cmdlist = cmdlist + ' |\n' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
251 else:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
252 cmdlist = cmdlist + '\n' + cmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
253 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
254 if garbage:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
255 rmcmd = 'rm -f'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
256 for file in garbage:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
257 rmcmd = rmcmd + ' ' + quote(file)
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
258 trapcmd = 'trap ' + quote(rmcmd + '; exit') + ' 1 2 3 13 14 15'
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
259 cmdlist = trapcmd + '\n' + cmdlist + '\n' + rmcmd
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
260 #
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
261 return cmdlist
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
262
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
263
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
264 # Reliably quote a string as a single argument for /bin/sh
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
265
18778
23ae47e00119 Use string.ascii_letters instead of string.letters (SF bug #226706).
Fred Drake <fdrake@acm.org>
parents: 17622
diff changeset
266 _safechars = string.ascii_letters + string.digits + '!@%_-+=:,./' # Safe unquoted
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
267 _funnychars = '"`$\\' # Unsafe inside "double quotes"
1078
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
268
245003c006a6 Misc changes and new modules. whrandom is "objectified". SOCKET.py
Guido van Rossum <guido@python.org>
parents:
diff changeset
269 def quote(file):
15870
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
270 for c in file:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
271 if c not in _safechars:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
272 break
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
273 else:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
274 return file
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
275 if '\'' not in file:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
276 return '\'' + file + '\''
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
277 res = ''
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
278 for c in file:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
279 if c in _funnychars:
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
280 c = '\\' + c
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
281 res = res + c
010d03853546 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15543
diff changeset
282 return '"' + res + '"'