GRASS Programmer's Manual
6.4.2(2012)
|
00001 # subprocess - Subprocesses with accessible I/O streams 00002 # 00003 # For more information about this module, see PEP 324. 00004 # 00005 # This module should remain compatible with Python 2.2, see PEP 291. 00006 # 00007 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> 00008 # 00009 # Licensed to PSF under a Contributor Agreement. 00010 # See http://www.python.org/2.4/license for licensing details. 00011 00012 r"""!subprocess - Subprocesses with accessible I/O streams 00013 00014 This module allows you to spawn processes, connect to their 00015 input/output/error pipes, and obtain their return codes. This module 00016 intends to replace several other, older modules and functions, like: 00017 00018 os.system 00019 os.spawn* 00020 os.popen* 00021 popen2.* 00022 commands.* 00023 00024 Information about how the subprocess module can be used to replace these 00025 modules and functions can be found below. 00026 00027 00028 00029 Using the subprocess module 00030 =========================== 00031 This module defines one class called Popen: 00032 00033 class Popen(args, bufsize=0, executable=None, 00034 stdin=None, stdout=None, stderr=None, 00035 preexec_fn=None, close_fds=False, shell=False, 00036 cwd=None, env=None, universal_newlines=False, 00037 startupinfo=None, creationflags=0): 00038 00039 00040 Arguments are: 00041 00042 args should be a string, or a sequence of program arguments. The 00043 program to execute is normally the first item in the args sequence or 00044 string, but can be explicitly set by using the executable argument. 00045 00046 On UNIX, with shell=False (default): In this case, the Popen class 00047 uses os.execvp() to execute the child program. args should normally 00048 be a sequence. A string will be treated as a sequence with the string 00049 as the only item (the program to execute). 00050 00051 On UNIX, with shell=True: If args is a string, it specifies the 00052 command string to execute through the shell. If args is a sequence, 00053 the first item specifies the command string, and any additional items 00054 will be treated as additional shell arguments. 00055 00056 On Windows: the Popen class uses CreateProcess() to execute the child 00057 program, which operates on strings. If args is a sequence, it will be 00058 converted to a string using the list2cmdline method. Please note that 00059 not all MS Windows applications interpret the command line the same 00060 way: The list2cmdline is designed for applications using the same 00061 rules as the MS C runtime. 00062 00063 bufsize, if given, has the same meaning as the corresponding argument 00064 to the built-in open() function: 0 means unbuffered, 1 means line 00065 buffered, any other positive value means use a buffer of 00066 (approximately) that size. A negative bufsize means to use the system 00067 default, which usually means fully buffered. The default value for 00068 bufsize is 0 (unbuffered). 00069 00070 stdin, stdout and stderr specify the executed programs' standard 00071 input, standard output and standard error file handles, respectively. 00072 Valid values are PIPE, an existing file descriptor (a positive 00073 integer), an existing file object, and None. PIPE indicates that a 00074 new pipe to the child should be created. With None, no redirection 00075 will occur; the child's file handles will be inherited from the 00076 parent. Additionally, stderr can be STDOUT, which indicates that the 00077 stderr data from the applications should be captured into the same 00078 file handle as for stdout. 00079 00080 If preexec_fn is set to a callable object, this object will be called 00081 in the child process just before the child is executed. 00082 00083 If close_fds is true, all file descriptors except 0, 1 and 2 will be 00084 closed before the child process is executed. 00085 00086 if shell is true, the specified command will be executed through the 00087 shell. 00088 00089 If cwd is not None, the current directory will be changed to cwd 00090 before the child is executed. 00091 00092 If env is not None, it defines the environment variables for the new 00093 process. 00094 00095 If universal_newlines is true, the file objects stdout and stderr are 00096 opened as a text files, but lines may be terminated by any of '\n', 00097 the Unix end-of-line convention, '\r', the Macintosh convention or 00098 '\r\n', the Windows convention. All of these external representations 00099 are seen as '\n' by the Python program. Note: This feature is only 00100 available if Python is built with universal newline support (the 00101 default). Also, the newlines attribute of the file objects stdout, 00102 stdin and stderr are not updated by the communicate() method. 00103 00104 The startupinfo and creationflags, if given, will be passed to the 00105 underlying CreateProcess() function. They can specify things such as 00106 appearance of the main window and priority for the new process. 00107 (Windows only) 00108 00109 00110 This module also defines two shortcut functions: 00111 00112 call(*args, **kwargs): 00113 Run command with arguments. Wait for command to complete, then 00114 return the returncode attribute. 00115 00116 The arguments are the same as for the Popen constructor. Example: 00117 00118 retcode = call(["ls", "-l"]) 00119 00120 00121 Exceptions 00122 ---------- 00123 Exceptions raised in the child process, before the new program has 00124 started to execute, will be re-raised in the parent. Additionally, 00125 the exception object will have one extra attribute called 00126 'child_traceback', which is a string containing traceback information 00127 from the childs point of view. 00128 00129 The most common exception raised is OSError. This occurs, for 00130 example, when trying to execute a non-existent file. Applications 00131 should prepare for OSErrors. 00132 00133 A ValueError will be raised if Popen is called with invalid arguments. 00134 00135 00136 Security 00137 -------- 00138 Unlike some other popen functions, this implementation will never call 00139 /bin/sh implicitly. This means that all characters, including shell 00140 metacharacters, can safely be passed to child processes. 00141 00142 00143 Popen objects 00144 ============= 00145 Instances of the Popen class have the following methods: 00146 00147 poll() 00148 Check if child process has terminated. Returns returncode 00149 attribute. 00150 00151 wait() 00152 Wait for child process to terminate. Returns returncode attribute. 00153 00154 communicate(input=None) 00155 Interact with process: Send data to stdin. Read data from stdout 00156 and stderr, until end-of-file is reached. Wait for process to 00157 terminate. The optional stdin argument should be a string to be 00158 sent to the child process, or None, if no data should be sent to 00159 the child. 00160 00161 communicate() returns a tuple (stdout, stderr). 00162 00163 Note: The data read is buffered in memory, so do not use this 00164 method if the data size is large or unlimited. 00165 00166 The following attributes are also available: 00167 00168 stdin 00169 If the stdin argument is PIPE, this attribute is a file object 00170 that provides input to the child process. Otherwise, it is None. 00171 00172 stdout 00173 If the stdout argument is PIPE, this attribute is a file object 00174 that provides output from the child process. Otherwise, it is 00175 None. 00176 00177 stderr 00178 If the stderr argument is PIPE, this attribute is file object that 00179 provides error output from the child process. Otherwise, it is 00180 None. 00181 00182 pid 00183 The process ID of the child process. 00184 00185 returncode 00186 The child return code. A None value indicates that the process 00187 hasn't terminated yet. A negative value -N indicates that the 00188 child was terminated by signal N (UNIX only). 00189 00190 00191 Replacing older functions with the subprocess module 00192 ==================================================== 00193 In this section, "a ==> b" means that b can be used as a replacement 00194 for a. 00195 00196 Note: All functions in this section fail (more or less) silently if 00197 the executed program cannot be found; this module raises an OSError 00198 exception. 00199 00200 In the following examples, we assume that the subprocess module is 00201 imported with "from subprocess import *". 00202 00203 00204 Replacing /bin/sh shell backquote 00205 --------------------------------- 00206 output=`mycmd myarg` 00207 ==> 00208 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] 00209 00210 00211 Replacing shell pipe line 00212 ------------------------- 00213 output=`dmesg | grep hda` 00214 ==> 00215 p1 = Popen(["dmesg"], stdout=PIPE) 00216 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 00217 output = p2.communicate()[0] 00218 00219 00220 Replacing os.system() 00221 --------------------- 00222 sts = os.system("mycmd" + " myarg") 00223 ==> 00224 p = Popen("mycmd" + " myarg", shell=True) 00225 sts = os.waitpid(p.pid, 0) 00226 00227 Note: 00228 00229 * Calling the program through the shell is usually not required. 00230 00231 * It's easier to look at the returncode attribute than the 00232 exitstatus. 00233 00234 A more real-world example would look like this: 00235 00236 try: 00237 retcode = call("mycmd" + " myarg", shell=True) 00238 if retcode < 0: 00239 print >>sys.stderr, "Child was terminated by signal", -retcode 00240 else: 00241 print >>sys.stderr, "Child returned", retcode 00242 except OSError, e: 00243 print >>sys.stderr, "Execution failed:", e 00244 00245 00246 Replacing os.spawn* 00247 ------------------- 00248 P_NOWAIT example: 00249 00250 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") 00251 ==> 00252 pid = Popen(["/bin/mycmd", "myarg"]).pid 00253 00254 00255 P_WAIT example: 00256 00257 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") 00258 ==> 00259 retcode = call(["/bin/mycmd", "myarg"]) 00260 00261 00262 Vector example: 00263 00264 os.spawnvp(os.P_NOWAIT, path, args) 00265 ==> 00266 Popen([path] + args[1:]) 00267 00268 00269 Environment example: 00270 00271 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) 00272 ==> 00273 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) 00274 00275 00276 Replacing os.popen* 00277 ------------------- 00278 pipe = os.popen(cmd, mode='r', bufsize) 00279 ==> 00280 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout 00281 00282 pipe = os.popen(cmd, mode='w', bufsize) 00283 ==> 00284 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin 00285 00286 00287 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) 00288 ==> 00289 p = Popen(cmd, shell=True, bufsize=bufsize, 00290 stdin=PIPE, stdout=PIPE, close_fds=True) 00291 (child_stdin, child_stdout) = (p.stdin, p.stdout) 00292 00293 00294 (child_stdin, 00295 child_stdout, 00296 child_stderr) = os.popen3(cmd, mode, bufsize) 00297 ==> 00298 p = Popen(cmd, shell=True, bufsize=bufsize, 00299 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) 00300 (child_stdin, 00301 child_stdout, 00302 child_stderr) = (p.stdin, p.stdout, p.stderr) 00303 00304 00305 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) 00306 ==> 00307 p = Popen(cmd, shell=True, bufsize=bufsize, 00308 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) 00309 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) 00310 00311 00312 Replacing popen2.* 00313 ------------------ 00314 Note: If the cmd argument to popen2 functions is a string, the command 00315 is executed through /bin/sh. If it is a list, the command is directly 00316 executed. 00317 00318 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) 00319 ==> 00320 p = Popen(["somestring"], shell=True, bufsize=bufsize 00321 stdin=PIPE, stdout=PIPE, close_fds=True) 00322 (child_stdout, child_stdin) = (p.stdout, p.stdin) 00323 00324 00325 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) 00326 ==> 00327 p = Popen(["mycmd", "myarg"], bufsize=bufsize, 00328 stdin=PIPE, stdout=PIPE, close_fds=True) 00329 (child_stdout, child_stdin) = (p.stdout, p.stdin) 00330 00331 The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, 00332 except that: 00333 00334 * subprocess.Popen raises an exception if the execution fails 00335 * the capturestderr argument is replaced with the stderr argument. 00336 * stdin=PIPE and stdout=PIPE must be specified. 00337 * popen2 closes all filedescriptors by default, but you have to specify 00338 close_fds=True with subprocess.Popen. 00339 00340 00341 """ 00342 00343 import sys 00344 mswindows = (sys.platform == "win32") 00345 00346 import os 00347 import types 00348 import traceback 00349 00350 if mswindows: 00351 import threading 00352 import msvcrt 00353 if 0: # <-- change this to use pywin32 instead of the _subprocess driver 00354 import pywintypes 00355 from win32api import GetStdHandle, STD_INPUT_HANDLE, \ 00356 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE 00357 from win32api import GetCurrentProcess, DuplicateHandle, \ 00358 GetModuleFileName, GetVersion 00359 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE 00360 from win32pipe import CreatePipe 00361 from win32process import CreateProcess, STARTUPINFO, \ 00362 GetExitCodeProcess, STARTF_USESTDHANDLES, \ 00363 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE 00364 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0 00365 else: 00366 from _subprocess import * 00367 class STARTUPINFO: 00368 dwFlags = 0 00369 hStdInput = None 00370 hStdOutput = None 00371 hStdError = None 00372 class pywintypes: 00373 error = IOError 00374 else: 00375 import select 00376 import errno 00377 import fcntl 00378 import pickle 00379 00380 __all__ = ["Popen", "PIPE", "STDOUT", "call"] 00381 00382 try: 00383 MAXFD = os.sysconf("SC_OPEN_MAX") 00384 except: 00385 MAXFD = 256 00386 00387 # True/False does not exist on 2.2.0 00388 try: 00389 False 00390 except NameError: 00391 False = 0 00392 True = 1 00393 00394 _active = [] 00395 00396 def _cleanup(): 00397 for inst in _active[:]: 00398 inst.poll() 00399 00400 PIPE = -1 00401 STDOUT = -2 00402 00403 00404 def call(*args, **kwargs): 00405 """!Run command with arguments. Wait for command to complete, then 00406 return the returncode attribute. 00407 00408 The arguments are the same as for the Popen constructor. Example: 00409 00410 retcode = call(["ls", "-l"]) 00411 """ 00412 return Popen(*args, **kwargs).wait() 00413 00414 00415 def list2cmdline(seq): 00416 """ 00417 Translate a sequence of arguments into a command line 00418 string, using the same rules as the MS C runtime: 00419 00420 1) Arguments are delimited by white space, which is either a 00421 space or a tab. 00422 00423 2) A string surrounded by double quotation marks is 00424 interpreted as a single argument, regardless of white space 00425 contained within. A quoted string can be embedded in an 00426 argument. 00427 00428 3) A double quotation mark preceded by a backslash is 00429 interpreted as a literal double quotation mark. 00430 00431 4) Backslashes are interpreted literally, unless they 00432 immediately precede a double quotation mark. 00433 00434 5) If backslashes immediately precede a double quotation mark, 00435 every pair of backslashes is interpreted as a literal 00436 backslash. If the number of backslashes is odd, the last 00437 backslash escapes the next double quotation mark as 00438 described in rule 3. 00439 """ 00440 00441 # See 00442 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp 00443 result = [] 00444 needquote = False 00445 for arg in seq: 00446 bs_buf = [] 00447 00448 # Add a space to separate this argument from the others 00449 if result: 00450 result.append(' ') 00451 00452 needquote = (" " in arg) or ("\t" in arg) 00453 if needquote: 00454 result.append('"') 00455 00456 for c in arg: 00457 if c == '\\': 00458 # Don't know if we need to double yet. 00459 bs_buf.append(c) 00460 elif c == '"': 00461 # Double backspaces. 00462 result.append('\\' * len(bs_buf)*2) 00463 bs_buf = [] 00464 result.append('\\"') 00465 else: 00466 # Normal char 00467 if bs_buf: 00468 result.extend(bs_buf) 00469 bs_buf = [] 00470 result.append(c) 00471 00472 # Add remaining backspaces, if any. 00473 if bs_buf: 00474 result.extend(bs_buf) 00475 00476 if needquote: 00477 result.extend(bs_buf) 00478 result.append('"') 00479 00480 return ''.join(result) 00481 00482 00483 class Popen(object): 00484 def __init__(self, args, bufsize=0, executable=None, 00485 stdin=None, stdout=None, stderr=None, 00486 preexec_fn=None, close_fds=False, shell=False, 00487 cwd=None, env=None, universal_newlines=False, 00488 startupinfo=None, creationflags=0): 00489 """!Create new Popen instance.""" 00490 _cleanup() 00491 00492 if not isinstance(bufsize, (int, long)): 00493 raise TypeError("bufsize must be an integer") 00494 00495 if mswindows: 00496 if preexec_fn is not None: 00497 raise ValueError("preexec_fn is not supported on Windows " 00498 "platforms") 00499 if close_fds: 00500 raise ValueError("close_fds is not supported on Windows " 00501 "platforms") 00502 else: 00503 # POSIX 00504 if startupinfo is not None: 00505 raise ValueError("startupinfo is only supported on Windows " 00506 "platforms") 00507 if creationflags != 0: 00508 raise ValueError("creationflags is only supported on Windows " 00509 "platforms") 00510 00511 self.stdin = None 00512 self.stdout = None 00513 self.stderr = None 00514 self.pid = None 00515 self.returncode = None 00516 self.universal_newlines = universal_newlines 00517 00518 # Input and output objects. The general principle is like 00519 # this: 00520 # 00521 # Parent Child 00522 # ------ ----- 00523 # p2cwrite ---stdin---> p2cread 00524 # c2pread <--stdout--- c2pwrite 00525 # errread <--stderr--- errwrite 00526 # 00527 # On POSIX, the child objects are file descriptors. On 00528 # Windows, these are Windows file handles. The parent objects 00529 # are file descriptors on both platforms. The parent objects 00530 # are None when not using PIPEs. The child objects are None 00531 # when not redirecting. 00532 00533 (p2cread, p2cwrite, 00534 c2pread, c2pwrite, 00535 errread, errwrite) = self._get_handles(stdin, stdout, stderr) 00536 00537 self._execute_child(args, executable, preexec_fn, close_fds, 00538 cwd, env, universal_newlines, 00539 startupinfo, creationflags, shell, 00540 p2cread, p2cwrite, 00541 c2pread, c2pwrite, 00542 errread, errwrite) 00543 00544 if p2cwrite: 00545 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize) 00546 if c2pread: 00547 if universal_newlines: 00548 self.stdout = os.fdopen(c2pread, 'rU', bufsize) 00549 else: 00550 self.stdout = os.fdopen(c2pread, 'rb', bufsize) 00551 if errread: 00552 if universal_newlines: 00553 self.stderr = os.fdopen(errread, 'rU', bufsize) 00554 else: 00555 self.stderr = os.fdopen(errread, 'rb', bufsize) 00556 00557 _active.append(self) 00558 00559 00560 def _translate_newlines(self, data): 00561 data = data.replace("\r\n", "\n") 00562 data = data.replace("\r", "\n") 00563 return data 00564 00565 00566 if mswindows: 00567 # 00568 # Windows methods 00569 # 00570 def _get_handles(self, stdin, stdout, stderr): 00571 """!Construct and return tupel with IO objects: 00572 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 00573 """ 00574 if stdin == None and stdout == None and stderr == None: 00575 return (None, None, None, None, None, None) 00576 00577 p2cread, p2cwrite = None, None 00578 c2pread, c2pwrite = None, None 00579 errread, errwrite = None, None 00580 00581 if stdin == None: 00582 p2cread = GetStdHandle(STD_INPUT_HANDLE) 00583 elif stdin == PIPE: 00584 p2cread, p2cwrite = CreatePipe(None, 0) 00585 # Detach and turn into fd 00586 p2cwrite = p2cwrite.Detach() 00587 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0) 00588 elif type(stdin) == types.IntType: 00589 p2cread = msvcrt.get_osfhandle(stdin) 00590 else: 00591 # Assuming file-like object 00592 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 00593 p2cread = self._make_inheritable(p2cread) 00594 00595 if stdout == None: 00596 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE) 00597 elif stdout == PIPE: 00598 c2pread, c2pwrite = CreatePipe(None, 0) 00599 # Detach and turn into fd 00600 c2pread = c2pread.Detach() 00601 c2pread = msvcrt.open_osfhandle(c2pread, 0) 00602 elif type(stdout) == types.IntType: 00603 c2pwrite = msvcrt.get_osfhandle(stdout) 00604 else: 00605 # Assuming file-like object 00606 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 00607 c2pwrite = self._make_inheritable(c2pwrite) 00608 00609 if stderr == None: 00610 errwrite = GetStdHandle(STD_ERROR_HANDLE) 00611 elif stderr == PIPE: 00612 errread, errwrite = CreatePipe(None, 0) 00613 # Detach and turn into fd 00614 errread = errread.Detach() 00615 errread = msvcrt.open_osfhandle(errread, 0) 00616 elif stderr == STDOUT: 00617 errwrite = c2pwrite 00618 elif type(stderr) == types.IntType: 00619 errwrite = msvcrt.get_osfhandle(stderr) 00620 else: 00621 # Assuming file-like object 00622 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 00623 errwrite = self._make_inheritable(errwrite) 00624 00625 return (p2cread, p2cwrite, 00626 c2pread, c2pwrite, 00627 errread, errwrite) 00628 00629 00630 def _make_inheritable(self, handle): 00631 """!Return a duplicate of handle, which is inheritable""" 00632 return DuplicateHandle(GetCurrentProcess(), handle, 00633 GetCurrentProcess(), 0, 1, 00634 DUPLICATE_SAME_ACCESS) 00635 00636 00637 def _find_w9xpopen(self): 00638 """!Find and return absolut path to w9xpopen.exe""" 00639 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)), 00640 "w9xpopen.exe") 00641 if not os.path.exists(w9xpopen): 00642 # Eeek - file-not-found - possibly an embedding 00643 # situation - see if we can locate it in sys.exec_prefix 00644 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix), 00645 "w9xpopen.exe") 00646 if not os.path.exists(w9xpopen): 00647 raise RuntimeError("Cannot locate w9xpopen.exe, which is " 00648 "needed for Popen to work with your " 00649 "shell or platform.") 00650 return w9xpopen 00651 00652 00653 def _execute_child(self, args, executable, preexec_fn, close_fds, 00654 cwd, env, universal_newlines, 00655 startupinfo, creationflags, shell, 00656 p2cread, p2cwrite, 00657 c2pread, c2pwrite, 00658 errread, errwrite): 00659 """!Execute program (MS Windows version)""" 00660 00661 if not isinstance(args, types.StringTypes): 00662 args = list2cmdline(args) 00663 00664 # Process startup details 00665 default_startupinfo = STARTUPINFO() 00666 if startupinfo == None: 00667 startupinfo = default_startupinfo 00668 if not None in (p2cread, c2pwrite, errwrite): 00669 startupinfo.dwFlags |= STARTF_USESTDHANDLES 00670 startupinfo.hStdInput = p2cread 00671 startupinfo.hStdOutput = c2pwrite 00672 startupinfo.hStdError = errwrite 00673 00674 if shell: 00675 default_startupinfo.dwFlags |= STARTF_USESHOWWINDOW 00676 default_startupinfo.wShowWindow = SW_HIDE 00677 comspec = os.environ.get("COMSPEC", "cmd.exe") 00678 args = comspec + " /c " + args 00679 if (GetVersion() >= 0x80000000L or 00680 os.path.basename(comspec).lower() == "command.com"): 00681 # Win9x, or using command.com on NT. We need to 00682 # use the w9xpopen intermediate program. For more 00683 # information, see KB Q150956 00684 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp) 00685 w9xpopen = self._find_w9xpopen() 00686 args = '"%s" %s' % (w9xpopen, args) 00687 # Not passing CREATE_NEW_CONSOLE has been known to 00688 # cause random failures on win9x. Specifically a 00689 # dialog: "Your program accessed mem currently in 00690 # use at xxx" and a hopeful warning about the 00691 # stability of your system. Cost is Ctrl+C wont 00692 # kill children. 00693 creationflags |= CREATE_NEW_CONSOLE 00694 00695 # Start the process 00696 try: 00697 hp, ht, pid, tid = CreateProcess(executable, args, 00698 # no special security 00699 None, None, 00700 # must inherit handles to pass std 00701 # handles 00702 1, 00703 creationflags, 00704 env, 00705 cwd, 00706 startupinfo) 00707 except pywintypes.error, e: 00708 # Translate pywintypes.error to WindowsError, which is 00709 # a subclass of OSError. FIXME: We should really 00710 # translate errno using _sys_errlist (or simliar), but 00711 # how can this be done from Python? 00712 raise WindowsError(*e.args) 00713 00714 # Retain the process handle, but close the thread handle 00715 self._handle = hp 00716 self.pid = pid 00717 ht.Close() 00718 00719 # Child is launched. Close the parent's copy of those pipe 00720 # handles that only the child should have open. You need 00721 # to make sure that no handles to the write end of the 00722 # output pipe are maintained in this process or else the 00723 # pipe will not close when the child process exits and the 00724 # ReadFile will hang. 00725 if p2cread != None: 00726 p2cread.Close() 00727 if c2pwrite != None: 00728 c2pwrite.Close() 00729 if errwrite != None: 00730 errwrite.Close() 00731 00732 00733 def poll(self): 00734 """!Check if child process has terminated. Returns returncode 00735 attribute.""" 00736 if self.returncode == None: 00737 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0: 00738 self.returncode = GetExitCodeProcess(self._handle) 00739 _active.remove(self) 00740 return self.returncode 00741 00742 00743 def wait(self): 00744 """!Wait for child process to terminate. Returns returncode 00745 attribute.""" 00746 if self.returncode == None: 00747 obj = WaitForSingleObject(self._handle, INFINITE) 00748 self.returncode = GetExitCodeProcess(self._handle) 00749 _active.remove(self) 00750 return self.returncode 00751 00752 00753 def _readerthread(self, fh, buffer): 00754 buffer.append(fh.read()) 00755 00756 00757 def communicate(self, input=None): 00758 """!Interact with process: Send data to stdin. Read data from 00759 stdout and stderr, until end-of-file is reached. Wait for 00760 process to terminate. The optional input argument should be a 00761 string to be sent to the child process, or None, if no data 00762 should be sent to the child. 00763 00764 communicate() returns a tuple (stdout, stderr).""" 00765 stdout = None # Return 00766 stderr = None # Return 00767 00768 if self.stdout: 00769 stdout = [] 00770 stdout_thread = threading.Thread(target=self._readerthread, 00771 args=(self.stdout, stdout)) 00772 stdout_thread.setDaemon(True) 00773 stdout_thread.start() 00774 if self.stderr: 00775 stderr = [] 00776 stderr_thread = threading.Thread(target=self._readerthread, 00777 args=(self.stderr, stderr)) 00778 stderr_thread.setDaemon(True) 00779 stderr_thread.start() 00780 00781 if self.stdin: 00782 if input != None: 00783 self.stdin.write(input) 00784 self.stdin.close() 00785 00786 if self.stdout: 00787 stdout_thread.join() 00788 if self.stderr: 00789 stderr_thread.join() 00790 00791 # All data exchanged. Translate lists into strings. 00792 if stdout != None: 00793 stdout = stdout[0] 00794 if stderr != None: 00795 stderr = stderr[0] 00796 00797 # Translate newlines, if requested. We cannot let the file 00798 # object do the translation: It is based on stdio, which is 00799 # impossible to combine with select (unless forcing no 00800 # buffering). 00801 if self.universal_newlines and hasattr(open, 'newlines'): 00802 if stdout: 00803 stdout = self._translate_newlines(stdout) 00804 if stderr: 00805 stderr = self._translate_newlines(stderr) 00806 00807 self.wait() 00808 return (stdout, stderr) 00809 00810 else: 00811 # 00812 # POSIX methods 00813 # 00814 def _get_handles(self, stdin, stdout, stderr): 00815 """!Construct and return tupel with IO objects: 00816 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 00817 """ 00818 p2cread, p2cwrite = None, None 00819 c2pread, c2pwrite = None, None 00820 errread, errwrite = None, None 00821 00822 if stdin == None: 00823 pass 00824 elif stdin == PIPE: 00825 p2cread, p2cwrite = os.pipe() 00826 elif type(stdin) == types.IntType: 00827 p2cread = stdin 00828 else: 00829 # Assuming file-like object 00830 p2cread = stdin.fileno() 00831 00832 if stdout == None: 00833 pass 00834 elif stdout == PIPE: 00835 c2pread, c2pwrite = os.pipe() 00836 elif type(stdout) == types.IntType: 00837 c2pwrite = stdout 00838 else: 00839 # Assuming file-like object 00840 c2pwrite = stdout.fileno() 00841 00842 if stderr == None: 00843 pass 00844 elif stderr == PIPE: 00845 errread, errwrite = os.pipe() 00846 elif stderr == STDOUT: 00847 errwrite = c2pwrite 00848 elif type(stderr) == types.IntType: 00849 errwrite = stderr 00850 else: 00851 # Assuming file-like object 00852 errwrite = stderr.fileno() 00853 00854 return (p2cread, p2cwrite, 00855 c2pread, c2pwrite, 00856 errread, errwrite) 00857 00858 00859 def _set_cloexec_flag(self, fd): 00860 try: 00861 cloexec_flag = fcntl.FD_CLOEXEC 00862 except AttributeError: 00863 cloexec_flag = 1 00864 00865 old = fcntl.fcntl(fd, fcntl.F_GETFD) 00866 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag) 00867 00868 00869 def _close_fds(self, but): 00870 for i in range(3, MAXFD): 00871 if i == but: 00872 continue 00873 try: 00874 os.close(i) 00875 except: 00876 pass 00877 00878 00879 def _execute_child(self, args, executable, preexec_fn, close_fds, 00880 cwd, env, universal_newlines, 00881 startupinfo, creationflags, shell, 00882 p2cread, p2cwrite, 00883 c2pread, c2pwrite, 00884 errread, errwrite): 00885 """!Execute program (POSIX version)""" 00886 00887 if isinstance(args, types.StringTypes): 00888 args = [args] 00889 00890 if shell: 00891 args = ["/bin/sh", "-c"] + args 00892 00893 if executable == None: 00894 executable = args[0] 00895 00896 # For transferring possible exec failure from child to parent 00897 # The first char specifies the exception type: 0 means 00898 # OSError, 1 means some other error. 00899 errpipe_read, errpipe_write = os.pipe() 00900 self._set_cloexec_flag(errpipe_write) 00901 00902 self.pid = os.fork() 00903 if self.pid == 0: 00904 # Child 00905 try: 00906 # Close parent's pipe ends 00907 if p2cwrite: 00908 os.close(p2cwrite) 00909 if c2pread: 00910 os.close(c2pread) 00911 if errread: 00912 os.close(errread) 00913 os.close(errpipe_read) 00914 00915 # Dup fds for child 00916 if p2cread: 00917 os.dup2(p2cread, 0) 00918 if c2pwrite: 00919 os.dup2(c2pwrite, 1) 00920 if errwrite: 00921 os.dup2(errwrite, 2) 00922 00923 # Close pipe fds. Make sure we doesn't close the same 00924 # fd more than once. 00925 if p2cread: 00926 os.close(p2cread) 00927 if c2pwrite and c2pwrite not in (p2cread,): 00928 os.close(c2pwrite) 00929 if errwrite and errwrite not in (p2cread, c2pwrite): 00930 os.close(errwrite) 00931 00932 # Close all other fds, if asked for 00933 if close_fds: 00934 self._close_fds(but=errpipe_write) 00935 00936 if cwd != None: 00937 os.chdir(cwd) 00938 00939 if preexec_fn: 00940 apply(preexec_fn) 00941 00942 if env == None: 00943 os.execvp(executable, args) 00944 else: 00945 os.execvpe(executable, args, env) 00946 00947 except: 00948 exc_type, exc_value, tb = sys.exc_info() 00949 # Save the traceback and attach it to the exception object 00950 exc_lines = traceback.format_exception(exc_type, 00951 exc_value, 00952 tb) 00953 exc_value.child_traceback = ''.join(exc_lines) 00954 os.write(errpipe_write, pickle.dumps(exc_value)) 00955 00956 # This exitcode won't be reported to applications, so it 00957 # really doesn't matter what we return. 00958 os._exit(255) 00959 00960 # Parent 00961 os.close(errpipe_write) 00962 if p2cread and p2cwrite: 00963 os.close(p2cread) 00964 if c2pwrite and c2pread: 00965 os.close(c2pwrite) 00966 if errwrite and errread: 00967 os.close(errwrite) 00968 00969 # Wait for exec to fail or succeed; possibly raising exception 00970 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB 00971 os.close(errpipe_read) 00972 if data != "": 00973 os.waitpid(self.pid, 0) 00974 child_exception = pickle.loads(data) 00975 raise child_exception 00976 00977 00978 def _handle_exitstatus(self, sts): 00979 if os.WIFSIGNALED(sts): 00980 self.returncode = -os.WTERMSIG(sts) 00981 elif os.WIFEXITED(sts): 00982 self.returncode = os.WEXITSTATUS(sts) 00983 else: 00984 # Should never happen 00985 raise RuntimeError("Unknown child exit status!") 00986 00987 _active.remove(self) 00988 00989 00990 def poll(self): 00991 """!Check if child process has terminated. Returns returncode 00992 attribute.""" 00993 if self.returncode == None: 00994 try: 00995 pid, sts = os.waitpid(self.pid, os.WNOHANG) 00996 if pid == self.pid: 00997 self._handle_exitstatus(sts) 00998 except os.error: 00999 pass 01000 return self.returncode 01001 01002 01003 def wait(self): 01004 """!Wait for child process to terminate. Returns returncode 01005 attribute.""" 01006 if self.returncode == None: 01007 pid, sts = os.waitpid(self.pid, 0) 01008 self._handle_exitstatus(sts) 01009 return self.returncode 01010 01011 01012 def communicate(self, input=None): 01013 """!Interact with process: Send data to stdin. Read data from 01014 stdout and stderr, until end-of-file is reached. Wait for 01015 process to terminate. The optional input argument should be a 01016 string to be sent to the child process, or None, if no data 01017 should be sent to the child. 01018 01019 communicate() returns a tuple (stdout, stderr).""" 01020 read_set = [] 01021 write_set = [] 01022 stdout = None # Return 01023 stderr = None # Return 01024 01025 if self.stdin: 01026 # Flush stdio buffer. This might block, if the user has 01027 # been writing to .stdin in an uncontrolled fashion. 01028 self.stdin.flush() 01029 if input: 01030 write_set.append(self.stdin) 01031 else: 01032 self.stdin.close() 01033 if self.stdout: 01034 read_set.append(self.stdout) 01035 stdout = [] 01036 if self.stderr: 01037 read_set.append(self.stderr) 01038 stderr = [] 01039 01040 while read_set or write_set: 01041 rlist, wlist, xlist = select.select(read_set, write_set, []) 01042 01043 if self.stdin in wlist: 01044 # When select has indicated that the file is writable, 01045 # we can write up to PIPE_BUF bytes without risk 01046 # blocking. POSIX defines PIPE_BUF >= 512 01047 bytes_written = os.write(self.stdin.fileno(), input[:512]) 01048 input = input[bytes_written:] 01049 if not input: 01050 self.stdin.close() 01051 write_set.remove(self.stdin) 01052 01053 if self.stdout in rlist: 01054 data = os.read(self.stdout.fileno(), 1024) 01055 if data == "": 01056 self.stdout.close() 01057 read_set.remove(self.stdout) 01058 stdout.append(data) 01059 01060 if self.stderr in rlist: 01061 data = os.read(self.stderr.fileno(), 1024) 01062 if data == "": 01063 self.stderr.close() 01064 read_set.remove(self.stderr) 01065 stderr.append(data) 01066 01067 # All data exchanged. Translate lists into strings. 01068 if stdout != None: 01069 stdout = ''.join(stdout) 01070 if stderr != None: 01071 stderr = ''.join(stderr) 01072 01073 # Translate newlines, if requested. We cannot let the file 01074 # object do the translation: It is based on stdio, which is 01075 # impossible to combine with select (unless forcing no 01076 # buffering). 01077 if self.universal_newlines and hasattr(open, 'newlines'): 01078 if stdout: 01079 stdout = self._translate_newlines(stdout) 01080 if stderr: 01081 stderr = self._translate_newlines(stderr) 01082 01083 self.wait() 01084 return (stdout, stderr) 01085 01086 01087 def _demo_posix(): 01088 # 01089 # Example 1: Simple redirection: Get process list 01090 # 01091 plist = Popen(["ps"], stdout=PIPE).communicate()[0] 01092 print "Process list:" 01093 print plist 01094 01095 # 01096 # Example 2: Change uid before executing child 01097 # 01098 if os.getuid() == 0: 01099 p = Popen(["id"], preexec_fn=lambda: os.setuid(100)) 01100 p.wait() 01101 01102 # 01103 # Example 3: Connecting several subprocesses 01104 # 01105 print "Looking for 'hda'..." 01106 p1 = Popen(["dmesg"], stdout=PIPE) 01107 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 01108 print repr(p2.communicate()[0]) 01109 01110 # 01111 # Example 4: Catch execution error 01112 # 01113 print 01114 print "Trying a weird file..." 01115 try: 01116 print Popen(["/this/path/does/not/exist"]).communicate() 01117 except OSError, e: 01118 if e.errno == errno.ENOENT: 01119 print "The file didn't exist. I thought so..." 01120 print "Child traceback:" 01121 print e.child_traceback 01122 else: 01123 print "Error", e.errno 01124 else: 01125 print >>sys.stderr, "Gosh. No error." 01126 01127 01128 def _demo_windows(): 01129 # 01130 # Example 1: Connecting several subprocesses 01131 # 01132 print "Looking for 'PROMPT' in set output..." 01133 p1 = Popen("set", stdout=PIPE, shell=True) 01134 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE) 01135 print repr(p2.communicate()[0]) 01136 01137 # 01138 # Example 2: Simple execution of program 01139 # 01140 print "Executing calc..." 01141 p = Popen("calc") 01142 p.wait() 01143 01144 01145 if __name__ == "__main__": 01146 if mswindows: 01147 _demo_windows() 01148 else: 01149 _demo_posix()