GRASS Programmer's Manual  6.4.2(2012)
subprocess.py
Go to the documentation of this file.
00001 # subprocess - Subprocesses with accessible I/O streams
00002 #
00003 # For more information about this module, see PEP 324.
00004 #
00005 # This module should remain compatible with Python 2.2, see PEP 291.
00006 #
00007 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
00008 #
00009 # Licensed to PSF under a Contributor Agreement.
00010 # See http://www.python.org/2.4/license for licensing details.
00011 
00012 r"""!subprocess - Subprocesses with accessible I/O streams
00013 
00014 This module allows you to spawn processes, connect to their
00015 input/output/error pipes, and obtain their return codes.  This module
00016 intends to replace several other, older modules and functions, like:
00017 
00018 os.system
00019 os.spawn*
00020 os.popen*
00021 popen2.*
00022 commands.*
00023 
00024 Information about how the subprocess module can be used to replace these
00025 modules and functions can be found below.
00026 
00027 
00028 
00029 Using the subprocess module
00030 ===========================
00031 This module defines one class called Popen:
00032 
00033 class Popen(args, bufsize=0, executable=None,
00034             stdin=None, stdout=None, stderr=None,
00035             preexec_fn=None, close_fds=False, shell=False,
00036             cwd=None, env=None, universal_newlines=False,
00037             startupinfo=None, creationflags=0):
00038 
00039 
00040 Arguments are:
00041 
00042 args should be a string, or a sequence of program arguments.  The
00043 program to execute is normally the first item in the args sequence or
00044 string, but can be explicitly set by using the executable argument.
00045 
00046 On UNIX, with shell=False (default): In this case, the Popen class
00047 uses os.execvp() to execute the child program.  args should normally
00048 be a sequence.  A string will be treated as a sequence with the string
00049 as the only item (the program to execute).
00050 
00051 On UNIX, with shell=True: If args is a string, it specifies the
00052 command string to execute through the shell.  If args is a sequence,
00053 the first item specifies the command string, and any additional items
00054 will be treated as additional shell arguments.
00055 
00056 On Windows: the Popen class uses CreateProcess() to execute the child
00057 program, which operates on strings.  If args is a sequence, it will be
00058 converted to a string using the list2cmdline method.  Please note that
00059 not all MS Windows applications interpret the command line the same
00060 way: The list2cmdline is designed for applications using the same
00061 rules as the MS C runtime.
00062 
00063 bufsize, if given, has the same meaning as the corresponding argument
00064 to the built-in open() function: 0 means unbuffered, 1 means line
00065 buffered, any other positive value means use a buffer of
00066 (approximately) that size.  A negative bufsize means to use the system
00067 default, which usually means fully buffered.  The default value for
00068 bufsize is 0 (unbuffered).
00069 
00070 stdin, stdout and stderr specify the executed programs' standard
00071 input, standard output and standard error file handles, respectively.
00072 Valid values are PIPE, an existing file descriptor (a positive
00073 integer), an existing file object, and None.  PIPE indicates that a
00074 new pipe to the child should be created.  With None, no redirection
00075 will occur; the child's file handles will be inherited from the
00076 parent.  Additionally, stderr can be STDOUT, which indicates that the
00077 stderr data from the applications should be captured into the same
00078 file handle as for stdout.
00079 
00080 If preexec_fn is set to a callable object, this object will be called
00081 in the child process just before the child is executed.
00082 
00083 If close_fds is true, all file descriptors except 0, 1 and 2 will be
00084 closed before the child process is executed.
00085 
00086 if shell is true, the specified command will be executed through the
00087 shell.
00088 
00089 If cwd is not None, the current directory will be changed to cwd
00090 before the child is executed.
00091 
00092 If env is not None, it defines the environment variables for the new
00093 process.
00094 
00095 If universal_newlines is true, the file objects stdout and stderr are
00096 opened as a text files, but lines may be terminated by any of '\n',
00097 the Unix end-of-line convention, '\r', the Macintosh convention or
00098 '\r\n', the Windows convention.  All of these external representations
00099 are seen as '\n' by the Python program.  Note: This feature is only
00100 available if Python is built with universal newline support (the
00101 default).  Also, the newlines attribute of the file objects stdout,
00102 stdin and stderr are not updated by the communicate() method.
00103 
00104 The startupinfo and creationflags, if given, will be passed to the
00105 underlying CreateProcess() function.  They can specify things such as
00106 appearance of the main window and priority for the new process.
00107 (Windows only)
00108 
00109 
00110 This module also defines two shortcut functions:
00111 
00112 call(*args, **kwargs):
00113     Run command with arguments.  Wait for command to complete, then
00114     return the returncode attribute.
00115 
00116     The arguments are the same as for the Popen constructor.  Example:
00117 
00118     retcode = call(["ls", "-l"])
00119 
00120 
00121 Exceptions
00122 ----------
00123 Exceptions raised in the child process, before the new program has
00124 started to execute, will be re-raised in the parent.  Additionally,
00125 the exception object will have one extra attribute called
00126 'child_traceback', which is a string containing traceback information
00127 from the childs point of view.
00128 
00129 The most common exception raised is OSError.  This occurs, for
00130 example, when trying to execute a non-existent file.  Applications
00131 should prepare for OSErrors.
00132 
00133 A ValueError will be raised if Popen is called with invalid arguments.
00134 
00135 
00136 Security
00137 --------
00138 Unlike some other popen functions, this implementation will never call
00139 /bin/sh implicitly.  This means that all characters, including shell
00140 metacharacters, can safely be passed to child processes.
00141 
00142 
00143 Popen objects
00144 =============
00145 Instances of the Popen class have the following methods:
00146 
00147 poll()
00148     Check if child process has terminated.  Returns returncode
00149     attribute.
00150 
00151 wait()
00152     Wait for child process to terminate.  Returns returncode attribute.
00153 
00154 communicate(input=None)
00155     Interact with process: Send data to stdin.  Read data from stdout
00156     and stderr, until end-of-file is reached.  Wait for process to
00157     terminate.  The optional stdin argument should be a string to be
00158     sent to the child process, or None, if no data should be sent to
00159     the child.
00160 
00161     communicate() returns a tuple (stdout, stderr).
00162 
00163     Note: The data read is buffered in memory, so do not use this
00164     method if the data size is large or unlimited.
00165 
00166 The following attributes are also available:
00167 
00168 stdin
00169     If the stdin argument is PIPE, this attribute is a file object
00170     that provides input to the child process.  Otherwise, it is None.
00171 
00172 stdout
00173     If the stdout argument is PIPE, this attribute is a file object
00174     that provides output from the child process.  Otherwise, it is
00175     None.
00176 
00177 stderr
00178     If the stderr argument is PIPE, this attribute is file object that
00179     provides error output from the child process.  Otherwise, it is
00180     None.
00181 
00182 pid
00183     The process ID of the child process.
00184 
00185 returncode
00186     The child return code.  A None value indicates that the process
00187     hasn't terminated yet.  A negative value -N indicates that the
00188     child was terminated by signal N (UNIX only).
00189 
00190 
00191 Replacing older functions with the subprocess module
00192 ====================================================
00193 In this section, "a ==> b" means that b can be used as a replacement
00194 for a.
00195 
00196 Note: All functions in this section fail (more or less) silently if
00197 the executed program cannot be found; this module raises an OSError
00198 exception.
00199 
00200 In the following examples, we assume that the subprocess module is
00201 imported with "from subprocess import *".
00202 
00203 
00204 Replacing /bin/sh shell backquote
00205 ---------------------------------
00206 output=`mycmd myarg`
00207 ==>
00208 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
00209 
00210 
00211 Replacing shell pipe line
00212 -------------------------
00213 output=`dmesg | grep hda`
00214 ==>
00215 p1 = Popen(["dmesg"], stdout=PIPE)
00216 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
00217 output = p2.communicate()[0]
00218 
00219 
00220 Replacing os.system()
00221 ---------------------
00222 sts = os.system("mycmd" + " myarg")
00223 ==>
00224 p = Popen("mycmd" + " myarg", shell=True)
00225 sts = os.waitpid(p.pid, 0)
00226 
00227 Note:
00228 
00229 * Calling the program through the shell is usually not required.
00230 
00231 * It's easier to look at the returncode attribute than the
00232   exitstatus.
00233 
00234 A more real-world example would look like this:
00235 
00236 try:
00237     retcode = call("mycmd" + " myarg", shell=True)
00238     if retcode < 0:
00239         print >>sys.stderr, "Child was terminated by signal", -retcode
00240     else:
00241         print >>sys.stderr, "Child returned", retcode
00242 except OSError, e:
00243     print >>sys.stderr, "Execution failed:", e
00244 
00245 
00246 Replacing os.spawn*
00247 -------------------
00248 P_NOWAIT example:
00249 
00250 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
00251 ==>
00252 pid = Popen(["/bin/mycmd", "myarg"]).pid
00253 
00254 
00255 P_WAIT example:
00256 
00257 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
00258 ==>
00259 retcode = call(["/bin/mycmd", "myarg"])
00260 
00261 
00262 Vector example:
00263 
00264 os.spawnvp(os.P_NOWAIT, path, args)
00265 ==>
00266 Popen([path] + args[1:])
00267 
00268 
00269 Environment example:
00270 
00271 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
00272 ==>
00273 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
00274 
00275 
00276 Replacing os.popen*
00277 -------------------
00278 pipe = os.popen(cmd, mode='r', bufsize)
00279 ==>
00280 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
00281 
00282 pipe = os.popen(cmd, mode='w', bufsize)
00283 ==>
00284 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
00285 
00286 
00287 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
00288 ==>
00289 p = Popen(cmd, shell=True, bufsize=bufsize,
00290           stdin=PIPE, stdout=PIPE, close_fds=True)
00291 (child_stdin, child_stdout) = (p.stdin, p.stdout)
00292 
00293 
00294 (child_stdin,
00295  child_stdout,
00296  child_stderr) = os.popen3(cmd, mode, bufsize)
00297 ==>
00298 p = Popen(cmd, shell=True, bufsize=bufsize,
00299           stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
00300 (child_stdin,
00301  child_stdout,
00302  child_stderr) = (p.stdin, p.stdout, p.stderr)
00303 
00304 
00305 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
00306 ==>
00307 p = Popen(cmd, shell=True, bufsize=bufsize,
00308           stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
00309 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
00310 
00311 
00312 Replacing popen2.*
00313 ------------------
00314 Note: If the cmd argument to popen2 functions is a string, the command
00315 is executed through /bin/sh.  If it is a list, the command is directly
00316 executed.
00317 
00318 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
00319 ==>
00320 p = Popen(["somestring"], shell=True, bufsize=bufsize
00321           stdin=PIPE, stdout=PIPE, close_fds=True)
00322 (child_stdout, child_stdin) = (p.stdout, p.stdin)
00323 
00324 
00325 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
00326 ==>
00327 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
00328           stdin=PIPE, stdout=PIPE, close_fds=True)
00329 (child_stdout, child_stdin) = (p.stdout, p.stdin)
00330 
00331 The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
00332 except that:
00333 
00334 * subprocess.Popen raises an exception if the execution fails
00335 * the capturestderr argument is replaced with the stderr argument.
00336 * stdin=PIPE and stdout=PIPE must be specified.
00337 * popen2 closes all filedescriptors by default, but you have to specify
00338   close_fds=True with subprocess.Popen.
00339 
00340 
00341 """
00342 
00343 import sys
00344 mswindows = (sys.platform == "win32")
00345 
00346 import os
00347 import types
00348 import traceback
00349 
00350 if mswindows:
00351     import threading
00352     import msvcrt
00353     if 0: # <-- change this to use pywin32 instead of the _subprocess driver
00354         import pywintypes
00355         from win32api import GetStdHandle, STD_INPUT_HANDLE, \
00356                              STD_OUTPUT_HANDLE, STD_ERROR_HANDLE
00357         from win32api import GetCurrentProcess, DuplicateHandle, \
00358                              GetModuleFileName, GetVersion
00359         from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE
00360         from win32pipe import CreatePipe
00361         from win32process import CreateProcess, STARTUPINFO, \
00362                                  GetExitCodeProcess, STARTF_USESTDHANDLES, \
00363                                  STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE
00364         from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
00365     else:
00366         from _subprocess import *
00367         class STARTUPINFO:
00368             dwFlags = 0
00369             hStdInput = None
00370             hStdOutput = None
00371             hStdError = None
00372         class pywintypes:
00373             error = IOError
00374 else:
00375     import select
00376     import errno
00377     import fcntl
00378     import pickle
00379 
00380 __all__ = ["Popen", "PIPE", "STDOUT", "call"]
00381 
00382 try:
00383     MAXFD = os.sysconf("SC_OPEN_MAX")
00384 except:
00385     MAXFD = 256
00386 
00387 # True/False does not exist on 2.2.0
00388 try:
00389     False
00390 except NameError:
00391     False = 0
00392     True = 1
00393 
00394 _active = []
00395 
00396 def _cleanup():
00397     for inst in _active[:]:
00398         inst.poll()
00399 
00400 PIPE = -1
00401 STDOUT = -2
00402 
00403 
00404 def call(*args, **kwargs):
00405     """!Run command with arguments.  Wait for command to complete, then
00406     return the returncode attribute.
00407 
00408     The arguments are the same as for the Popen constructor.  Example:
00409 
00410     retcode = call(["ls", "-l"])
00411     """
00412     return Popen(*args, **kwargs).wait()
00413 
00414 
00415 def list2cmdline(seq):
00416     """
00417     Translate a sequence of arguments into a command line
00418     string, using the same rules as the MS C runtime:
00419 
00420     1) Arguments are delimited by white space, which is either a
00421        space or a tab.
00422 
00423     2) A string surrounded by double quotation marks is
00424        interpreted as a single argument, regardless of white space
00425        contained within.  A quoted string can be embedded in an
00426        argument.
00427 
00428     3) A double quotation mark preceded by a backslash is
00429        interpreted as a literal double quotation mark.
00430 
00431     4) Backslashes are interpreted literally, unless they
00432        immediately precede a double quotation mark.
00433 
00434     5) If backslashes immediately precede a double quotation mark,
00435        every pair of backslashes is interpreted as a literal
00436        backslash.  If the number of backslashes is odd, the last
00437        backslash escapes the next double quotation mark as
00438        described in rule 3.
00439     """
00440 
00441     # See
00442     # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp
00443     result = []
00444     needquote = False
00445     for arg in seq:
00446         bs_buf = []
00447 
00448         # Add a space to separate this argument from the others
00449         if result:
00450             result.append(' ')
00451 
00452         needquote = (" " in arg) or ("\t" in arg)
00453         if needquote:
00454             result.append('"')
00455 
00456         for c in arg:
00457             if c == '\\':
00458                 # Don't know if we need to double yet.
00459                 bs_buf.append(c)
00460             elif c == '"':
00461                 # Double backspaces.
00462                 result.append('\\' * len(bs_buf)*2)
00463                 bs_buf = []
00464                 result.append('\\"')
00465             else:
00466                 # Normal char
00467                 if bs_buf:
00468                     result.extend(bs_buf)
00469                     bs_buf = []
00470                 result.append(c)
00471 
00472         # Add remaining backspaces, if any.
00473         if bs_buf:
00474             result.extend(bs_buf)
00475 
00476         if needquote:
00477             result.extend(bs_buf)
00478             result.append('"')
00479 
00480     return ''.join(result)
00481 
00482 
00483 class Popen(object):
00484     def __init__(self, args, bufsize=0, executable=None,
00485                  stdin=None, stdout=None, stderr=None,
00486                  preexec_fn=None, close_fds=False, shell=False,
00487                  cwd=None, env=None, universal_newlines=False,
00488                  startupinfo=None, creationflags=0):
00489         """!Create new Popen instance."""
00490         _cleanup()
00491 
00492         if not isinstance(bufsize, (int, long)):
00493             raise TypeError("bufsize must be an integer")
00494 
00495         if mswindows:
00496             if preexec_fn is not None:
00497                 raise ValueError("preexec_fn is not supported on Windows "
00498                                  "platforms")
00499             if close_fds:
00500                 raise ValueError("close_fds is not supported on Windows "
00501                                  "platforms")
00502         else:
00503             # POSIX
00504             if startupinfo is not None:
00505                 raise ValueError("startupinfo is only supported on Windows "
00506                                  "platforms")
00507             if creationflags != 0:
00508                 raise ValueError("creationflags is only supported on Windows "
00509                                  "platforms")
00510 
00511         self.stdin = None
00512         self.stdout = None
00513         self.stderr = None
00514         self.pid = None
00515         self.returncode = None
00516         self.universal_newlines = universal_newlines
00517 
00518         # Input and output objects. The general principle is like
00519         # this:
00520         #
00521         # Parent                   Child
00522         # ------                   -----
00523         # p2cwrite   ---stdin--->  p2cread
00524         # c2pread    <--stdout---  c2pwrite
00525         # errread    <--stderr---  errwrite
00526         #
00527         # On POSIX, the child objects are file descriptors.  On
00528         # Windows, these are Windows file handles.  The parent objects
00529         # are file descriptors on both platforms.  The parent objects
00530         # are None when not using PIPEs. The child objects are None
00531         # when not redirecting.
00532 
00533         (p2cread, p2cwrite,
00534          c2pread, c2pwrite,
00535          errread, errwrite) = self._get_handles(stdin, stdout, stderr)
00536 
00537         self._execute_child(args, executable, preexec_fn, close_fds,
00538                             cwd, env, universal_newlines,
00539                             startupinfo, creationflags, shell,
00540                             p2cread, p2cwrite,
00541                             c2pread, c2pwrite,
00542                             errread, errwrite)
00543 
00544         if p2cwrite:
00545             self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
00546         if c2pread:
00547             if universal_newlines:
00548                 self.stdout = os.fdopen(c2pread, 'rU', bufsize)
00549             else:
00550                 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
00551         if errread:
00552             if universal_newlines:
00553                 self.stderr = os.fdopen(errread, 'rU', bufsize)
00554             else:
00555                 self.stderr = os.fdopen(errread, 'rb', bufsize)
00556 
00557         _active.append(self)
00558 
00559 
00560     def _translate_newlines(self, data):
00561         data = data.replace("\r\n", "\n")
00562         data = data.replace("\r", "\n")
00563         return data
00564 
00565 
00566     if mswindows:
00567         #
00568         # Windows methods
00569         #
00570         def _get_handles(self, stdin, stdout, stderr):
00571             """!Construct and return tupel with IO objects:
00572             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
00573             """
00574             if stdin == None and stdout == None and stderr == None:
00575                 return (None, None, None, None, None, None)
00576 
00577             p2cread, p2cwrite = None, None
00578             c2pread, c2pwrite = None, None
00579             errread, errwrite = None, None
00580 
00581             if stdin == None:
00582                 p2cread = GetStdHandle(STD_INPUT_HANDLE)
00583             elif stdin == PIPE:
00584                 p2cread, p2cwrite = CreatePipe(None, 0)
00585                 # Detach and turn into fd
00586                 p2cwrite = p2cwrite.Detach()
00587                 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)
00588             elif type(stdin) == types.IntType:
00589                 p2cread = msvcrt.get_osfhandle(stdin)
00590             else:
00591                 # Assuming file-like object
00592                 p2cread = msvcrt.get_osfhandle(stdin.fileno())
00593             p2cread = self._make_inheritable(p2cread)
00594 
00595             if stdout == None:
00596                 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)
00597             elif stdout == PIPE:
00598                 c2pread, c2pwrite = CreatePipe(None, 0)
00599                 # Detach and turn into fd
00600                 c2pread = c2pread.Detach()
00601                 c2pread = msvcrt.open_osfhandle(c2pread, 0)
00602             elif type(stdout) == types.IntType:
00603                 c2pwrite = msvcrt.get_osfhandle(stdout)
00604             else:
00605                 # Assuming file-like object
00606                 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
00607             c2pwrite = self._make_inheritable(c2pwrite)
00608 
00609             if stderr == None:
00610                 errwrite = GetStdHandle(STD_ERROR_HANDLE)
00611             elif stderr == PIPE:
00612                 errread, errwrite = CreatePipe(None, 0)
00613                 # Detach and turn into fd
00614                 errread = errread.Detach()
00615                 errread = msvcrt.open_osfhandle(errread, 0)
00616             elif stderr == STDOUT:
00617                 errwrite = c2pwrite
00618             elif type(stderr) == types.IntType:
00619                 errwrite = msvcrt.get_osfhandle(stderr)
00620             else:
00621                 # Assuming file-like object
00622                 errwrite = msvcrt.get_osfhandle(stderr.fileno())
00623             errwrite = self._make_inheritable(errwrite)
00624 
00625             return (p2cread, p2cwrite,
00626                     c2pread, c2pwrite,
00627                     errread, errwrite)
00628 
00629 
00630         def _make_inheritable(self, handle):
00631             """!Return a duplicate of handle, which is inheritable"""
00632             return DuplicateHandle(GetCurrentProcess(), handle,
00633                                    GetCurrentProcess(), 0, 1,
00634                                    DUPLICATE_SAME_ACCESS)
00635 
00636 
00637         def _find_w9xpopen(self):
00638             """!Find and return absolut path to w9xpopen.exe"""
00639             w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),
00640                                     "w9xpopen.exe")
00641             if not os.path.exists(w9xpopen):
00642                 # Eeek - file-not-found - possibly an embedding
00643                 # situation - see if we can locate it in sys.exec_prefix
00644                 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
00645                                         "w9xpopen.exe")
00646                 if not os.path.exists(w9xpopen):
00647                     raise RuntimeError("Cannot locate w9xpopen.exe, which is "
00648                                        "needed for Popen to work with your "
00649                                        "shell or platform.")
00650             return w9xpopen
00651 
00652 
00653         def _execute_child(self, args, executable, preexec_fn, close_fds,
00654                            cwd, env, universal_newlines,
00655                            startupinfo, creationflags, shell,
00656                            p2cread, p2cwrite,
00657                            c2pread, c2pwrite,
00658                            errread, errwrite):
00659             """!Execute program (MS Windows version)"""
00660 
00661             if not isinstance(args, types.StringTypes):
00662                 args = list2cmdline(args)
00663 
00664             # Process startup details
00665             default_startupinfo = STARTUPINFO()
00666             if startupinfo == None:
00667                 startupinfo = default_startupinfo
00668             if not None in (p2cread, c2pwrite, errwrite):
00669                 startupinfo.dwFlags |= STARTF_USESTDHANDLES
00670                 startupinfo.hStdInput = p2cread
00671                 startupinfo.hStdOutput = c2pwrite
00672                 startupinfo.hStdError = errwrite
00673 
00674             if shell:
00675                 default_startupinfo.dwFlags |= STARTF_USESHOWWINDOW
00676                 default_startupinfo.wShowWindow = SW_HIDE
00677                 comspec = os.environ.get("COMSPEC", "cmd.exe")
00678                 args = comspec + " /c " + args
00679                 if (GetVersion() >= 0x80000000L or
00680                         os.path.basename(comspec).lower() == "command.com"):
00681                     # Win9x, or using command.com on NT. We need to
00682                     # use the w9xpopen intermediate program. For more
00683                     # information, see KB Q150956
00684                     # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
00685                     w9xpopen = self._find_w9xpopen()
00686                     args = '"%s" %s' % (w9xpopen, args)
00687                     # Not passing CREATE_NEW_CONSOLE has been known to
00688                     # cause random failures on win9x.  Specifically a
00689                     # dialog: "Your program accessed mem currently in
00690                     # use at xxx" and a hopeful warning about the
00691                     # stability of your system.  Cost is Ctrl+C wont
00692                     # kill children.
00693                     creationflags |= CREATE_NEW_CONSOLE
00694 
00695             # Start the process
00696             try:
00697                 hp, ht, pid, tid = CreateProcess(executable, args,
00698                                          # no special security
00699                                          None, None,
00700                                          # must inherit handles to pass std
00701                                          # handles
00702                                          1,
00703                                          creationflags,
00704                                          env,
00705                                          cwd,
00706                                          startupinfo)
00707             except pywintypes.error, e:
00708                 # Translate pywintypes.error to WindowsError, which is
00709                 # a subclass of OSError.  FIXME: We should really
00710                 # translate errno using _sys_errlist (or simliar), but
00711                 # how can this be done from Python?
00712                 raise WindowsError(*e.args)
00713 
00714             # Retain the process handle, but close the thread handle
00715             self._handle = hp
00716             self.pid = pid
00717             ht.Close()
00718 
00719             # Child is launched. Close the parent's copy of those pipe
00720             # handles that only the child should have open.  You need
00721             # to make sure that no handles to the write end of the
00722             # output pipe are maintained in this process or else the
00723             # pipe will not close when the child process exits and the
00724             # ReadFile will hang.
00725             if p2cread != None:
00726                 p2cread.Close()
00727             if c2pwrite != None:
00728                 c2pwrite.Close()
00729             if errwrite != None:
00730                 errwrite.Close()
00731 
00732 
00733         def poll(self):
00734             """!Check if child process has terminated.  Returns returncode
00735             attribute."""
00736             if self.returncode == None:
00737                 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:
00738                     self.returncode = GetExitCodeProcess(self._handle)
00739                     _active.remove(self)
00740             return self.returncode
00741 
00742 
00743         def wait(self):
00744             """!Wait for child process to terminate.  Returns returncode
00745             attribute."""
00746             if self.returncode == None:
00747                 obj = WaitForSingleObject(self._handle, INFINITE)
00748                 self.returncode = GetExitCodeProcess(self._handle)
00749                 _active.remove(self)
00750             return self.returncode
00751 
00752 
00753         def _readerthread(self, fh, buffer):
00754             buffer.append(fh.read())
00755 
00756 
00757         def communicate(self, input=None):
00758             """!Interact with process: Send data to stdin.  Read data from
00759             stdout and stderr, until end-of-file is reached.  Wait for
00760             process to terminate.  The optional input argument should be a
00761             string to be sent to the child process, or None, if no data
00762             should be sent to the child.
00763 
00764             communicate() returns a tuple (stdout, stderr)."""
00765             stdout = None # Return
00766             stderr = None # Return
00767 
00768             if self.stdout:
00769                 stdout = []
00770                 stdout_thread = threading.Thread(target=self._readerthread,
00771                                                  args=(self.stdout, stdout))
00772                 stdout_thread.setDaemon(True)
00773                 stdout_thread.start()
00774             if self.stderr:
00775                 stderr = []
00776                 stderr_thread = threading.Thread(target=self._readerthread,
00777                                                  args=(self.stderr, stderr))
00778                 stderr_thread.setDaemon(True)
00779                 stderr_thread.start()
00780 
00781             if self.stdin:
00782                 if input != None:
00783                     self.stdin.write(input)
00784                 self.stdin.close()
00785 
00786             if self.stdout:
00787                 stdout_thread.join()
00788             if self.stderr:
00789                 stderr_thread.join()
00790 
00791             # All data exchanged.  Translate lists into strings.
00792             if stdout != None:
00793                 stdout = stdout[0]
00794             if stderr != None:
00795                 stderr = stderr[0]
00796 
00797             # Translate newlines, if requested.  We cannot let the file
00798             # object do the translation: It is based on stdio, which is
00799             # impossible to combine with select (unless forcing no
00800             # buffering).
00801             if self.universal_newlines and hasattr(open, 'newlines'):
00802                 if stdout:
00803                     stdout = self._translate_newlines(stdout)
00804                 if stderr:
00805                     stderr = self._translate_newlines(stderr)
00806 
00807             self.wait()
00808             return (stdout, stderr)
00809 
00810     else:
00811         #
00812         # POSIX methods
00813         #
00814         def _get_handles(self, stdin, stdout, stderr):
00815             """!Construct and return tupel with IO objects:
00816             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
00817             """
00818             p2cread, p2cwrite = None, None
00819             c2pread, c2pwrite = None, None
00820             errread, errwrite = None, None
00821 
00822             if stdin == None:
00823                 pass
00824             elif stdin == PIPE:
00825                 p2cread, p2cwrite = os.pipe()
00826             elif type(stdin) == types.IntType:
00827                 p2cread = stdin
00828             else:
00829                 # Assuming file-like object
00830                 p2cread = stdin.fileno()
00831 
00832             if stdout == None:
00833                 pass
00834             elif stdout == PIPE:
00835                 c2pread, c2pwrite = os.pipe()
00836             elif type(stdout) == types.IntType:
00837                 c2pwrite = stdout
00838             else:
00839                 # Assuming file-like object
00840                 c2pwrite = stdout.fileno()
00841 
00842             if stderr == None:
00843                 pass
00844             elif stderr == PIPE:
00845                 errread, errwrite = os.pipe()
00846             elif stderr == STDOUT:
00847                 errwrite = c2pwrite
00848             elif type(stderr) == types.IntType:
00849                 errwrite = stderr
00850             else:
00851                 # Assuming file-like object
00852                 errwrite = stderr.fileno()
00853 
00854             return (p2cread, p2cwrite,
00855                     c2pread, c2pwrite,
00856                     errread, errwrite)
00857 
00858 
00859         def _set_cloexec_flag(self, fd):
00860             try:
00861                 cloexec_flag = fcntl.FD_CLOEXEC
00862             except AttributeError:
00863                 cloexec_flag = 1
00864 
00865             old = fcntl.fcntl(fd, fcntl.F_GETFD)
00866             fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
00867 
00868 
00869         def _close_fds(self, but):
00870             for i in range(3, MAXFD):
00871                 if i == but:
00872                     continue
00873                 try:
00874                     os.close(i)
00875                 except:
00876                     pass
00877 
00878 
00879         def _execute_child(self, args, executable, preexec_fn, close_fds,
00880                            cwd, env, universal_newlines,
00881                            startupinfo, creationflags, shell,
00882                            p2cread, p2cwrite,
00883                            c2pread, c2pwrite,
00884                            errread, errwrite):
00885             """!Execute program (POSIX version)"""
00886 
00887             if isinstance(args, types.StringTypes):
00888                 args = [args]
00889 
00890             if shell:
00891                 args = ["/bin/sh", "-c"] + args
00892 
00893             if executable == None:
00894                 executable = args[0]
00895 
00896             # For transferring possible exec failure from child to parent
00897             # The first char specifies the exception type: 0 means
00898             # OSError, 1 means some other error.
00899             errpipe_read, errpipe_write = os.pipe()
00900             self._set_cloexec_flag(errpipe_write)
00901 
00902             self.pid = os.fork()
00903             if self.pid == 0:
00904                 # Child
00905                 try:
00906                     # Close parent's pipe ends
00907                     if p2cwrite:
00908                         os.close(p2cwrite)
00909                     if c2pread:
00910                         os.close(c2pread)
00911                     if errread:
00912                         os.close(errread)
00913                     os.close(errpipe_read)
00914 
00915                     # Dup fds for child
00916                     if p2cread:
00917                         os.dup2(p2cread, 0)
00918                     if c2pwrite:
00919                         os.dup2(c2pwrite, 1)
00920                     if errwrite:
00921                         os.dup2(errwrite, 2)
00922 
00923                     # Close pipe fds.  Make sure we doesn't close the same
00924                     # fd more than once.
00925                     if p2cread:
00926                         os.close(p2cread)
00927                     if c2pwrite and c2pwrite not in (p2cread,):
00928                         os.close(c2pwrite)
00929                     if errwrite and errwrite not in (p2cread, c2pwrite):
00930                         os.close(errwrite)
00931 
00932                     # Close all other fds, if asked for
00933                     if close_fds:
00934                         self._close_fds(but=errpipe_write)
00935 
00936                     if cwd != None:
00937                         os.chdir(cwd)
00938 
00939                     if preexec_fn:
00940                         apply(preexec_fn)
00941 
00942                     if env == None:
00943                         os.execvp(executable, args)
00944                     else:
00945                         os.execvpe(executable, args, env)
00946 
00947                 except:
00948                     exc_type, exc_value, tb = sys.exc_info()
00949                     # Save the traceback and attach it to the exception object
00950                     exc_lines = traceback.format_exception(exc_type,
00951                                                            exc_value,
00952                                                            tb)
00953                     exc_value.child_traceback = ''.join(exc_lines)
00954                     os.write(errpipe_write, pickle.dumps(exc_value))
00955 
00956                 # This exitcode won't be reported to applications, so it
00957                 # really doesn't matter what we return.
00958                 os._exit(255)
00959 
00960             # Parent
00961             os.close(errpipe_write)
00962             if p2cread and p2cwrite:
00963                 os.close(p2cread)
00964             if c2pwrite and c2pread:
00965                 os.close(c2pwrite)
00966             if errwrite and errread:
00967                 os.close(errwrite)
00968 
00969             # Wait for exec to fail or succeed; possibly raising exception
00970             data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
00971             os.close(errpipe_read)
00972             if data != "":
00973                 os.waitpid(self.pid, 0)
00974                 child_exception = pickle.loads(data)
00975                 raise child_exception
00976 
00977 
00978         def _handle_exitstatus(self, sts):
00979             if os.WIFSIGNALED(sts):
00980                 self.returncode = -os.WTERMSIG(sts)
00981             elif os.WIFEXITED(sts):
00982                 self.returncode = os.WEXITSTATUS(sts)
00983             else:
00984                 # Should never happen
00985                 raise RuntimeError("Unknown child exit status!")
00986 
00987             _active.remove(self)
00988 
00989 
00990         def poll(self):
00991             """!Check if child process has terminated.  Returns returncode
00992             attribute."""
00993             if self.returncode == None:
00994                 try:
00995                     pid, sts = os.waitpid(self.pid, os.WNOHANG)
00996                     if pid == self.pid:
00997                         self._handle_exitstatus(sts)
00998                 except os.error:
00999                     pass
01000             return self.returncode
01001 
01002 
01003         def wait(self):
01004             """!Wait for child process to terminate.  Returns returncode
01005             attribute."""
01006             if self.returncode == None:
01007                 pid, sts = os.waitpid(self.pid, 0)
01008                 self._handle_exitstatus(sts)
01009             return self.returncode
01010 
01011 
01012         def communicate(self, input=None):
01013             """!Interact with process: Send data to stdin.  Read data from
01014             stdout and stderr, until end-of-file is reached.  Wait for
01015             process to terminate.  The optional input argument should be a
01016             string to be sent to the child process, or None, if no data
01017             should be sent to the child.
01018 
01019             communicate() returns a tuple (stdout, stderr)."""
01020             read_set = []
01021             write_set = []
01022             stdout = None # Return
01023             stderr = None # Return
01024 
01025             if self.stdin:
01026                 # Flush stdio buffer.  This might block, if the user has
01027                 # been writing to .stdin in an uncontrolled fashion.
01028                 self.stdin.flush()
01029                 if input:
01030                     write_set.append(self.stdin)
01031                 else:
01032                     self.stdin.close()
01033             if self.stdout:
01034                 read_set.append(self.stdout)
01035                 stdout = []
01036             if self.stderr:
01037                 read_set.append(self.stderr)
01038                 stderr = []
01039 
01040             while read_set or write_set:
01041                 rlist, wlist, xlist = select.select(read_set, write_set, [])
01042 
01043                 if self.stdin in wlist:
01044                     # When select has indicated that the file is writable,
01045                     # we can write up to PIPE_BUF bytes without risk
01046                     # blocking.  POSIX defines PIPE_BUF >= 512
01047                     bytes_written = os.write(self.stdin.fileno(), input[:512])
01048                     input = input[bytes_written:]
01049                     if not input:
01050                         self.stdin.close()
01051                         write_set.remove(self.stdin)
01052 
01053                 if self.stdout in rlist:
01054                     data = os.read(self.stdout.fileno(), 1024)
01055                     if data == "":
01056                         self.stdout.close()
01057                         read_set.remove(self.stdout)
01058                     stdout.append(data)
01059 
01060                 if self.stderr in rlist:
01061                     data = os.read(self.stderr.fileno(), 1024)
01062                     if data == "":
01063                         self.stderr.close()
01064                         read_set.remove(self.stderr)
01065                     stderr.append(data)
01066 
01067             # All data exchanged.  Translate lists into strings.
01068             if stdout != None:
01069                 stdout = ''.join(stdout)
01070             if stderr != None:
01071                 stderr = ''.join(stderr)
01072 
01073             # Translate newlines, if requested.  We cannot let the file
01074             # object do the translation: It is based on stdio, which is
01075             # impossible to combine with select (unless forcing no
01076             # buffering).
01077             if self.universal_newlines and hasattr(open, 'newlines'):
01078                 if stdout:
01079                     stdout = self._translate_newlines(stdout)
01080                 if stderr:
01081                     stderr = self._translate_newlines(stderr)
01082 
01083             self.wait()
01084             return (stdout, stderr)
01085 
01086 
01087 def _demo_posix():
01088     #
01089     # Example 1: Simple redirection: Get process list
01090     #
01091     plist = Popen(["ps"], stdout=PIPE).communicate()[0]
01092     print "Process list:"
01093     print plist
01094 
01095     #
01096     # Example 2: Change uid before executing child
01097     #
01098     if os.getuid() == 0:
01099         p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
01100         p.wait()
01101 
01102     #
01103     # Example 3: Connecting several subprocesses
01104     #
01105     print "Looking for 'hda'..."
01106     p1 = Popen(["dmesg"], stdout=PIPE)
01107     p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
01108     print repr(p2.communicate()[0])
01109 
01110     #
01111     # Example 4: Catch execution error
01112     #
01113     print
01114     print "Trying a weird file..."
01115     try:
01116         print Popen(["/this/path/does/not/exist"]).communicate()
01117     except OSError, e:
01118         if e.errno == errno.ENOENT:
01119             print "The file didn't exist.  I thought so..."
01120             print "Child traceback:"
01121             print e.child_traceback
01122         else:
01123             print "Error", e.errno
01124     else:
01125         print >>sys.stderr, "Gosh.  No error."
01126 
01127 
01128 def _demo_windows():
01129     #
01130     # Example 1: Connecting several subprocesses
01131     #
01132     print "Looking for 'PROMPT' in set output..."
01133     p1 = Popen("set", stdout=PIPE, shell=True)
01134     p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
01135     print repr(p2.communicate()[0])
01136 
01137     #
01138     # Example 2: Simple execution of program
01139     #
01140     print "Executing calc..."
01141     p = Popen("calc")
01142     p.wait()
01143 
01144 
01145 if __name__ == "__main__":
01146     if mswindows:
01147         _demo_windows()
01148     else:
01149         _demo_posix()
 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines