module POSIX::Spawn

The POSIX::Spawn module implements a compatible subset of Ruby 1.9's Process::spawn and related methods using the IEEE Std 1003.1 posix_spawn(2) system interfaces where available, or a pure Ruby fork/exec based implementation when not.

In Ruby 1.9, a versatile new process spawning interface was added (Process::spawn) as the foundation for enhanced versions of existing process-related methods like Kernel#system, Kernel#`, and IO#popen. These methods are backward compatible with their Ruby 1.8 counterparts but support a large number of new options. The POSIX::Spawn module implements many of these methods with support for most of Ruby 1.9's features.

The argument signatures for all of these methods follow a new convention, making it possible to take advantage of Process::spawn features:

spawn([env], command, [argv1, ...], [options])
system([env], command, [argv1, ...], [options])
popen([[env], command, [argv1, ...]], mode="r", [options])

The env, command, and options arguments are described below.

Environment

If a hash is given in the first argument (env), the child process's environment becomes a merge of the parent's and any modifications specified in the hash. When a value in env is nil, the variable is unset in the child:

# set FOO as BAR and unset BAZ.
spawn({"FOO" => "BAR", "BAZ" => nil}, 'echo', 'hello world')

Command

The command and optional argvN string arguments specify the command to execute and any program arguments. When only command is given and includes a space character, the command text is executed by the system shell interpreter, as if by:

/bin/sh -c 'command'

When command does not include a space character, or one or more argvN arguments are given, the command is executed as if by execve(2) with each argument forming the new program's argv.

NOTE: Use of the shell variation is generally discouraged unless you indeed want to execute a shell program. Specifying an explicitly argv is typically more secure and less error prone in most cases.

Options

When a hash is given in the last argument (options), it specifies a current directory and zero or more fd redirects for the child process.

The :chdir option specifies the current directory. Note that :chdir is not thread-safe on systems that provide posix_spawn(2), because it forces a temporary change of the working directory of the calling process.

spawn(command, :chdir => "/var/tmp")

The :in, :out, :err, a Fixnum, an IO object or an Array option specify fd redirection. For example, stderr can be merged into stdout as follows:

spawn(command, :err => :out)
spawn(command, 2 => 1)
spawn(command, STDERR => :out)
spawn(command, STDERR => STDOUT)

The key is a fd in the newly spawned child process (stderr in this case). The value is a fd in the parent process (stdout in this case).

You can also specify a filename for redirection instead of an fd:

spawn(command, :in => "/dev/null")   # read mode
spawn(command, :out => "/dev/null")  # write mode
spawn(command, :err => "log")        # write mode
spawn(command, 3 => "/dev/null")     # read mode

When redirecting to stdout or stderr, the files are opened in write mode; otherwise, read mode is used.

It's also possible to control the open flags and file permissions directly by passing an array value:

spawn(command, :in=>["file"])       # read mode assumed
spawn(command, :in=>["file", "r"])  # explicit read mode
spawn(command, :out=>["log", "w"])  # explicit write mode, 0644 assumed
spawn(command, :out=>["log", "w", 0600])
spawn(command, :out=>["log", File::APPEND | File::CREAT, 0600])

The array is a [filename, open_mode, perms] tuple. open_mode can be a string or an integer. When open_mode is omitted or nil, File::RDONLY is assumed. The perms element should be an integer. When perms is omitted or nil, 0644 is assumed.

The :close It's possible to direct an fd be closed in the child process. This is important for implementing `popen`-style logic and other forms of IPC between processes using `IO.pipe`:

rd, wr = IO.pipe
pid = spawn('echo', 'hello world', rd => :close, :stdout => wr)
wr.close
output = rd.read
Process.wait(pid)

Spawn Implementation

The #spawn method uses the best available implementation given the current platform and Ruby version. In order of preference, they are:

1. The posix_spawn based C extension method (pspawn).
2. Process::spawn when available (Ruby 1.9 only).
3. A simple pure-Ruby fork/exec based spawn implementation compatible
   with Ruby >= 1.8.7.

Constants

OFLAGS

Mapping of string open modes to integer oflag versions.

VERSION

Public Instance Methods

_pspawn(p1, p2, p3) click to toggle source

POSIX::Spawn#_pspawn(env, argv, options)

env - Hash of the new environment. argv - The [[cmdname, argv0], argv1, …] exec array. options - The options hash with fd redirect and close operations.

Returns the pid of the newly spawned process.

static VALUE
rb_posixspawn_pspawn(VALUE self, VALUE env, VALUE argv, VALUE options)
{
        int i, ret = 0;
        char **envp = NULL;
        VALUE dirname;
        VALUE cmdname;
        VALUE unsetenv_others_p = Qfalse;
        char *file;
        char *cwd = NULL;
        pid_t pid;
        posix_spawn_file_actions_t fops;
        posix_spawnattr_t attr;
        sigset_t mask;
        short flags = 0;

        /* argv is a [[cmdname, argv0], argv1, argvN, ...] array. */
        if (TYPE(argv) != T_ARRAY ||
            TYPE(RARRAY_PTR(argv)[0]) != T_ARRAY ||
            RARRAY_LEN(RARRAY_PTR(argv)[0]) != 2)
                rb_raise(rb_eArgError, "Invalid command name");

        long argc = RARRAY_LEN(argv);
        char *cargv[argc + 1];

        cmdname = RARRAY_PTR(argv)[0];
        file = StringValuePtr(RARRAY_PTR(cmdname)[0]);

        cargv[0] = StringValuePtr(RARRAY_PTR(cmdname)[1]);
        for (i = 1; i < argc; i++)
                cargv[i] = StringValuePtr(RARRAY_PTR(argv)[i]);
        cargv[argc] = NULL;

        if (TYPE(options) == T_HASH) {
                unsetenv_others_p = rb_hash_delete(options, ID2SYM(rb_intern("unsetenv_others")));
        }

        if (RTEST(env)) {
                /*
                 * Make sure env is a hash, and all keys and values are strings.
                 * We do this before allocating space for the new environment to
                 * prevent a leak when raising an exception after the calloc() below.
                 */
                Check_Type(env, T_HASH);
                rb_hash_foreach(env, each_env_check_i, 0);

                if (RHASH_SIZE(env) > 0) {
                        int size = 0;

                        char **curr = environ;
                        if (curr) {
                                while (*curr != NULL) ++curr, ++size;
                        }

                        if (unsetenv_others_p == Qtrue) {
                                /*
                                 * ignore the parent's environment by pretending it had
                                 * no entries. the loop below will do nothing.
                                 */
                                size = 0;
                        }

                        char **new_env = calloc(size+RHASH_SIZE(env)+1, sizeof(char*));
                        for (i = 0; i < size; i++) {
                                new_env[i] = strdup(environ[i]);
                        }
                        envp = new_env;

                        rb_hash_foreach(env, each_env_i, (VALUE)envp);
                }
        }

        posixspawn_file_actions_init(&fops, options);
        posix_spawnattr_init(&attr);

        /* child does not block any signals */
        flags |= POSIX_SPAWN_SETSIGMASK;
        sigemptyset(&mask);
        posix_spawnattr_setsigmask(&attr, &mask);

        /* Child reverts SIGPIPE handler to the default. */
        flags |= POSIX_SPAWN_SETSIGDEF;
        sigaddset(&mask, SIGPIPE);
        posix_spawnattr_setsigdefault(&attr, &mask);

#if defined(POSIX_SPAWN_USEVFORK) || defined(__GLIBC__)
        /* Force USEVFORK on GNU libc. If this is undefined, it's probably
         * because you forgot to define _GNU_SOURCE at the top of this file.
         */
        flags |= POSIX_SPAWN_USEVFORK;
#endif

        /* setup pgroup options */
        posixspawn_set_pgroup(options, &attr, &flags);

        posix_spawnattr_setflags(&attr, flags);

        if (RTEST(dirname = rb_hash_delete(options, ID2SYM(rb_intern("chdir"))))) {
                char *new_cwd = StringValuePtr(dirname);
                cwd = getcwd(NULL, 0);
                if (chdir(new_cwd) == -1) {
                        free(cwd);
                        cwd = NULL;
                        ret = errno;
                }
        }

        if (ret == 0) {
                if (RHASH_SIZE(options) == 0) {
                        ret = posix_spawnp(&pid, file, &fops, &attr, cargv, envp ? envp : environ);
                        if (cwd) {
                                /* Ignore chdir failures here.  There's already a child running, so
                                 * raising an exception here would do more harm than good. */
                                if (chdir(cwd) == -1) {}
                        }
                } else {
                        ret = -1;
                }
        }

        if (cwd)
                free(cwd);

        posix_spawn_file_actions_destroy(&fops);
        posix_spawnattr_destroy(&attr);
        if (envp) {
                char **ep = envp;
                while (*ep != NULL) free(*ep), ++ep;
                free(envp);
        }

        if (RHASH_SIZE(options) > 0) {
                rb_raise(rb_eArgError, "Invalid option: %s", RSTRING_PTR(rb_inspect(rb_funcall(options, rb_intern("first"), 0))));
                return -1;
        }

        if (ret != 0) {
                char error_context[PATH_MAX+32];
                snprintf(error_context, sizeof(error_context), "when spawning '%s'", file);
                errno = ret;
                rb_sys_fail(error_context);
        }

        return INT2FIX(pid);
}
`(cmd) click to toggle source

Executes a command in a subshell using the system's shell interpreter and returns anything written to the new process's stdout. This method is compatible with Kernel#`.

Returns the String output of the command.

# File lib/posix/spawn.rb, line 279
def `(cmd)
  r, w = IO.pipe
  command_and_args = system_command_prefixes + [cmd, {:out => w, r => :close}]
  pid = spawn(*command_and_args)

  if pid > 0
    w.close
    out = r.read
    ::Process.waitpid(pid)
    out
  else
    ''
  end
ensure
  [r, w].each{ |io| io.close rescue nil }
end
fspawn(*args) click to toggle source

Spawn a child process with a variety of options using a pure Ruby fork + exec. Supports the standard spawn interface as described in the POSIX::Spawn module documentation.

# File lib/posix/spawn.rb, line 195
def fspawn(*args)
  env, argv, options = extract_process_spawn_arguments(*args)
  valid_options = [:chdir, :unsetenv_others, :pgroup]

  if badopt = options.find{ |key,val| !fd?(key) && !valid_options.include?(key) }
    raise ArgumentError, "Invalid option: #{badopt[0].inspect}"
  elsif !argv.is_a?(Array) || !argv[0].is_a?(Array) || argv[0].size != 2
    raise ArgumentError, "Invalid command name"
  end

  fork do
    begin
      # handle FD => {FD, :close, [file,mode,perms]} options
      options.each do |key, val|
        if fd?(key)
          key = fd_to_io(key)

          if fd?(val)
            val = fd_to_io(val)
            key.reopen(val)
            if key.respond_to?(:close_on_exec=)
              key.close_on_exec = false
              val.close_on_exec = false
            end
          elsif val == :close
            if key.respond_to?(:close_on_exec=)
              key.close_on_exec = true
            else
              key.close
            end
          elsif val.is_a?(Array)
            file, mode_string, perms = *val
            key.reopen(File.open(file, mode_string, perms))
          end
        end
      end

      # setup child environment
      ENV.replace({}) if options[:unsetenv_others] == true
      env.each { |k, v| ENV[k] = v }

      # { :chdir => '/' } in options means change into that dir
      ::Dir.chdir(options[:chdir]) if options[:chdir]

      # { :pgroup => pgid } options
      pgroup = options[:pgroup]
      pgroup = 0 if pgroup == true
      Process::setpgid(0, pgroup) if pgroup

      # do the deed
      if RUBY_VERSION =~ /\A1\.8/
        ::Kernel::exec(*argv)
      else
        argv_and_options = argv + [{:close_others=>false}]
        ::Kernel::exec(*argv_and_options)
      end
    ensure
      exit!(127)
    end
  end
end
popen4(*argv) click to toggle source

Spawn a child process with all standard IO streams piped in and out of the spawning process. Supports the standard spawn interface as described in the POSIX::Spawn module documentation.

Returns a [pid, stdin, stdout, stderr] tuple, where pid is the new process's pid, stdin is a writeable IO object, and stdout / stderr are readable IO objects. The caller should take care to close all IO objects when finished and the child process's status must be collected by a call to Process::waitpid or equivalent.

# File lib/posix/spawn.rb, line 305
def popen4(*argv)
  # create some pipes (see pipe(2) manual -- the ruby docs suck)
  ird, iwr = IO.pipe
  ord, owr = IO.pipe
  erd, ewr = IO.pipe

  # spawn the child process with either end of pipes hooked together
  opts =
    ((argv.pop if argv[-1].is_a?(Hash)) || {}).merge(
      # redirect fds        # close other sides
      :in  => ird,          iwr  => :close,
      :out => owr,          ord  => :close,
      :err => ewr,          erd  => :close
    )
  pid = spawn(*(argv + [opts]))

  [pid, iwr, ord, erd]
ensure
  # we're in the parent, close child-side fds
  [ird, owr, ewr].each { |fd| fd.close if fd }
end
pspawn(*args) click to toggle source

Spawn a child process with a variety of options using the posix_spawn(2) systems interfaces. Supports the standard spawn interface as described in the POSIX::Spawn module documentation.

Raises NotImplementedError when the posix_spawn_ext module could not be loaded due to lack of platform support.

# File lib/posix/spawn.rb, line 176
def pspawn(*args)
  env, argv, options = extract_process_spawn_arguments(*args)
  raise NotImplementedError unless respond_to?(:_pspawn)

  if defined? JRUBY_VERSION
    # On the JVM, changes made to the environment are not propagated down
    # to C via get/setenv, so we have to fake it here.
    unless options[:unsetenv_others] == true
      env = ENV.merge(env)
      options[:unsetenv_others] = true
    end
  end

  _pspawn(env, argv, options)
end
spawn(*args) click to toggle source

Spawn a child process with a variety of options using the best available implementation for the current platform and Ruby version.

spawn(, command, [argv1, …], [options])

env - Optional hash specifying the new process's environment. command - A string command name, or shell program, used to determine the

program to execute.

argvN - Zero or more string program arguments (argv). options - Optional hash of operations to perform before executing the

new child process.

Returns the integer pid of the newly spawned process.

Raises any number of Errno

exceptions on failure.

# File lib/posix/spawn.rb, line 160
def spawn(*args)
  if respond_to?(:_pspawn)
    pspawn(*args)
  elsif ::Process.respond_to?(:spawn)
    ::Process::spawn(*args)
  else
    fspawn(*args)
  end
end
system(*args) click to toggle source

Executes a command and waits for it to complete. The command's exit status is available as $?. Supports the standard spawn interface as described in the POSIX::Spawn module documentation.

This method is compatible with Kernel#system.

Returns true if the command returns a zero exit status, or false for non-zero exit.

# File lib/posix/spawn.rb, line 265
def system(*args)
  pid = spawn(*args)
  return false if pid <= 0
  ::Process.waitpid(pid)
  $?.exitstatus == 0
rescue Errno::ENOENT
  false
end

Private Instance Methods

adjust_process_spawn_argv(args) click to toggle source

Converts the various supported command argument variations into a standard argv suitable for use with exec. This includes detecting commands to be run through the shell (single argument strings with spaces).

The args array may follow any of these variations:

'true' => [['true', 'true']] 'echo', 'hello', 'world' => [['echo', 'echo'], 'hello', 'world'] 'echo hello world' => [['/bin/sh', '/bin/sh'], '-c', 'echo hello world']

'echo', 'fuuu'], 'hello' => [['echo', 'fuuu'], 'hello'

Returns a [[cmdname, argv0], argv1, …] array.

# File lib/posix/spawn.rb, line 531
def adjust_process_spawn_argv(args)
  if args.size == 1 && args[0] =~ /[ |>]/
    # single string with these characters means run it through the shell
    command_and_args = system_command_prefixes + [args[0]]
    [*command_and_args]
  elsif !args[0].respond_to?(:to_ary)
    # [argv0, argv1, ...]
    [[args[0], args[0]], *args[1..-1]]
  else
    # [[cmdname, argv0], argv1, ...]
    args
  end
end
default_file_reopen_info(fd, file) click to toggle source

The default [file, flags, mode] tuple for a given fd and filename. The default flags vary based on the what fd is being redirected. stdout and stderr default to write, while stdin and all other fds default to read.

fd - The file descriptor that is being redirected. This may be an IO

object, integer fd number, or :in, :out, :err for one of the standard
streams.

file - The string path to the file that fd should be redirected to.

Returns a [file, flags, mode] tuple.

# File lib/posix/spawn.rb, line 448
def default_file_reopen_info(fd, file)
  case fd
  when :in, STDIN, $stdin, 0
    [file, "r", 0644]
  when :out, STDOUT, $stdout, 1
    [file, "w", 0644]
  when :err, STDERR, $stderr, 2
    [file, "w", 0644]
  else
    [file, "r", 0644]
  end
end
extract_process_spawn_arguments(*args) click to toggle source

Turns the various varargs incantations supported by Process::spawn into a simple [env, argv, options] tuple. This just makes life easier for the extension functions.

The following method signature is supported:

Process::spawn([env], command, ..., [options])

The env and options hashes are optional. The command may be a variable number of strings or an Array full of strings that make up the new process's argv.

Returns an [env, argv, options] tuple. All elements are guaranteed to be non-nil. When no env or options are given, empty hashes are returned.

# File lib/posix/spawn.rb, line 355
def extract_process_spawn_arguments(*args)
  # pop the options hash off the end if it's there
  options =
    if args[-1].respond_to?(:to_hash)
      args.pop.to_hash
    else
      {}
    end
  flatten_process_spawn_options!(options)
  normalize_process_spawn_redirect_file_options!(options)

  # shift the environ hash off the front if it's there and account for
  # possible :env key in options hash.
  env =
    if args[0].respond_to?(:to_hash)
      args.shift.to_hash
    else
      {}
    end
  env.merge!(options.delete(:env)) if options.key?(:env)

  # remaining arguments are the argv supporting a number of variations.
  argv = adjust_process_spawn_argv(args)

  [env, argv, options]
end
fd?(object) click to toggle source

Determine whether object is fd-like.

Returns true if object is an instance of IO, Fixnum >= 0, or one of the the symbolic names :in, :out, or :err.

# File lib/posix/spawn.rb, line 465
def fd?(object)
  case object
  when Fixnum
    object >= 0
  when :in, :out, :err, STDIN, STDOUT, STDERR, $stdin, $stdout, $stderr, IO
    true
  else
    object.respond_to?(:to_io) && !object.to_io.nil?
  end
end
fd_to_io(object) click to toggle source

Convert a fd identifier to an IO object.

Returns nil or an instance of IO.

# File lib/posix/spawn.rb, line 479
def fd_to_io(object)
  case object
  when STDIN, STDOUT, STDERR, $stdin, $stdout, $stderr
    object
  when :in, 0
    STDIN
  when :out, 1
    STDOUT
  when :err, 2
    STDERR
  when Fixnum
    object >= 0 ? IO.for_fd(object) : nil
  when IO
    object
  else
    object.respond_to?(:to_io) ? object.to_io : nil
  end
end
flatten_process_spawn_options!(options) click to toggle source

Convert { [fd1, fd2, …] => (:close|fd) } options to individual keys, like: { fd1 => :close, fd2 => :close }. This just makes life easier for the spawn implementations.

options - The options hash. This is modified in place.

Returns the modified options hash.

# File lib/posix/spawn.rb, line 389
def flatten_process_spawn_options!(options)
  options.to_a.each do |key, value|
    if key.respond_to?(:to_ary)
      key.to_ary.each { |fd| options[fd] = value }
      options.delete(key)
    end
  end
end
normalize_process_spawn_redirect_file_options!(options) click to toggle source

Convert variations of redirecting to a file to a standard tuple.

:in => '/some/file' => ['/some/file', 'r', 0644] :out => '/some/file' => ['/some/file', 'w', 0644] :err => '/some/file' => ['/some/file', 'w', 0644] STDIN => '/some/file' => ['/some/file', 'r', 0644]

Returns the modified options hash.

# File lib/posix/spawn.rb, line 416
def normalize_process_spawn_redirect_file_options!(options)
  options.to_a.each do |key, value|
    next if !fd?(key)

    # convert string and short array values to
    if value.respond_to?(:to_str)
      value = default_file_reopen_info(key, value)
    elsif value.respond_to?(:to_ary) && value.size < 3
      defaults = default_file_reopen_info(key, value[0])
      value += defaults[value.size..-1]
    else
      value = nil
    end

    # replace string open mode flag maybe and replace original value
    if value
      value[1] = OFLAGS[value[1]] if value[1].respond_to?(:to_str)
      options[key] = value
    end
  end
end
system_command_prefixes() click to toggle source

Derives the shell command to use when running the spawn.

On a Windows machine, this will yield:

[['cmd.exe', 'cmd.exe'], '/c']

Note: 'cmd.exe' is used if the COMSPEC environment variable

is not specified. If you would like to use something other
than 'cmd.exe', specify its path in ENV['COMSPEC']

On all other systems, this will yield:

[['/bin/sh', '/bin/sh'], '-c']

Returns a platform-specific [[<shell>, <shell>], <command-switch>] array.

# File lib/posix/spawn.rb, line 510
def system_command_prefixes
  if RUBY_PLATFORM =~ /(mswin|mingw|cygwin|bccwin)/
    sh = ENV['COMSPEC'] || 'cmd.exe'
    [[sh, sh], '/c']
  else
    [['/bin/sh', '/bin/sh'], '-c']
  end
end