Projects / GNU Parallel

GNU Parallel

GNU parallel is a shell tool for executing jobs in parallel locally or using remote computers. A job is typically a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. If you use xargs today you will find GNU parallel very easy to use, as GNU parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. If you use ppss or pexec you will find GNU parallel will often make the command easier to read. GNU parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU parallel as input for other programs.

Tags	Text Processing parallel Parallel processing Multicore Clustering/Distributed Networks Command Line Tools Filters System Administration
Licenses	GPLv3
Operating Systems	POSIX Linux Solaris Unix Darwin FreeBSD
Implementation	Perl

Website

Download

Release tags

All releases

Recent releases

20120422 ('Mærsk')

22 Apr 2012 12:17

Release Notes: A race condition bug caused restructuring of a central piece of the code, so this release should be considered beta quality. sql --dbsize will now list sizes of all databases if given no database.

20120322 ('#518696')

22 Mar 2012 23:14

Release Notes: This is a bugfix release with no new features, probably a good release for stable longterm use.

20120222 ('Whitney')

23 Feb 2012 04:57

Release Notes: --workdir . will use the current working directory. If the current working directory is beneath your home directory, the value . is treated as the relative path to your home directory. This means that if your home directory is different on remote computers (e.g. if your login is different), the relative path will still be relative to your home directory. A texinfo file is now generated from the POD file using pod2texi.

20120122 ('Dead SOPA')

23 Jan 2012 01:24

Release Notes: --header now uses the first input line as column names, and you can then use {colname} as a replacement string in the command. (This also works with multiple ":::"s.) --header <regexp> matches a header as a regular expression and repeats the header for each block with --pipe. --resume resumes from the last unfinished job. (This is useful if you stop GNU Parallel and restart it later with the same arguments.) niceload now propagates the exit status correctly. The options -g, -B, -T, -U, -W, and -Y have been retired, as warned six months ago.