Toolserver:Batch project

From MediaWiki.org
Jump to navigation Jump to search
Wikimedia Community Logo-Toolserver.svg

This page was moved from the Toolserver wiki.
Toolserver has been replaced by Toolforge. As such, the instructions here may no longer work, but may still be of historical interest.
Please help by updating examples, links, template links, etc. If a page is still relevant, move it to a normal title and leave a redirect.

This page is obsolete: There is no longer any need to use the batch project when jobs are submitted using the job scheduling system. The previous content of this page is provided for historical interest.


On Linux, we encourage people to use nice for long-running processes. On Solaris, there is a different solution: the batch project.

A "project" is something like a group. By default, all processes start in the "default" project:

% id -p
uid=1072(river) gid=100(users) projid=3(default)

You can start a process in a different project using the newtask command. To start a new process in the "batch" project:

% newtask -p batch zsh
% id -p
uid=1072(river) gid=100(users) projid=100(batch)

The difference between "default" and "batch" is that default has project.cpu-shares=100, while batch has project.cpu-shares=20. This means that if the CPU is fully loaded, tasks in the default project will be given 100 shares, while tasks in the batch project will only be given 20 shares for the same time period. Therefore short-lived and interactive processes get higher CPU priority.

Note that CPU shares are allocated to projects, not processes; this means that however many batch tasks are running, all of them will only get 20 CPU shares between them - not 20 shares each.

When used with cron, newtask is similar to nice; for example:

* * * * * newtask -p batch phoenix /home/user/someprogram python /home/user/program.py >/dev/null

To see which of your processes are in which project, you can use ps:

% ps -o uid,pid,project,args -u river
  UID   PID  PROJECT COMMAND
 1072   552    batch zsh
 1072   614    batch ps -o uid,pid,project,args -u river
 1072   496  default /usr/lib/ssh/sshd
 1072   502  default -zsh
%

If you are running batch tasks from cron, you should probably look at batch job scheduling; this will handle setting the project for you.

The newtask command can also be used to change a running process to the batch project. The syntax is

newtask -c <PID> -p batch

Category:Documentation Category:Commands