Background job handling

Asked by kls on 2014-08-04

Hello Gearman-Team,

I have question related to background job handling.

I have the following test case (pseudocode):

on localhost


# reset job server (RHEL)
service gearmand restart
result > success
(gearmand 1.1.8)

# empty queue
$worker->addFunction('reverse', 'reverse_fn');
while ($worker->work()) { 1; }
result > done
CTRL-C (execution break)

/usr/bin/gearadmin --show-jobs
output > .
(queue is empty)

# queue it up
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:1"
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:2"
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:3"

# empty queue again
$worker->addFunction('reverse', 'reverse_fn');
while ($worker->work()) { 1; }
result > done

/usr/bin/gearadmin --show-jobs
output > .
(queue is empty)

all fine, but...

<actual test>

$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:4"

/usr/bin/gearadmin --show-jobs
output > H:localhost.localdomain:4 0 0 1
output > .

# reset job server
service gearmand restart
result > success

/usr/bin/gearadmin --show-jobs
output > H:localhost.localdomain:1 0 0 1
output > .

Job handle is "H:localhost.localdomain:1" expected "H:localhost.localdomain:4".

While reading documentation and examples I found no way to pull state of the background job without knowing of job handle, but from my point of view job handle is internal thing and completelly unusable in deal with background jobs from outside of the job server.

So, how to pull state of the long running background job in reliable way or it is impossible at all?

Sorry for my english.

Looking forward to your answers.


Question information

English Edit question
Gearman Edit question
No assignee Edit question
Solved by:
Last query:
Last reply:
kls (kls-z) said : #1

Searching related files for "job_handle_count" (case sensitive):

   88 uint32_t job_retries; // Set maximum job retry count.
   89 uint8_t worker_wakeup; // Set maximum number of workers to wake up per job.
   90: uint32_t job_handle_count;
   91 uint32_t thread_count;
   92 uint32_t function_count;

 1276 }
 1278: server.job_handle_count= 1;
 1280 return true;

  197 int checked_length;
  198 checked_length= snprintf(server_job->job_handle, GEARMAND_JOB_HANDLE_SIZE, "%s:%u",
  199: server->job_handle_prefix, server->job_handle_count);
  201 if (checked_length >= GEARMAND_JOB_HANDLE_SIZE || checked_length < 0)
  202 {
  203 gearmand_log_error(GEARMAN_DEFAULT_LOG_PARAM, "Job handle plus handle count beyond GEARMAND_JOB_HANDLE_SIZE: %s:%u",
  204: server->job_handle_prefix, server->job_handle_count);
  205 }
  213 }
  215: server->job_handle_count++;
  216 server_job->data= data;
  217 server_job->data_size= data_size;

Job handle based on job_handle_count, which is not durable.

kls (kls-z) said : #2

### Settings for gearmand
OPTIONS="--listen --round-robin --queue-type=MySQL --mysql-host=localhost --mysql-port=3306 --mysql-user=gearman --mysql-password=<....> --mysql-db=gearman --mysql-table=gearman_queue"

chjgcn (chjgcn) said : #3

You can use job unique. It do not change even if server restart.

Best chjgcn (chjgcn) said : #4

The unique parameter can be passed like:
    $gmclient->doBackground('reverse', 'this is a test', 'this_is_unique_string');
, and then use unique to get job status:
    $status = $gmclient->jobStatusByUniqueKey('this_is_unique_string');

kls (kls-z) said : #5

Thank you very much!

Your answer is complete enough to solve my problem!

proto array gearman_client_job_status_by_unique_key(object client, string unique_key)
- get the status for a backgound job using the unique key passed in during job submission, rather than job handle.

P.S. This function is not documented.

kls (kls-z) said : #6

Thanks chjgcn, that solved my question.