Gearman

Background job handling

Asked by kls

Hello Gearman-Team,

I have question related to background job handling.

I have the following test case (pseudocode):

on localhost

<prepare>

# reset job server (RHEL)
service gearmand restart
result > success
(gearmand 1.1.8)

# empty queue
$worker->addFunction('reverse', 'reverse_fn');
while ($worker->work()) { 1; }
result > done
CTRL-C (execution break)

/usr/bin/gearadmin --show-jobs
output > .
(queue is empty)

# queue it up
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:1"
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:2"
$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:3"

# empty queue again
$worker->addFunction('reverse', 'reverse_fn');
while ($worker->work()) { 1; }
result > done
CTRL-C

/usr/bin/gearadmin --show-jobs
output > .
(queue is empty)

all fine, but...

<actual test>

$jobHandle = $gmclient->doBackground('reverse', 'this is a test');
result > $jobHandle == "H:localhost.localdomain:4"

/usr/bin/gearadmin --show-jobs
output > H:localhost.localdomain:4 0 0 1
output > .

# reset job server
service gearmand restart
result > success

/usr/bin/gearadmin --show-jobs
output > H:localhost.localdomain:1 0 0 1
output > .

Job handle is "H:localhost.localdomain:1" expected "H:localhost.localdomain:4".

While reading documentation and examples I found no way to pull state of the background job without knowing of job handle, but from my point of view job handle is internal thing and completelly unusable in deal with background jobs from outside of the job server.

So, how to pull state of the long running background job in reliable way or it is impossible at all?

Sorry for my english.

Looking forward to your answers.

Thanks.

Question information

Language:
English Edit question
Status:
Solved
For:
Gearman Edit question
Assignee:
No assignee Edit question
Solved by:
chjgcn
Solved:
Last query:
Last reply:
Revision history for this message
kls (kls-z) said : 		#1

Searching related files for "job_handle_count" (case sensitive):

./gearmand-1.1.12/libgearman-server/struct/server.h:
   88 uint32_t job_retries; // Set maximum job retry count.
   89 uint8_t worker_wakeup; // Set maximum number of workers to wake up per job.
   90: uint32_t job_handle_count;
   91 uint32_t thread_count;
   92 uint32_t function_count;

./gearmand-1.1.12/libgearman-server/gearmand.cc:
 1276 }
 1277
 1278: server.job_handle_count= 1;
 1279
 1280 return true;

./gearmand-1.1.12/libgearman-server/job.cc:
  197 int checked_length;
  198 checked_length= snprintf(server_job->job_handle, GEARMAND_JOB_HANDLE_SIZE, "%s:%u",
  199: server->job_handle_prefix, server->job_handle_count);
  200
  201 if (checked_length >= GEARMAND_JOB_HANDLE_SIZE || checked_length < 0)
  202 {
  203 gearmand_log_error(GEARMAN_DEFAULT_LOG_PARAM, "Job handle plus handle count beyond GEARMAND_JOB_HANDLE_SIZE: %s:%u",
  204: server->job_handle_prefix, server->job_handle_count);
  205 }
  206
  ...
  213 }
  214
  215: server->job_handle_count++;
  216 server_job->data= data;
  217 server_job->data_size= data_size;

Job handle based on job_handle_count, which is not durable.

Revision history for this message
kls (kls-z) said : 		#2

### Settings for gearmand
OPTIONS="--listen 127.0.0.1 --round-robin --queue-type=MySQL --mysql-host=localhost --mysql-port=3306 --mysql-user=gearman --mysql-password=<....> --mysql-db=gearman --mysql-table=gearman_queue"

Revision history for this message
chjgcn (chjgcn) said : 		#3

You can use job unique. It do not change even if server restart.

Revision history for this message
Best chjgcn (chjgcn) said : 		#4

The unique parameter can be passed like:
    $gmclient->doBackground('reverse', 'this is a test', 'this_is_unique_string');
, and then use unique to get job status:
    $status = $gmclient->jobStatusByUniqueKey('this_is_unique_string');

Revision history for this message
kls (kls-z) said : 		#5

Thank you very much!

Your answer is complete enough to solve my problem!

proto array gearman_client_job_status_by_unique_key(object client, string unique_key)
- get the status for a backgound job using the unique key passed in during job submission, rather than job handle.

P.S. This function is not documented.

Revision history for this message
kls (kls-z) said : 		#6

Thanks chjgcn, that solved my question.