eCAP

Trying to accumulate vb chunks -> missing first chunk

Asked by Magus

Hello,

I'm trying to accumulate all of the vb chunks in order to have the whole virgin body. I used the passthru sample adapter as a start. I accumulate the chunks in noteVbContentAvailable() (took that part of the code from the modifying adapter, removing the call to adaptContent). Then, in noteVbContentDone, I stream the whole buffer into a file (yeah, that's dirty, but it's just for testing purpose). The result is that the begining of the body is missing (might be the first chunk missing ?). I guess I'm probably doing something wrong here. Is noteVbContentAvailable the right place to accumulate chunks ?

For reference, here are the code changes I made to the passthru adapter :

Added Xaction private member :

std::string buffer;

Added in noteVbContentAvailable, right after Must(receivingVb == opOn) :

const libecap::Area vb = hostx->vbContent(0, libecap::nsize);
std::string chunk = vb.toString();
buffer += chunk;

Added in noteVbContentDone, just before hostx->noteAbContentDone(atEnd) :

using namespace std;
ofstream outfile;
outfile.open("/tmp/test.dat", ios::out|ios::binary|ios::app);
outfile << buffer;
outfile.close();

Thanks in advance for any kind of help you could offer.

P.S : I apologize if my english is far from perfection, it isn't my primary language

EDIT : I applied the various patch suggested in the bug reports, I'm missing the first chunk nevertheless

Question information

Language:
English Edit question
Status:
Solved
For:
eCAP Edit question
Assignee:
No assignee Edit question
Solved by:
Alex Rousskov
Solved:
Last query:
Last reply:
Revision history for this message
Magus (davdeclerck) said : 		#1

Hello again,

I've investigated a bit more about this, and as my understanding of the API improves, I think I've found the cause. That leads me to some questions however...

In the Passthru adapter, right after abMake, abContent and abContentShift are called, this occurs before the first call to noteVbContentAvailable. As abContentShift calls vbContentShift, this explains why my first chunk is missing : that part of the body is shifted before I get a chance to handle it in noteVbContentAvailable. The transaction (if I'm using the right term for the right thing) is different in the Modifying adapter though : noteVbContentAvailable is called right after abMake.

Basically, the begining of the transaction looks different in the Passthru and modifying adapters. I've taken clean sources, applied all the patches specified in the bug reports, and added some kind output to follow each method call and write that to a file.

This is the transaction I got for the modifying adapter :

Xaction::start
Xaction::abMake

     Xaction::noteVbContentAvailable
     Xaction::abContent / offset = 0, size = 4294967295
     Xaction::abContentShift / size = 1181
     Xaction::abContent / offset = 0, size = 4294967295
     => Repeated as long as there is Vb data

Xaction::noteVbContentAvailable
Xaction::abContent / offset = 0, size = 4294967295
Xaction::abContentShift / size = 401
Xaction::noteVbContentDone
Xaction::abContent / offset = 0, size = 4294967295
Xaction::stop

Now here is the transaction I got for the passthru adapter :

Xaction::start
Xaction::abMake
Xaction::abContent / offset = 0, size = 4294967295
Xaction::abContentShift / size = 1181
Xaction::noteVbContentAvailable
Xaction::abContent / offset = 0, size = 4294967295
Xaction::abContent / offset = 0, size = 4294967295

     Xaction::noteVbContentAvailable
     Xaction::abContent / offset = 0, size = 4294967295
     Xaction::abContentShift / size = 1452
     Xaction::abContent / offset = 0, size = 4294967295
     => Repeated as long as there is Vb data

Xaction::noteVbContentAvailable
Xaction::abContent / offset = 0, size = 4294967295
Xaction::abContentShift / size = 1452
Xaction::noteVbContentDone
Xaction::abContent / offset = 0, size = 4294967295
Xaction::stop

As you can see, the begining is quite different.

Now, I've noticed there's a major difference in the abMake code of the passthru and modifying adapters.

In the passthru adapter, we just call noteAbContentAvailable, inconditionnally, while in the modifying adapter, we first check that there is actually some Ab data - if (!buffer.empty()) - before calling noteAbContentAvailable.

I don't know if I'm right or not, but, may we consider that this condition will anyways always be false in the case of the modifying adapter ? Hence, noteAbContentAvailable would never be called from abMake... (should it be called from here anyways ?)

Now, in order to obtain the same kind of transaction in the passthru adapter, I simply commented out the call to hostx->noteAbContentAvailable(); in abMake. I tested on several heavy sites without any troubles, but is this the right thing to do ?

Thanks in advance for your help

Revision history for this message
Best Alex Rousskov (rousskov) said : 		#2

The value of (!buffer.empty()) expression in the modifying adapter depends on whether the virgin body was provided before the call to abMake(). There is no guarantee that abMake() will be called before any virgin body is supplied.

In a correctly functioning adapter, it should be OK to call noteAbContentAvailable when the size of the available adapted body is zero. It would most likely be a waste of CPU cycles, but it should not lead to any bugs.

The passthru adapter does not maintain its own body buffer and so it cannot "miss" the first chunk of the virgin body. If you modify the passthru adapter to handle its own body buffer, you need to make sure that the code works regardless of the virgin and adapted messages order. For example, abMake() may be called before or after noteVbContentAvailable().

Also, you cannot assume that a note*() notification reflects the current body buffer or transaction status -- notifications are asynchronous and may become stale. Your only guarantee is that the notifications from the same source are received in the order they were sent.

HTH,

Alex.

Revision history for this message
Magus (davdeclerck) said : 		#3

Thanks Alex Rousskov, that solved my question.

Revision history for this message
Sandeep Kuttal (skuttal) said : 		#4

Hi Magus,

I tried the way you have discussed about the code to save the contents to the disk. But I am not able to get the contents of the message body inside them. Can you suggest? Thanks