How to pass meta information from Reqmod to Respmod

I'll describe what "global" problem I'm trying to solve (so that I'm not trapped in"XY-problem")

I'm using eCAP to dump deciphered traffic from Squid to files on local hard drive.
Now every "packet" is dumped to single file, containing headers (grabbed with image() function call) and body (while reading virgin chunks, they're appended to file).

What I need now is to dump them by pairs (Response + Request).
So my idea is to try to utilize meta tags to "mark" requests and this way find corresponding responses to dump them to files.

Another idea is to add meta tag, let's call it "X-Ecap-ID" with custom value, f.ex. 1585883 (numbers will differ on each new request).
So the request is dumped to file with name "1585883.txt", then this magic number is passed as meta tag to response and then response adapter sees this tag and appends its headers+body to "1585883.txt" as well.

A little update: I patched names.cc:
+ const libecap::Name libecap::metaEcapID("X-Ecap-ID", libecap::Name::NextId());
and names.h:
+ extern const Name metaEcapID;

Re-compiled libecap.
Then in my RESPMOD adapter in Xaction::start() method added the following line:
libecap::Area val = hostx->option(libecap::metaEcapID);

However I have no idea how to set this new meta option in REQMOD adapter:
tried const libecap::Name metaname = libecap::metaSubscriberId("X-Ecap-ID", libecap::Name::NextId()); - doesn't work

After I created my custom metainfo, I tried
const libecap::Name metaname2 = libecap::metaEcapID("X-Ecap-ID", libecap::Name::NextId());
However it doesn't compile with error "no match for call to (const libecap::Name) (const char [10], libecap::Name::Id)" which I have no idea how to solve right now.

In my squid.conf I have the following setting:
adaptation_masterx_shared_names X-Ecap-ID

> how can one add such information to Xaction so that this meta info will be added to transaction?

Implement the libecap::Options API methods in your Xaction class. The host application uses those methods to retrieve your Xaction meta info.

> I'm using eCAP to dump traffic to files. Every message is dumped to single file, containing message header and body. I need to dump both request message and the corresponding response message to the same file.

You want to correlate the virgin request with the virgin response(s) to that request. If your host application does not provide a suitable way to identify the master transaction, then the adapter itself can create an equivalent of a master transaction ID:

1. For each REQMOD transaction, create a unique ID that changes with each REQMOD transaction (e.g., 1, 2, 3, etc.).
2. Send that ID (as a meta header field called, for example, "X-Xaction-ID") to the host application.
3. Configure your host application to share meta fields named "X-Xaction-ID" across adaptation transactions that belong to the same master transaction.
4. Receive the X-Xaction-ID meta header field in RESPMOD.

As described in the (now linked to this Question) FAQ, to "send" a meta header field to the host application, your libecap::adapter::Xaction child must implement the libecap::Options API. To "receive" a meta header field from the host application, your adapter transaction should use the libecap::Options API of the corresponding libecap::host::Xaction. Needless to say, you have to use the same meta header field name in both directions.

You do not need to modify libecap or your host application to implement the above approach.

Please note that the libecap::Name class has a constructor that does not require assigning an ID to the name. For simplicity sake, you can use that constructor when naming your meta header field. If you want to optimize, then you can use the Name::NextId() function to get a name ID which makes name comparisons faster. The name ID in this paragraph has nothing to do with the unique transaction ID discussed earlier in this answer (they are completely different/unrelated IDs of different objects).

    // the name of the meta header field that ties requests and responses
    const libecap::Name XactionId("X-Xaction-ID", libecap::Name::NextId());

If you optimize by using NextId(), then for both REQMOD and RESPMOD code to have access to the _same_ name, it has to be created _once_ in adapter lifetime. Multiple calls to NextId() when creating Name objects will, naturally, give you different names.

P.S. Please note that due to retries by the host application, one request might result in several responses, all belonging to the same master transaction. In the extreme cases, one master transaction may even have multiple requests (e.g., Squid may send multiple same-master-transaction CONNECT requests for REQMOD adaptation when doing SslBump).
FAQ #2117: “How to pass information from REQMOD to RESPMOD?”.

    // the name of the meta header field that ties requests and responses
    const libecap::Name XactionId("X-Xaction-ID", libecap::Name::NextId());

I tried this either. "XactionId" is not a member of libecap.

How should I "register" such name? REQMOD adapter created it, but RESPMOD adapter tries to parse values from Names.h (or Names.cc, I don't know) and obviously it doesn't know about any "XactionId" objects.

Currently I have the working "example":
1. Modified libecap (names.cc and names.h) as described in message #2
names.cc:
+ const libecap::Name libecap::metaEcapID("X-Ecap-ID", libecap::Name::NextId());

names.h
+ extern const Name metaEcapID;

This way every adapter will know object "metaEcapID"

2. In REQMOD:
 const libecap::Area Adapter::Xaction::option(const libecap::Name &name) const {
        std::string filename = "abc"; (it's random string in reality, simplified in this comment)
 if (name == libecap::metaEcapID){}
             return libecap::Area::FromTempString(filename);
 return libecap::Area();
}
This is modified options implementation from ClamAV adapter.

3. In RESPMOD:
in Xaction::Start() function:
libecap::Area val = hostx->option(libecap::metaEcapID);

This way RESPMOD finds "metaEcapID" object in names.cc/names.h.
------------------------------------

Whenever I'm trying to send a new instance of libecap::Name, it's sent by REQMOD, but RESPMOD (at the time of compilation) doesn't know it and refuses to compile.
I appreciate all your answers, Alex Rousskov, but, honestly, libecap is extremely poorly documented, IMO.
I'm not C++ dev, yet 5+ years programming using Python, and those modules (called libraries in C++) are much better documented.
No offence, of course.

> const libecap::Name XactionId("X-Xaction-ID", libecap::Name::NextId());
> I tried this either. "XactionId" is not a member of libecap.

The error message you are quoting does not seem to match my sketch -- the sketch does not try to place or access XactionId in libecap namespace. I do not know what exactly went wrong (not enough information), but whatever it was, it is likely to be a basic C++ mistake not specific to eCAP. If you can find a local C++ developer, they should be able to resolve such problems within minutes by looking at your code and compiler errors.

> How should I "register" such name?

There is no need to register custom meta header field names names because the host application code does not need to know about them. On the adapters side, there are two cases:

1. A single loadable module contains two adaptation services (REQMOD and RESPMOD). In this case, a single XactionId object constructed with a NextId() parameter should work well. The two services will see the same object and will be able to interoperate. Just like two functions in the same library, the services share the same C++ naming space and the same set of objects in this case. From C++ point of view, it is just one library code.

2. Two loadable modules, each containing an adapter service. There are probably several ways to make things work in this case, but giving each adapter its own XactionId object may be the simplest. In this case, do not use the NextId() parameter when creating those distinct XactionId objects because NextId() will be called twice and, hence, return different values in each adapter/library, making the names effectively different. If you use just the name text/letters (i.e., using a libecap::Name constructor with a single parameter), then the names will be compared using their spelling, not IDs, and same-spelling names will be considered the same.

> REQMOD adapter created it, but RESPMOD adapter tries to parse values from Names.h (or Names.cc, I don't know) and obviously it doesn't know about any "XactionId" objects.

Any names specific to your adapter should be declared and defined in your custom adapter(s) code, not libecap code.

None of my recommendations suggest or rely on modifying libecap files! If you modify libecap files such as Name.cc for the purpose of making your adapter "work", then you are on your own.

> libecap is extremely poorly documented, IMO.

eCAP does not have enough overview-level documentation and lacks tutorials. Individual class and method documentation can be improved as well, although it is not "extremely poor". Quality patches welcomed.

However, your current primary problems have little to do with eCAP documentation AFAICT -- you simply do not know programming and C/C++ well enough to comfortably write what you want to write. Being a loadable compiled module with no dependency on the host application that loads it (and other similar modules!), an eCAP adapter has to operate in a fairly complex environment that is very different from a typical Python code environment. Migrating from Python to C++ is already a serious challenge; a direct migration from writing Python to writing complex dynamic libraries in C++ is likely to be insanely painful, especially when combined with limited fundamental programming knowledge. I am happy to answer specific eCAP questions, but such answers cannot fill such a huge gap (unfortunately).