How to construct adopted content and send it?

Asked by Dajan Zvekic on 2011-01-31

What I want to do is to for every request that comes through my proxy to send custom html web page.
So I guess I need to adopt body of the content. This is example how to add a header.

libecap::shared_ptr<libecap::Message> adapted = hostx->virgin().clone();
Must(adapted != 0);
// delete ContentLength header because we may change the length
// unknown length may have performance implications for the host
adapted->header().removeAny(libecap::headerContentLength);
// add a custom header
static const libecap::Name name("X-Ecap");
const libecap::Header::Value value =libecap::Area::FromTempString(libecap::MyHost().uri());
adapted->header().add(name, value);

As I found out adopted message contains: first line, header, body and trailer. hostx->virgin.clone copies only header and not the body (is that so?), so I want to add a custom body to this adopted message (This should do the trick?), and I dont know how to do that.

If I was not clear in what I want, here is example. Someone is using my proxy, and it types in browser www.google.com, I want that my proxy send him a custom html page with some messages, so user will see something like this:

HELO WORLD!!!
OK

Question information

Language:
English Edit question
Status:
Solved
For:
eCAP Edit question
Assignee:
No assignee Edit question
Solved by:
Dajan Zvekic
Solved:
2011-02-09
Last query:
2011-02-09
Last reply:
2011-02-03
Alex Rousskov (rousskov) said : #1

To satisfy a virgin request with a custom response, you need libecap v0.1.0 or later. See that version announcement on this site for caveats.

There are two ways to do request satisfaction in eCAP. The simplest way is to block the request using the blockVirgin() method of the host transaction. The adapter will not have any control over the actual response in this case. See your host application documentation on how to customize block messages. For example, for Squid, see deny_info in squid.conf.documented. See also question #143521 on this site.

The second way is producing a brand new response as the adapted message, from scratch. This method gives the adapter full control over the response headers and body. To create a response in REQMOD, call MyHost().newResponse() method and customize the response as needed before calling useAdapted(). Do not forget to indicate that the response will have a body by calling addBody(). To actually supply the adapted message body to the host transaction, implement the ab*() methods of the adapter transaction. The modifying adapter in the sample shows how to implement them.

Dajan Zvekic (dajann) said : #2

Ok, I downloaded libecap v0.1.0 and squid branch from lp:~rousskov/squid/3p2-ecap.
But, I can not configure or compile it.
After running
%autoconf
i got errors:
configure.ac:11: error: possibly undefined macro: AM_INIT_AUTOMAKE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:14: error: possibly undefined macro: AM_MAINTAINER_MODE
configure.ac:34: error: possibly undefined macro: AM_PROG_CC_C_O
configure.ac:98: error: possibly undefined macro: AM_CONDITIONAL
configure.ac:104: error: possibly undefined macro: AC_ENABLE_SHARED
configure.ac:106: error: possibly undefined macro: AC_DISABLE_SHARED

and then after running:
%./configure --prefix=/usr/local/squid ..enable-ecap
I go this: configure: error: cannot find install-sh, install.sh, or shtool in cfgaux "."/cfgaux

and actually there is no cfgaux folder in downloaded branch.

Alex Rousskov (rousskov) said : #3

You need to bootstrap bzr sources first because Squid project does not version generated files. Try running bootstrap.sh.

Dajan Zvekic (dajann) said : #4

Ok.. I installed everything but now when I run squid, I am getting:
terminate called after throwing an instance of 'libecap::TextException'
  what(): Modifying Adapter: configuration error: victim value is not set
Aborted

I tried to comment line in modifying adapter where exception is thrown, and then I was able to start squid, but then proxy just didn't work.

I used same squid.conf file used for adapter version 0.0.3.. and it worked. :S

What is the problem?

Dajan Zvekic (dajann) said : #5

Ok, I solved victim problem reading one of older questions. So I am done with installing new versions and it actually works.
Now to my problem of creating new content. Will contact you when I have problem in this department.

Is it ok to post in this thread? Because question is actually about that, not the installation.

Dajan Zvekic (dajann) said : #6

Hi Alex,

I'm having trouble to understand what I need to do to construct and send new response.
This is what I think I understood up until now.

I want to construct new response and I will do something like this:

        libecap::shared_ptr<libecap::Message> adapted = libecap::MyHost().newResponse(); //create new response
 adapted->addBody(); //supply new response with body
 adapted->header().add(name,value); //supply body with headers

So, I took a look at ab* methods and I still can not understand how to construct new body and pass it to adopted message.
I am guessing that I need to construct new body in abMake() method. But how to construct it? (for example if I have simple html code, and I want that to be my body).

And I guess I do not need adaptContent() method, because I am constructing new response, not adapting existing one?

Thank you for your responses.

Alex Rousskov (rousskov) said : #7

Keep in mind that you cannot shovel the adapted body down host throat. The host application may not be ready to take the adapted body for various reasons. eCAP is using a "pull" model for bodies -- the recipient must ask for what it wants and can control body consumption. This is true for virgin and adapted bodies.

The adapter transaction indicates body availability by calling addBody() or adapting the virgin message with a body. If the host transactions may tell the adapter to make a body by calling abMake(). You may create the whole body in that method or just prepare to start making body, depending on your needs.

If abMake() has been called, the host transaction can call other ab*() methods to ask the adapter transaction for body pieces. Implement those methods as you need. The modifying adapter, for example, implements them to give the host transaction the modified body which the adapter stores in a buffer.

Keep in mind that, in general, bodies may be gigabytes in size and that virgin and adapted bodies should be allowed to flow in parallel (but in opposite directions) unless you want to stall message progress while you are accumulating the virgin body.

Message headers are "instantaneous" -- they are available at once to the recipient. Message bodies are, in general, not fixed blobs but _streams_ of data.

Alex Rousskov (rousskov) said : #8

s/If the host transactions may tell/The host transaction may tell/

Dajan Zvekic (dajann) said : #9

Hi Alex,

Your latest answer helped me sort some things in my head about adapters and content creation, but I do not fully understand couple of things.

1. I want to create new response, is this the way to do it?

Among other things in start() method I have this code.
void Adapter::Xaction:start(){

        libecap::shared_ptr<libecap::Message> createdResp = libecap::MyHost().newResponse();
 createdResp->header().add(name,value);
 createdResp->addBody();
 abMake();

}

Here I created a new response, added header and told that response have body with addBody() method. Then I called abMake to create body for response.

2. abMake() method implementation.

How do I make body? Let's say I assign something to buffer, how to say to adapter that this is body for created response?
void Adapter::Xaction::abMake{

buffer = "<html><body><h1>Forwarded here..</h1></body></html>"; //I want that this simple html code be my response body

}

So how to add this body to my new response and send that response?
I guess after creation of body I call noteAbContentAvailable(); //when buffer is not empty?, Or do I call abStopMaking() after creation of body.

3. Do I need to wait for whole request message body be received to create new body? It does not make sense. What would I want to do is to receive request headers, store this headers, and send my own body. For that I do not need to receive body of the request message?

Alex Rousskov (rousskov) said : #10

1. You do not call abMake(). Host transaction will call that method if the host decides it needs the adapted message body (which is usually the case, especially when you are making the message from scratch).

Do not forget to call useAdapted() when you are done constructing adapted headers.

2. You decide what the body content is. Your example looks fine to me, provided it matches the adapted headers.

Yes, after you have the first byte of the body (at least), you call noteAbContentAvailable(). The host transaction will probably call you back and take at least a part of the available body. Again, you do not push the body to the host transaction. The host transaction pulls it from the adapter transaction and calls abStopMaking() when it had enough. The adapter transaction just informs the host when more body bytes are available (until abStopMaking() has been called).

In most cases, adapter calls host methods and host calls adapter methods. For example, adapter calls vb*() methods of the host transaction and the host transaction calls ab*() methods of the adapter transaction.

3. No, you do not need to wait for the whole request body to produce your response body. And if you do not need the virgin body at all, call vbDiscard() instead of vbMake(). Do that as soon as you made the decision not to receive the virgin body while making adapted message.

Summary for your specific use case:

1. In start(), call newResponse(), add all headers, call useAdapted().
2. In abMake(), make (or start making) the adapted body and call noteAbContentAvailable().
3. Call noteAbContentDone() when you made everything (regardless of whether the host took the whole body yet). noteAbContentDone() may be called from abMake() after noteAbContentAvailable() if you made the entire body at once.
4. Implement other pure virtual ab*() methods to feed adapted body to the host. See sample adapters for examples.

Keep in mind that the host application may call your abMake() from inside of the useAdapted() call. Similarly, other ab*() methods may be called while receiving your noteAbContentAvailable() notification.

The above ignores the fact that you may need to do different things to different virgin messages. For example, satisfying a request to an image with an HTML response is usually not a good idea. There are many other complications like dealing with HTML frames.

Dajan Zvekic (dajann) said : #11

Thank you a lot.
I succeeded in implementing simple version of needed use case.
Your inputs were very very helpful. :))