eCAP

the latest Modifying sample can not replace strings in response

Asked by Benquan Yu

Hello,

I'm trying to modifying sample, here is my Squid 3.2.1 (eCAP 0.2.0) code:

ecap_enable on
ecap_service req_sevice reqmod_precache 0 victim=Shipping replacement=Shiiiiiiiiipping uri=ecap://e-cap.org/ecap/services/sample/modifying
ecap_service resp_sevice respmod_precache 0 victim=Movies replacement=Moooooooovies uri=ecap://e-cap.org/ecap/services/sample/modifying
loadable_modules /usr/local/lib/ecap_adapter_modifying.so

adaptation_service_set reqFilter req_sevice
adaptation_service_set respFilter resp_sevice

adaptation_access reqFilter allow all
adaptation_access respFilter allow all

I can verify that the header was inserted corrected by the eCAP, but the world "Movies" in the HTML body were not replaced, can you give me a clue what's the reason? I have not change any code yet, just the original sample code fresh downloaded.

Question information

Language:
English Edit question
Status:
Solved
For:
eCAP Edit question
Assignee:
No assignee Edit question
Solved by:
Alex Rousskov
Solved:
Last query:
Last reply:
Revision history for this message
Benquan Yu (benquan) said : 		#1

My problem is that hostx->virgin().body() is always null. Why the host does not send body to ecap? Do I missed a key config option?

Revision history for this message
Alex Rousskov (rousskov) said : 		#2

Is it possible that you are looking at HTTP GET request or an HTTP response without a body (e.g., 304 Not Modified)? Most requests and some responses do not have bodies.

It may help if you test REQMOD separately from RESPMOD. Focus on one of them and make it work. Then do the other one. Then combine the two.

If you are still stuck, posting your Squid cache.log with debug_options set to ALL,9 while reproducing the problem using a _single_ request may help.

Revision history for this message
Benquan Yu (benquan) said : 		#3

Thanks Alex for looking into this. I changed to test RESPMOD only, and still, the body() in 0 in,

I added some debug statement like this:

class Debugger {
public:
  Debugger(int level) : ostr((libecap::MyHost()).openDebug(level)) {}
  ~Debugger() {
    (libecap::MyHost()).closeDebug(ostr);
  }
  std::ostream& debug() { return *ostr; }
private:
  std::ostream *ostr;
};

void Adapter::Xaction::start() {
 Must(hostx);
 Debugger debugger(6);
 debugger.debug() << "Adapter::Xaction::start()" << "virgin->body():" << (bool)(hostx->virgin().body())
  << " Header:" << hostx->virgin().header().image();
....
}
void Adapter::Xaction::noteVbContentAvailable()
{
        Debugger debugger(6);
 debugger.debug() << "noteVbContentAvailable";
......
}

and I can see start() get called (and there is no body), but noteVbContentAvailable() is not called. Log looks like this:
Adapter::Xaction::start()virgin->body():0 adapted->body():0 Header:HTTP/1.1 304 Not Modified

The page I tested was:
$ cat test.html
<html>
<body>
  I love Movies...
</body>
</html>

I can see the header in FireBug like this:
X-ECAP-Replace Movies to: Moooooooovies

But "Movies" the HTML body was not replaced.

The full cache.log is here: https://docs.google.com/file/d/0B6WBG8jlCbG1TXpBMWQ0ZFcyTk0/edit

Revision history for this message
Benquan Yu (benquan) said : 		#4

Posting the full start() function:

void Adapter::Xaction::start() {
 Must(hostx);
 Debugger debugger(6);
 debugger.debug() << "Adapter::Xaction::start()" << "virgin->body():" << (bool)(hostx->virgin().body())
  << " Header:" << hostx->virgin().header().image();

 if (hostx->virgin().body()) {
  receivingVb = opOn;
  hostx->vbMake(); // ask host to supply virgin body
 } else {
  // we are not interested in vb if there is not one
  receivingVb = opNever;
 }

 /* adapt message header */

 libecap::shared_ptr<libecap::Message> adapted = hostx->virgin().clone();
 Must(adapted != 0);

 // delete ContentLength header because we may change the length
 // unknown length may have performance implications for the host
 adapted->header().removeAny(libecap::headerContentLength);

 // add a custom header
 static const libecap::Name name("X-ECAP-Replace");
 const libecap::Header::Value value =
  libecap::Area::FromTempString(service->victim + " to: " + service->replacement);
 adapted->header().add(name, value);

 if (!adapted->body()) {
  sendingAb = opNever; // there is nothing to send
  lastHostCall()->useAdapted(adapted);
 } else {
  hostx->useAdapted(adapted);
 }
}

Revision history for this message
Best Alex Rousskov (rousskov) said : 		#5

> Adapter::Xaction::start()virgin->body():0 adapted->body():0 Header:HTTP/1.1 304 Not Modified

As I mentioned earlier, 304 responses do not have bodies. Your client and/or Squid is sending an If-Modified-Since or a similar conditional HTTP request and is getting an HTTP 304 Not Modified response back from the origin server.

To see response body in pre-cache RESPMOD, you need to clear browser and/or Squid caches or otherwise force a reload of the cached content.

Revision history for this message
Benquan Yu (benquan) said : 		#6

Thanks Alex, I get body now. And the reason the Modifying sample did not work is because the content is compressed. Does the host provide facility to compress/un-compress content? Or each adapter should handle it?

response head:
Content-Encoding gzip
Content-Type text/html
Transfer-Encoding chunked
Vary Accept-Encoding

Revision history for this message
Alex Rousskov (rousskov) said : 		#7

Currently, there is no eCAP mechanism to ask the host application to compress or decompress content-encoding. The host application is responsible for transfer-encoding only. The adapter has to do the rest.

Revision history for this message
Benquan Yu (benquan) said : 		#8

Thanks Alex Rousskov, that solved my question.