Add parameters to empty POST body

Asked by Sergio Fukaya on 2016-07-28

Hi Alex,
I was wondering if you could help me. I'm new to squid and ecap. I'm struggling to get the big picture even after seeing the adapter samples and libecap sources (it's not clear what's the calling sequence of XActions method, who calls them too). I would like to create a reqmode adapter that adds username and password to a POST request to a SSL login URL (I'm using Squid sneak and peek).
I started using adapter_modifying, changed squid.conf to reqmode_precache. But I'm not struggling to add a body. The adapter sample removes the content-length header, so it's send as 0. And at Service Start it tests if body is empty and seems to call lasthostcall or something.. I tried to modify vbContentDone to add my parameters to the buffer variable, but I'm feeling that it has not been calling. The content length is still 0 and no parameters are sent.
Could you give me some directions of what I should have done?
Thanks in advance!

Best Regards,

Question information

English Edit question
eCAP Edit question
No assignee Edit question
Last query:
Last reply:

This question was reopened

Alex Rousskov (rousskov) said : #1

> Could you give me some directions of what I should have done?

It will be difficult for you to accomplish what you want with your current level of [mis]understanding of how HTTP and eCAP work. If you cannot outsource your work, I recommend the following steps:

1. Start by making the sample modifying adapter work as is in RESPMOD, without modifying the adapter code. This will give you a working example that you can use to track calling sequences and such.
2. Adjust the sample modifying adapter to work in REQMOD. I believe the easiest way to do that is to preserve the old Content-Length header instead of deleting it. This means that you should configure the adapter to have the same victim and replacement lengths (so that the body length does not change when the body content changes). Make no other changes at this stage. Make sure that everything works and that you understand how it works.
3. Modify the adapter to buffer the entire request body (if any) before sending the adapted request to the host application. This will be difficult unless you understand how eCAP works. This buffering will hurt performance but it will simplify future Content-Length handling because the correct adapted Content-Length value will be known when the adapter accumulates the entire virgin request and adapts its body before giving the adapted message to the host application. When you are done, the adapter will support victim and replacement strings of different lengths.
4. Modify the adapter to make the body adaptations you actually need/want. This step will be easy.

If you are stuck, trying asking specific questions, one question per Answers entry.

> The adapter sample removes the content-length header, so it's send as 0.

Removing a header field does not make it zero. If you see a zero Content-Length header than your adapter probably gave the host application a zero-length adapted body OR the host application decided that it needs a Content-Length header for the request and added one with a bogus value.

> And at Service Start it tests if body is empty and seems to call lasthostcall or something..

Service::start() does not and cannot look at the message body at all -- there are no HTTP messages yet when the service is started. You might be thinking about Xaction::start() instead. Xaction::start() calls lastHostCall()->useAdapted() if there is no body in the virgin message. This is normal -- not all HTTP messages have bodies.

Alex Rousskov (rousskov) said : #2

> the host application decided that it needs a Content-Length header for the request and added one with a bogus value.

Actually, the value of zero is not bogus if the adapted POST message has an empty body. In that case, Content-Length:0 is actually correct!

Please keep in mind that the host application may need to know the request body length at useAdapted() call time. HTTP/1.1 chunking allows sending an HTTP request without a Content-Length header, but not all origin servers support that, and the host application itself may insist on knowing the body length a priori. If your adapter buffers and adapts the request before giving the adapted request to the host application, it will be able to supply the right Content-Length value at useAdapted() time.

Sergio Fukaya (sfukaya) said : #3

Hi Alex,

Thank you for your prompt reply. Unfortunately, I have no means to outsource my work. Actually, I'm trying to make a PoC that we could hide the username and password from the user. The problem is that I have an access to an Internet Portal services for 10 concurrent users, but the provider creates only 1 user account that should be shared with all users. To make things worse, the options to change the contact e-mail and password are available.

I started to see if I could hide the username and password with Apache MOD_PROXY and other modules, it seems that it only apply to GET submissions. And I couldn't find a way to intercept SSL messages. I need to hide the users credentials of the authentication request and delete (replace) the HTML for changing email and password of the response. That's how I came here.

I have read many articles on how to configure squid with eCAP and some iCAP, but none of them give a glimpse of how eCAP actually works. I also searched quite a lot here, but I couldn't figure out yet how it works. You're right! I need more HTTP and eCAP background. It would be great if there was some kind of a picture / diagram showing squid (this would be the host application, right?) calling the ECAP Service / Adapter. I believe that this is some kind of state machine and there is something (perhaps the way HTTP works) that performs the control flow (calls the methods).

For the request adapter I had done the following:

void Adapter::Xaction::noteVbContentDone(bool atEnd)
    //Debug(flXaction) << "Adapter::Xaction::noteVbContentDone hostx=" << hostx;
 const std::string &username = service->username;
 const std::string &password = service->password;

 std::ostringstream ostr; //output string stream

    Must(receivingVb == opOn);
 if (sendingAb == opOn) {
        buffer += "username=" + username + "&pw=" + password;
        ostr << "" << (buffer.size()); //use the string stream just like cout,
        hostx->adapted().header().add(libecap::headerContentLength, libecap::Area::FromTempString(ostr.str()));

  sendingAb = opComplete;

But it didn't work. All I know is that I need two adapters: one for the request (when successfully authenticated, it returns 302 and the client asks for other URL's) and other for the response with the victim HTML. Maybe I should try to do the response adapter first as you suggested.

Sorry for my mistake when I mentioned that Service::start() was testing the message body. I burnt the midnight oil trying to make my adapter work and wrote this question just before leaving home on my smartphone (I didn't have the code at that moment).

I will see if I can go further! Thanks for the tips!
Best Regards

Sergio Fukaya (sfukaya) said : #4

I forgot to mention that I am receiving the following message:

"Bad Request
Your browser sent a request that this server could not understand.

Reference #7.be8c62be.1469753913.112a16da "

Can you help with this problem?

Provide an answer of your own, or ask Sergio Fukaya for more information if necessary.

To post a message you must log in.