eCAP

Question about loading ecap-adapter module on cygwin

Asked by HDM1991

Hi.
Recently, I want to run squid with eCAP feature on windows.

Firstly, I compiled eCAP-0.2.0 on cygwin. As a result, I some files as follow.
--------------------------eCAP-----------------------------------------
DDD@DDD-PC /usr/local/lib
$ ls
libecap.a libecap.la pkgconfig
--------------------------eCAP end-----------------------------------------

There is only libecap.a. But I still squid-3.3.3-2 with --enable-ecap option successfuly on cygwin.
Then, I compile ecap_adapter_sample-0.2.1. I get some files as following.
---------------------------ecap_adapter_sample-0.2.1--------------------------------
DDD@DDD-PC /usr/local/lib
$ ls
ecap_adapter_minimal.a ecap_adapter_minimal.la
ecap_adapter_modifying.a ecap_adapter_modifying.la
ecap_adapter_passthru.a ecap_adapter_passthru.la libecap.a libecap.la
pkgconfig
---------------------------ecap_adapter_sample-0.2.1 end---------------------------

Clearly, Squid can't load it. When I configure Squid to load it, I
just these error message.
--------------------------Squid error message-------------------------------------
$ ./squid.exe -N -C -d1
2014/10/20 23:33:26| WARNING cache_mem is larger than total disk cache space!
2014/10/20 23:33:26| Starting Squid Cache version 3.3.3 for i686-pc-cygwin...
2014/10/20 23:33:26| Process ID 3200
2014/10/20 23:33:26| Process Roles: master worker
2014/10/20 23:33:26| With 3072 file descriptors available
2014/10/20 23:33:26| Initializing IP Cache...
2014/10/20 23:33:26| DNS Socket created at [::], FD 4
2014/10/20 23:33:26| DNS Socket created at 0.0.0.0, FD 5
2014/10/20 23:33:26| Adding nameserver 8.8.8.8 from squid.conf
2014/10/20 23:33:26| Logfile: opening log daemon:/var/log/squid/access.log
2014/10/20 23:33:26| Logfile Daemon: opening log /var/log/squid/access.log
2014/10/20 23:33:26| WARNING: no_suid: setuid(0): (22) Invalid argument
2014/10/20 23:33:26| WARNING: no_suid: setuid(0): (22) Invalid argument
2014/10/20 23:33:27| Unlinkd pipe opened on FD 11
2014/10/20 23:33:27| Store logging disabled
2014/10/20 23:33:27| Swap maxSize 102400 + 262144 KB, estimated 28041 objects
2014/10/20 23:33:27| Target number of buckets: 1402
2014/10/20 23:33:27| Using 8192 Store buckets
2014/10/20 23:33:27| Max Mem size: 262144 KB
2014/10/20 23:33:27| Max Swap size: 102400 KB
2014/10/20 23:33:27| Rebuilding storage in /var/cache/squid (dirty log)
2014/10/20 23:33:27| Using Least Load store dir selection
2014/10/20 23:33:27| Set Current Directory to /var/cache/squid
2014/10/20 23:33:27| Loaded Icons.
2014/10/20 23:33:27| HTCP Disabled.
2014/10/20 23:33:27| Loading Squid module from
'/usr/local/lib/ecap_adapter_modifying.so'
2014/10/20 23:33:27| FATAL: dying from an unhandled exception: %1 is
not a valid Win32 application.
terminate called after throwing an instance of 'TextException'
what(): %1 is not a valid Win32 application.
Aborted (core dumped)
--------------------------Squid error message end------------------------------------

A .dll is necessary.
So, I recompiled it as .dll by writing compile and link command by myself.
------------------------compile and link command-------------------------------------
g++ -DHAVE_CONFIG_H -I../src -I/usr/local/include -g -O3 -Wall -Wwrite-strings -Woverloaded-virtual -pipe -MT adapter_modifying.lo -MD -MP -MF .deps/adapter_modifying.Tpo -c adapter_modifying.cc -DDLL_EXPORT -DPIC -o adapter_modifying.o

g++ -shared -o cygmodify.dll -Wl,--out-implib=libmodify.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive adapter_minimal.o -Wl,--no-whole-archive /usr/local/lib/libecap.a
------------------------compile and link command end-------------------------------------

But, when I configure squid to load it, it does't work well.
Here are the message than squid output.
-----------------------squid message-------------------------------------------
HDM@HDM-PC /cygdrive/c/squdid/sbin
$ ./squid.exe -N -C -d1
2014/10/22 19:29:30| WARNING cache_mem is larger than total disk cache space!
2014/10/22 19:29:30| Starting Squid Cache version 3.3.3 for i686-pc-cygwin...
2014/10/22 19:29:30| Process ID 2040
2014/10/22 19:29:30| Process Roles: master worker
2014/10/22 19:29:30| With 3072 file descriptors available
2014/10/22 19:29:30| Initializing IP Cache...
2014/10/22 19:29:30| DNS Socket created at [::], FD 4
2014/10/22 19:29:30| DNS Socket created at 0.0.0.0, FD 5
2014/10/22 19:29:30| Adding nameserver 8.8.8.8 from squid.conf
2014/10/22 19:29:30| Logfile: opening log daemon:/cygdrive/c/squdid/var/logs/access.log
2014/10/22 19:29:30| Logfile Daemon: opening log /cygdrive/c/squdid/var/logs/access.log
2014/10/22 19:29:30| WARNING: no_suid: setuid(0): (22) Invalid argument
2014/10/22 19:29:30| WARNING: no_suid: setuid(0): (22) Invalid argument
2014/10/22 19:29:30| Unlinkd pipe opened on FD 11
2014/10/22 19:29:30| Store logging disabled
2014/10/22 19:29:30| Swap maxSize 102400 + 262144 KB, estimated 28041 objects
2014/10/22 19:29:30| Target number of buckets: 1402
2014/10/22 19:29:30| Using 8192 Store buckets
2014/10/22 19:29:30| Max Mem size: 262144 KB
2014/10/22 19:29:30| Max Swap size: 102400 KB
2014/10/22 19:29:30| Rebuilding storage in /usr/local/squid/var/cache/squid (dirty log)
2014/10/22 19:29:30| Using Least Load store dir selection
2014/10/22 19:29:30| Set Current Directory to /usr/local/squid/var/cache/squid
2014/10/22 19:29:30| Loaded Icons.
2014/10/22 19:29:30| HTCP Disabled.
2014/10/22 19:29:30| Loading Squid module from '/usr/local/lib/cygmodify.dll'
2014/10/22 19:29:30| Squid plugin modules loaded: 1
2014/10/22 19:29:30| Adaptation support is on
2014/10/22 19:29:30| WARNING: configured ecap_service was not loaded: ecap://e-cap.org/ecap/services/sample/modifying
2014/10/22 19:29:30| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 14 flags=9
2014/10/22 19:29:30| Done reading /usr/local/squid/var/cache/squid swaplog (8 entries)
2014/10/22 19:29:30| Finished rebuilding storage from disk.
2014/10/22 19:29:30| 8 Entries scanned
2014/10/22 19:29:30| 0 Invalid entries.
2014/10/22 19:29:30| 0 With invalid flags.
2014/10/22 19:29:30| 8 Objects loaded.
2014/10/22 19:29:30| 0 Objects expired.
2014/10/22 19:29:30| 0 Objects cancelled.
2014/10/22 19:29:30| 0 Duplicate URLs purged.
2014/10/22 19:29:30| 0 Swapfile clashes avoided.
2014/10/22 19:29:30| Took 0.02 seconds (512.82 objects/sec).
2014/10/22 19:29:30| Beginning Validation Procedure
2014/10/22 19:29:30| Completed Validation Procedure
2014/10/22 19:29:30| Validated 8 Entries
2014/10/22 19:29:30| store_swap_size = 364.00 KB
2014/10/22 19:29:31| storeLateRelease: released 0 objects
-----------------------squid message end-------------------------------------------

I’m confused about this warning. Does anybody give me some hints. Thanks.

HDM1991

Question information

Language:
English Edit question
Status:
Solved
For:
eCAP Edit question
Assignee:
No assignee Edit question
Solved by:
Alex Rousskov
Solved:
Last query:
Last reply:

This question was reopened

Revision history for this message
Best Alex Rousskov (rousskov) said : 		#1

> 2014/10/22 19:29:30| Loading Squid module from '/usr/local/lib/cygmodify.dll'
> 2014/10/22 19:29:30| Squid plugin modules loaded: 1
> 2014/10/22 19:29:30| Adaptation support is on
> 2014/10/22 19:29:30| WARNING: configured ecap_service was not loaded: ecap://e-cap.org/ecap/services/sample/modifying

When a host application like Squid loads an eCAP adapter module, the adapter must register its eCAP services with the host application. Services are identified by their URIs. Host configuration (e.g., squid.conf) contains references to those URIs. The configuration and loaded services must match. In your example, Squid loads /usr/local/lib/cygmodify.dll, but the loaded DLL code does not register any services at all OR does not register any services with the ecap://e-cap.org/ecap/services/sample/modifying URI. The former is more likely IMHO.

In sample adapters, registration is implemented using C++ static initialization. The libecap/common/registry.h header file is a good starting point to research the details of the registration mechanism.

I do not know much about Windows and cygwin, but I can speculate that one or more of the following cases are happening:

1. Your cygwin-built(?) Squid does not activate (i.e., does not call) C++ static initialization code when loading DLLs.

2. The DLL itself lacks C++ static initialization code for some reason. There is nothing for Squid to call.

3. Squid static initialization activation code is incompatible with the DLL static initialization code in some ways (e.g., the two are using different naming conventions).

If you are a developer, you should be able to identify which of the above cases are true by adding debugging and/or running Squid under debugger. This will not be easy, but should be possible. Reading general documentation about static initialization in DLLs and in cygwin-built DLLs may also help.

The above may give you some hints, but it is not really an answer to the "How to make eCAP work on Windows?" question you seem to be asking. Still, I will mark this as a "Proposed Answer" so that Launchpad does not automatically expire and delete your question. If you figure this out, please do post an update with instructions here! If you do not figure it out, others might benefit from the information you have already provided (thank you!) and the above hints.

Revision history for this message
HDM1991 (lionxyes) said : 		#2

Thanks a lot. These hints are very helpful.

I will do my best to solve it. I'm a coder.

Revision history for this message
HDM1991 (lionxyes) said : 		#3

Thanks Alex Rousskov, that solved my question.

Revision history for this message
HDM1991 (lionxyes) said : 		#4

Ah, I think I find the reason.

Wether when I link Squid or ecap-adapter module, I linked libecap.a that is a static library.
It should not be like this. Squid and ecap-adapter module should link libecap.dll(or liecap.so).

When Squid and ecap-adapter module linked libecap.a，they all have a copy of libecap. There two copy
libecap in memory. That's not right. Libeap should be the middle of Squid between ecap-adapter module.
but in this case, obviously, It's not.

What do you think of this?

Revision history for this message
HDM1991 (lionxyes) said : 		#5

I have solved this problem.

It's neccessary that Squid and ecap-adapter module link llibecap.dll(or libecap.so).

Now, Squid is running with eCAP feature on my winows virtual machine.

Thank you very much.

Revision history for this message
Alex Rousskov (rousskov) said : 		#6

Glad you solved the problem! If possible, please post the (manual?) compilation commands you had to use so that others may reproduce your results.

Revision history for this message
HDM1991 (lionxyes) said : 		#7

Sorry. Let's you waiting so long. Here they are.

Ready
Step 1: Install cygwin.

    https://cygwin.com/install.html

Step 2: Get squid-3.3.3-2 src by cygwin setup-x86.exe or setup-x86_64.exe. They are GUI package manager of cygwin.

    HDM@HDM-PC /usr/src/squid-3.3.3-2.src
    $ ls
    3.3.3-cygwin.patch squid.cygport squid-3.3.3.tar.bz2 squid-3.3.3.tar.bz2.asc

    3.3.3-cygwin.patch is used to patch squid-3.3.3-2 src.
    squid.cygprot. we will introduce it later.

Step 3: Install cygport by setup-x86.exe or setup-x86_64.exe. We will use cygport to unpack sources, apply patches and compile squid-3.3.3.-2. squid.cygprot tell cygport how to compile squid-3.3.3-3. Please look the content of it.

Compile eCAP
Step 1: compile ecap-0.2.0 as before.

    HDM@HDM-PC /usr/src/ecap-0.2.0
    $ ./configure && make

Now, We get libcommon.a, libadapter.a and libhost.a

Step 2: link it by runing command as following.

    HDM@HDM-PC /usr/src/libecap-0.2.0/src/libecap
    $ g++ -shared -o cygecap.dll -Wl,--out-implib=libecap.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive common/.libs/libcommon.a adapter/.libs/libadapter.a host/.libs/libhost.a -Wl,--no-whole-archive -lstdc++ -lm -lc

    HDM@HDM-PC /usr/src/libecap-0.2.0/src/libecap
        $ ls | grep dll || a
        cygecap.dll
        libecap.dll.a

Compile Squid-3.3.3-2
Step 1: copy libecap.dll.a to /usr/lib and copy cygecap.dll to /usr/bin

    HDM@HDM-PC /usr/lib
    $ ls | grep libecap
    libecap.dll.a
    HDM@HDM-PC /usr/bin
    $ ls | grep cygecap
    cygecap.dll

Step 2: Add --enable-ecap option by editing squid.cygport file
////////////////////////squid.cygport/////////////////////////
.
.
.
src_compile() {
        cd ${S}
        ./bootstrap.sh
        lndirs
        cd ${B}
        cygconf \
          --sysconfdir=/etc/squid \
          --datadir=/usr/share/squid \
          --libexecdir=/usr/lib/squid \
          --disable-strict-error-checking \
          --with-logdir=/var/log/squid \
          --with-swapdir=/var/cache/squid \
          --with-pidfile=/var/run/squid.pid \
          --enable-ecap \
          --enable-ssl \
          --enable-esi \
          --enable-disk-io="AIO,Blocking,DiskThreads,IpcIo,Mmapped" \
          --enable-auth-basic="DB,LDAP,MSNT,MSNT-multi-domain,NCSA,POP3,RADIUS,SASL,SMB,fake,getpwnam" \
          --enable-auth-ntlm='fake,smb_lm' \
          --enable-auth-negotiate='kerberos,wrapper' \
          --enable-external-acl-helpers='LDAP_group,SQL_session,eDirectory_userip,file_userip,kerberos_ldap_group,session,time_quota,unix_group,wbinfo_group' \
          --with-filedescriptors=3072

        # FIXME: error: 'snprintf' was not declared in this scope
        for fn in `find . -name Makefile`
        do
          sed -i -e "s#c++0x#gnu++0x#" ${fn}
        done

        make
}
.
.
.
///////////////////////////////////squid.cygport////////////////////////////

Step 3: compile squid-3.3.3-2 src. Then we can get .o files.

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src
  $ ls
  3.3.3-cygwin.patch squid.cygport squid-3.3.3.tar.bz2 squid-3.3.3.tar.bz2.asc squid-3.3.3-2.i686

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src
  $ cygport --32 squid.cygport prep compile install

Step 4: link it by runing command as following.

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src/squid-3.3.3-2.i686/src/squid-3.3.3/src
  $ g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments -pipe -D_REENTRANT -g -O2 -std=gnu++0x .libs/squid.exeS.o -g -o squid.exe AclRegs.o AuthReg.o AccessLogEntry.o AsyncEngine.o YesNoNone.o cache_cf.o CacheDigest.o cache_manager.o carp.o cbdata.o ChunkedCodingParser.o client_db.o client_side.o client_side_reply.o client_side_request.o BodyPipe.o clientStream.o CompletionDispatcher.o ConfigOption.o ConfigParser.o CpuAffinity.o CpuAffinityMap.o CpuAffinitySet.o debug.o disk.o DiskIO/DiskIOModule.o DiskIO/ReadRequest.o DiskIO/WriteRequest.o dlink.o dns_internal.o DnsLookupDetails.o errorpage.o ETag.o event.o EventLoop.o external_acl.o ExternalACLEntry.o FadingCounter.o fatal.o fd.o fde.o filemap.o forward.o fqdncache.o ftp.o gopher.o helper.o HelperChildConfig.o htcp.o http.o HttpStatusLine.o HttpHdrCc.o HttpHdrRange.o HttpHdrSc.o HttpHdrScTarget.o HttpHdrContRange.o HttpHeader.o HttpHeaderTools.o HttpBody.o HttpMsg.o HttpParser.o HttpReply.o RequestFlags.o HttpRequest.o HttpRequestMethod.o icp_v2.o icp_v3.o int.o internal.o ipc.o ipcache.o SquidList.o main.o mem.o mem_node.o MemBuf.o MemObject.o mime.o mime_header.o multicast.o neighbors.o Packer.o Parsing.o pconn.o peer_digest.o peer_proxy_negotiate_auth.o peer_select.o peer_sourcehash.o peer_userhash.o redirect.o refresh.o RemovalPolicy.o send-announce.o MemBlob.o snmp_core.o snmp_agent.o SquidMath.o SquidNew.o stat.o StatCounters.o StatHist.o String.o StrList.o stmem.o store.o StoreFileSystem.o store_io.o StoreIOState.o store_client.o store_digest.o store_dir.o store_key_md5.o store_log.o store_rebuild.o store_swapin.o store_swapmeta.o store_swapout.o StoreMeta.o StoreMetaMD5.o StoreMetaSTD.o StoreMetaSTDLFS.o StoreMetaUnpacker.o StoreMetaURL.o StoreMetaVary.o StoreStats.o StoreSwapLogData.o Server.o SwapDir.o MemStore.o time.o tools.o tunnel.o unlinkd.o url.o URLScheme.o urn.o wccp.o wccp2.o whois.o wordlist.o LoadableModule.o LoadableModules.o DiskIO/DiskIOModules_gen.o err_type.o err_detail_type.o globals.o hier_code.o icp_opcode.o lookup_t.o repl_modules.o swap_log_op.o DiskIO/Blocking/BlockingDiskIOModule.o DiskIO/DiskThreads/DiskThreadsDiskIOModule.o DiskIO/IpcIo/IpcIoDiskIOModule.o DiskIO/Mmapped/MmappedDiskIOModule.o -Wl,--export-all-symbols auth/.libs/libacls.a ident/.libs/libident.a acl/.libs/libacls.a acl/.libs/libstate.a auth/.libs/libauth.a libBlocking.a libDiskThreads.a libIpcIo.a libMmapped.a acl/.libs/libapi.a base/.libs/libbase.a ./.libs/libsquid.a ip/.libs/libip.a fs/.libs/libfs.a ipc/.libs/libipc.a mgr/.libs/libmgr.a anyp/.libs/libanyp.a comm/.libs/libcomm.a eui/.libs/libeui.a icmp/.libs/libicmp.a icmp/.libs/libicmp-core.a log/.libs/liblog.a format/.libs/libformat.a repl/liblru.a -lpthread -lcrypt adaptation/.libs/libadaptation.a -L/usr/local/lib esi/.libs/libesi.a ../lib/libTrie/src/libTrie.a /usr/lib/libexpat.dll.a ssl/.libs/libsslsquid.a ssl/.libs/libsslutil.a snmp/.libs/libsnmp.a ../snmplib/libsnmplib.a ../lib/.libs/libmisccontainers.a ../lib/.libs/libmiscencoding.a ../lib/.libs/libmiscutil.a -lssl -lcrypto -L../compat -lcompat-squid -L.. /usr/lib/libltdl.dll.a /usr/lib/libecap.dll.a

  Now,we can get the new squid.exe.

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src/squid-3.3.3-2.i686/src/squid-3.3.3/src
  $ find . -name squid.exe
  ./.libs/squid.exe
  ./squid.exe

Compile ecap_adapter_sample-0.2.1
Step 1: compile ecap-0.2.0 as usually.
  HDM@HDM-PC /usr/src/ecap_adapter_sample-0.2.1
  $ ./configure && make

Step 2: link it by runing command as following.
  HDM@HDM-PC /usr/src/ecap_adapter_sample-0.2.1/src
  $ g++ -shared -o cygmodifying.dll -Wl,--out-implib=libmodifying.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive adapter_modifying.o -Wl,--no-whole-archive /usr/lib/libecap.dll.a

  HDM@HDM-PC /usr/src/ecap_adapter_sample-0.2.1/src
  $ g++ -shared -o cygrecord.dll -Wl,--out-implib=librecord.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive adapter_passthru.o -Wl,--no-whole-archive /usr/lib/libecap.dll.a

  HDM@HDM-PC /usr/src/ecap_adapter_sample-0.2.1/src
  $ g++ -shared -o cygminimal.dll -Wl,--out-implib=librecord.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive adapter_minimal.o -Wl,--no-whole-archive /usr/lib/libecap.dll.a

Ok. It' over.

Other
As you see, we compile squid-3.3.3-2 by cygport in the above steps. We also can compile by ourself by parsing squid.cygprot. So, we can control the install dir option etc. Here are my method.

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src/squid-3.3.3-2.i686/src/squid-3.3.3
  $ ./configure --prefix=/cygdrive/c/squid --disable-strict-error-checking --enable-ecap --enable-ssl --enable-esi --enable-disk-io="AIO,Blocking,DiskThreads,IpcIo,Mmapped" --enable-auth-basic="DB,LDAP,MSNT,MSNT-multi-domain,NCSA,POP3,RADIUS,SASL,SMB,fake,getpwnam" --enable-auth-ntlm='fake,smb_lm' --enable-auth-negotiate='kerberos,wrapper' --enable-external-acl-helpers='LDAP_group,SQL_session,eDirectory_userip,file_userip,kerberos_ldap_group,session,time_quota,unix_group,wbinfo_group' --with-filedescriptors=3072

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src/squid-3.3.3-2.i686/src/squid-3.3.3
  $ ./test.sh

  HDM@HDM-PC /usr/src/squid-3.3.3-2.src/squid-3.3.3-2.i686/src/squid-3.3.3
  $ make && make install

  -------------------------test.sh----------------------------------------
  #!/bin/bash

  for fn in `find . -name Makefile`
  do
    sed -i -e "s#c++0x#gnu++0x#" ${fn}
  done
  -------------------------test.sh end ----------------------------------------

Morely, I write link command by myself because I don't know well about g++, automake, autoconf. So, maybe someone else can improve it. Thank you.

Revision history for this message
HDM1991 (lionxyes) said : 		#8

Thanks Alex Rousskov, that solved my question.