App stats problem

Hello,

The periodic refreshes in FO and IR states may be configured by the rohc_comp_set_periodic_refreshes() function of the library API. By default, they are set to 700 and 1700 packets. With the default values, you cannot see any periodic refreshes with network captures with less than 1700 packets.

That is not the case with the capture that you use:
    $ capinfos test/non_regression/inputs/ipv6/udp/rtp/audio-vlc/source.pcap | grep ^Number
    Number of packets: 4490

So, I checked what was wrong. There is a bug. Some profiles such as UDP or RTP do not run the periodic refreshes code for every packets. Please try with the following patch:

=== modified file 'src/comp/c_generic.c'
--- src/comp/c_generic.c 2014-06-21 10:59:41 +0000
+++ src/comp/c_generic.c 2015-03-22 20:39:58 +0000
@@ -870,6 +870,10 @@ int c_generic_encode(struct rohc_comp_ct
        /* decide in which state to go */
        assert(g_context->decide_state != NULL);
        g_context->decide_state(context);
+ if(context->mode == ROHC_U_MODE)
+ {
+ periodic_down_transition(context);
+ }

        /* compute how many bits are needed to send header fields */
        if(!encode_uncomp_fields(context, uncomp_pkt))
@@ -1463,11 +1467,6 @@ void decide_state(struct rohc_comp_ctxt
        }

        change_state(context, next_state);
-
- if(context->mode == ROHC_U_MODE)
- {
- periodic_down_transition(context);
- }
 }

Regards,
Didier

Thank you for your answer.

I change the code with your patch. I am now able to see the change FO->IR after 1700 packets.

However, there is still some problems concerning the FO_TIEMOUT. Indeed, 3 SO pakcets are send, then it resend FO packets... I used the default number. Is it normal?

Thank you again

I have another question.

Is it possible to study the behaviour of the decompressor when the rohc packet is damaged?

I know that i can create the rohc packet with the non_regression_test, and i can compare it with the packet before the compression. but is it possible to damaged te packet (high BER, for example) to sutdy the decompressor behavior?

thank you again

Hello,

> I change the code with your patch. I am now able to see the change
> FO->IR after 1700 packets.

Great! Thank you for testing. I'll commit this change for the next 1.7.x bugfix release.

> However, there is still some problems concerning the FO_TIEMOUT.
> Indeed, 3 SO pakcets are send, then it resend FO packets... I used the
> default number. Is it normal?

I'm not sure to understand what you write about. There is no FO_TIMEOUT parameter in RFC 3095 nor in the source code of the ROHC library. To better understand your question/problem, could you please point me on a specific paragraph of the RFC or at a specific snippet of code from the ROHC library?

> Is it possible to study the behaviour of the decompressor when the
> rohc packet is damaged?
>
> I know that i can create the rohc packet with the non_regression_test,
> and i can compare it with the packet before the compression. but is it
> possible to damaged te packet (high BER, for example) to sutdy the
> decompressor behavior?

The non-regression tool is not able to damage ROHC packets, however the library provides different tools to test its robustness.

The ./test/robustness/damaged_packet/test_damaged_packet tool compresses a flow of uncompressed packets, damages voluntary one of them and then checks that the last packet is of the expected type. The tool is able to try packet/context repair (as described in the RFC 3095) if asked to do so. Run it without any argument for more help:
 $ ./test/robustness/damaged_packet/test_damaged_packet

The ./test/robustness/lost_packet/test_lost_packet tool compresses a flow of uncompressed packets, loses one or more packets and then checks how many packets fail to be decompressed because of the lost packets. Run it without any argument for more help:
 $ ./test/robustness/lost_packet/test_lost_packet

The ./test/robustness/malformed_rohc_packets/test_malformed_rohc_packets tool decompresses a flow of ROHC packets and checks that the decompression fails for the last N packets. Run it without any argument for more help:
 $ ./test/robustness/malformed_rohc_packets/test_malformed_rohc_packets

Let me know if those tools fit your need or not.

Regards,
Didier

Hello,

Im sorry, I wanted to talk about the CHANGE_FO_COUNT. Indeed, using the same test, im able to see the switch between the FO->IR after 1700 packets (CHANGE_IR_COUNT) , BUT im not able to see the switch between SO->FO after 700 packets (CHANGE_FO_COUNT). Is it normal?

Thank you for the different tests, i will use it soon.

Thank you

Hello,

How to you check for the SO->FO periodic transitions?

On my side, I run the rohc_stats application through the rohc_stats.sh wrapper that runs the app_stats application and then generates some graphs with gnuplot:
  $ ./app/stats/rohc_stats.sh test/non_regression/inputs/ipv6/udp/rtp/audio-vlc/source.pcap /tmp/stats/
  $ firefox /tmp/stats/index.html

The graph entitled "Context states" shows that the number of times the library compressed a packet in the SO, FO and IR states.

Regards,
Didier

Hello,

thank you for your answer.

I have a little problem with firefox, that the reason why im not able to generate the graph.

However, the app create a raw.txt file, where im able to see differents caracteristics: packet's number, size before and after compression... and the packet type. so i use this file to see the switch between each states.

Is it clear?

thank you

Hello Didier,

I have a question concerning the ROHC feedback channel and the app stats.

Did the app stats consider that this feedback channel is present? Did it use it ?

Thank you

Hello Didier,

I have a question (again). Indeed, when i use the app stats for an ipv4/tcp packets, im not able to see any switch after the CHANGE_IR_COUNT and CHANGE_FO_COUNT. Im only able to see the switch in the first nine packets. Is it normal?

Thank you again

Hello,

> However, the app create a raw.txt file, where im able to see differents caracteristics:
> packet's number, size before and after compression... and the packet type. so i use this
> file to see the switch between each states.

Using the raw output of the rohc_stats tool is OK. There is an example for SO->FO transition at packet 3371:
  STAT 3370 1 U-mode 3 SO 8 RTP/UOR-2 1376 60 1327 11
  STAT 3371 1 U-mode 2 FO 8 RTP/UOR-2 1376 60 1327 11

It is not easy to see why from those lines alone. The library logs helped me a lot. Search for a debug trace that contains:
   timeouts for periodic refreshes: FO = 700 / 700

> I have a question concerning the ROHC feedback channel and the app stats.
>
> Did the app stats consider that this feedback channel is present? Did it use it ?

No, the rohc_stats tool doesn't handle feedback yet. It could be easily added however. The test/non_regression/test_non_regression.c tool is an example of how a feedback channel may be added.

> I have a question (again). Indeed, when i use the app stats for an ipv4/tcp
> packets, im not able to see any switch after the CHANGE_IR_COUNT and
> CHANGE_FO_COUNT. Im only able to see the switch in the first nine
> packets. Is it normal?

If you use the IPv4/TCP capture included with the ROHC library, that's expected since the capture contains too few packets:
    $ capinfos ./test/non_regression/inputs/ipv4/tcp/source.pcap | grep ^Number
    Number of packets: 18

I have just tested with a longer capture: it indeed doesn't work. Periodic refreshes were not implemented in the TCP profile yet. I have tried to add support for refreshes. It seems to work for the TCP stream I captured. Please test the patch with yours and report me if it works for you too.

Please find the patch there: http://pastebin.com/9MVeyVbZ

Regards,
Didier

This question was expired because it remained in the 'Needs information' state without activity for the last 15 days.

Hello Didier,

Im sorry, i was not able to work on ROHC during May...

Is it possible to send again the patch? He is no more available.
Thank you

Moreover, i try to use sniffer app. Im able to see in the terminal the numer of packets received, but im not able to see the statistics concerning the compression. I opened the syslog, and i just can see the number of context used... Is it normal?

Thank you Didier

Hello,

> Is it possible to send again the patch? He is no more available.

http://pastebin.com/6MPpKEfD

> Moreover, i try to use sniffer app. Im able to see in the terminal the
> numer of packets received, but im not able to see the statistics
> concerning the compression. I opened the syslog, and i just can see
> the number of context used... Is it normal?

The statistics are recorded in logs when you send SIGUSR1 to the process:
  # killall -USR1 rohc_sniffer

Then search in the logs several lines with stats info:

dump ROHC sniffer statistics...
general:
  total packets: 371000 packets
  bad packets: 0 packets (0%)
  loss (estim.):
    0 packets among 0 bursts (0%)
    packets per burst: max 0, avg 0, min 0
  mis-ordered packets (estim.): 0 packets (0%)
  duplicated packets (estim.): 0 packets (0%)
compression gain:
  pre-compress: 1296 MB
  post-compress: 316 MB
  compress ratio: 24% of total, ie. 76% of gain
  used and re-used contexts: 131658
packets per profile:
  Uncompressed profile: 9205 packets (2%)
  IP/UDP/RTP profile: 0 packets (0%)
  IP/UDP profile: 10026 packets (2%)
  IP-only profile: 2 packets (0%)
  IP/TCP profile: 351766 packets (94%)
  IP/UDP-Lite profile: 0 packets (0%)
packets per mode:
  U-mode: 370999 packets (100%)
  O-mode: 0 packets (0%)
  R-mode: 0 packets (0%)
packets per state:
  IR state: 167241 packets (45%)
  FO state: 877 packets (0%)
  SO state: 202881 packets (54%)
packets per packet type:
  packet type IR: 167241 packets (45%)
  packet type IR-DYN: 97 packets (0%)
  packet type UO-0: 168 packets (0%)
  packet type UO-1: 0 packets (0%)
  packet type RTP/UO-1-ID: 0 packets (0%)
  packet type RTP/UO-1-TS: 0 packets (0%)
  packet type RTP/UO-1: 0 packets (0%)
  packet type UOR-2: 41 packets (0%)
  packet type RTP/UOR-2: 0 packets (0%)
  packet type UOR-2-ID: 0 packets (0%)
  packet type UOR-2-TS: 0 packets (0%)
  packet type Uncomp/Normal: 0 packets (0%)
  packet type TCP/co_common: 12516 packets (3%)
  packet type TCP/rnd_1: 7 packets (0%)
  packet type TCP/rnd_2: 0 packets (0%)
  packet type TCP/rnd_3: 5 packets (0%)
  packet type TCP/rnd_4: 0 packets (0%)
  packet type TCP/rnd_5: 0 packets (0%)
  packet type TCP/rnd_6: 0 packets (0%)
  packet type TCP/rnd_7: 16 packets (0%)
  packet type TCP/rnd_8: 1 packets (0%)
  packet type TCP/seq_1: 39343 packets (10%)
  packet type TCP/seq_2: 0 packets (0%)
  packet type TCP/seq_3: 1346 packets (0%)
  packet type TCP/seq_4: 0 packets (0%)
  packet type TCP/seq_5: 126501 packets (34%)
  packet type TCP/seq_6: 0 packets (0%)
  packet type TCP/seq_7: 21024 packets (5%)
  packet type TCP/seq_8: 2693 packets (0%)
all ROHC sniffer statistics dumped

Regards,
Didier

Hello Didier,

Thank you for your answer.

I have a question concerning the ROHC context. Indeed, if, for example, i send 16 different (ip adress change) packets at the same time. The ROHC compressor will be able to create 16 different contexts. Hower, if i send 17 different packets, what's happened?

In my test, i try this:

send one packet type 1
send one packet type 2
send one packet type 3
send one packet type 4
send one packet type 5
send one packet type 6
send one packet type 7
send one packet type 8
send one packet type 9
send one packet type 10
send one packet type 11
send one packet type 12
send one packet type 13
send one packet type 14
send one packet type 15
send one packet type 16
send one packet type 1
send one packet type 2
send one packet type 3
send one packet type 4
send one packet type 5
send one packet type 6
...
Everything is OK.

But i don't understand what happened when i send 17 packets:
send one packet type 1
send one packet type 2
send one packet type 3
send one packet type 4
send one packet type 5
send one packet type 6
send one packet type 7
send one packet type 8
send one packet type 9
send one packet type 10
send one packet type 11
send one packet type 12
send one packet type 13
send one packet type 14
send one packet type 15
send one packet type 16
send one packet type 17
send one packet type 1
send one packet type 2
send one packet type 3
send one packet type 4
send one packet type 5
send one packet type 6
...
They are still a compression (not important as when i send 16 different packets)

Did the ROHC used LRU algorithm? If yes, why im still able to see a compression when i send 17 different packets?

Thank you

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Hello,

> They are still a compression (not important as when i send 16 different
> packets)
>
> Did the ROHC used LRU algorithm? If yes, why im still able to see a
> compression when i send 17 different packets?

Yes, the ROHC library uses a LRU algorithm. When all available contexts are already in use and a new context is needed, the context that was used the longest time ago is recycled. That's why the compression works but it is a little bit less efficient.

Regards,
Didier

Ok, thank you for the answer. Indeed, i am able to see the difference between the compression of 16 and 17 different packets.

I have another question concerning the optimization of the library. Indeed, the compression of IPv6 use less CPU charge than the compression of IPv4. The header of IPv6 packet (40 bytes) is longer than the header of IPv4 packet (20 bytes) and i send the same size of packets at the same speed. Is it normal? Is ROHC library more optimized for IPv6 packets than IPv4 packets?

Thank you

Hello,

> I have another question concerning the optimization of the library.
> Indeed, the compression of IPv6 use less CPU charge than the
> compression of IPv4. The header of IPv6 packet (40 bytes) is longer
> than the header of IPv4 packet (20 bytes) and i send the same size
> of packets at the same speed. Is it normal? Is ROHC library more
> optimized for IPv6 packets than IPv4 packets?

I don't think the ROHC library is more optimized for IPv6 streams than for IPv4 streams. I mean, there was no specific optimization work performed for IPv4 streams that was not done for IPv6. However, I see at least 2 reasons for the speed difference:
1/ The IPv4 header includes a checksum that the compressor verifies and that the decompressor computes when building the decompressed IPv4 header.
2/ The IPv4 header includes an identification field called IP-ID that requires additional code and processing time. IPv6 does not include such a field.

If IPv4 performances are problematic, I see nothing to do for 2/. However 1/ could be optimized by computing partial checksums only on the fields that changed between packets.

What is the difference of performances between v4 and v6?

Regards,
Didier

Patch for context periodic refreshes of the RTP profile merged in branches 1.7.x and master.
Patch for context periodic refreshes of the TCP profile merged in branch master only (no TCP bugfixes on 1.7.x).