Synchronous Development in Open-Source Projects: A Higher-Level Perspective

Automated Software Engineering – Supplementary Website

Thomas Bock, Claus Hunsen, Mitchell Joblin, and Sven Apel

Saarland University University of Passau Siemens

Complete Material

Downloads

Website Content

Research Questions and Hypotheses

Research Questions

RQ1
Which abstraction level of the source code captures the collaboration of developers best: files or features?
That is, which of the two abstraction levels of the source code leads to identifying a stronger statistical dependence between technical activities of developer pairs?
RQ2
Which abstraction level of the mailing list captures the coordination of developers best: message-based communication or conversation-based communication?
That is, which of the two abstraction levels of the mailing list leads to identifying a stronger statistical dependence between technical activities and social activities on the mailing list?

Hypotheses

  • Hypotheses Related to C-Bursts
    H1
    Feature-based collaboration captures developer collaboration more accurately than file-based collaboration.
    H1.1
    The number of empirical C-bursts is higher than the number of simulated C-bursts.
    H1.2
    The synchronicity degree of empirical C-bursts is higher than the one of simulated C-bursts.
    H1.3
    The code growth ∆L is higher in synchronous commits than in non-synchronous commits.
    H1.4
    The implementation effort ∆W is lower in synchronous commits than in non-synchronous commits.
    To evaluate H1, we evaluate the following four sub-hypotheses:
    H1+H1.1
    The number of empirical feature-based C-bursts is higher than the number of empirical file-based C-bursts.
    H1+H1.2
    The synchronicity degree of empirical feature-based C-bursts is higher than the synchronicity degree of empirical file-based C-bursts.
    H1+H1.3
    The code growth ∆L in synchronous commits is higher on features than on files.
    H1+H1.4
    The implementation effort ∆W in synchronous commits is lower on features than on files.
  • Hypotheses Related to E-Bursts
    H2
    Conversation-based communication captures developer coordination more accurately than message-based communication.
    H2.1
    The relation between the number of C-bursts and the number of E-bursts is described by a linear relationship.
    To evaluate H2, we evaluate the following sub-hypothesis:
    H2+H2.1
    The linear relation between the number of C-bursts and the number of E-bursts has a higher goodness of fit for conversation-based E-bursts than for message-based E-bursts.
  • Hypotheses Related to C-Bursts and E-Bursts
    H3
    The temporal correlation between C-bursts and E-bursts is higher for feature-based C-bursts than for file-based C-bursts and higher for conversation-based E-bursts than for message-based E-bursts.
    H3.1
    C-bursts and E-bursts are temporally correlated, that is, the DTW distances between empirical C-curves and empirical E-curves are smaller than between simulated curves.
    To evaluate H3, we evaluate the following sub-hypothesis:
    H3+H3.1
    C-bursts and E-bursts are temporally correlated, that is, the DTW distances between empirical C-curves and empirical E-curves are smaller than between simulated curves.

Statistical Tests for Main Hypotheses

Content

Hypothesis H1

(Jump to comparison regarding H1.1 | H1.2 | H1.3 | H1.4)
Hypothesis H1+H1.1. Comparing abstraction levels regarding H1.1. A p-value < 0.05 means that the empirical numbers of C-bursts per developer pair of the feature-based analysis are significantly higher than the corresponding empirical numbers of C-bursts per developer pair of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature > file)
feature > file
ξ U p-value Cliff's
Delta
BusyBox
1 135485888 0.2028024 0.0006
5 135266090 0.0095987932 0.0023
10 135194592 0.0036622953 0.0028
15 135199201 0.006504682 0.0028
OpenSSL
1 35673033 7.246036e-14 0.0126
5 35367796 3.616927e-18 0.0211
10 35122324 3.910268e-24 0.0279
15 34969476 1.707343e-27 0.0321
QEMU
1 25891734214 1.000092e-48 0.0028
5 25746439556 2.888522e-183 0.0084
10 25612361104 0 0.0136
15 25515286716 0 0.0173

Synchronicity-degree comparison for BusyBox
Synchronicity-degree comparison for BusyBox
Synchronicity-degree comparison for OpenSSL
Synchronicity-degree comparison for OpenSSL
Synchronicity-degree comparison for QEMU
Synchronicity-degree comparison for QEMU
Hypothesis H1+H1.2. Comparing abstraction levels regarding H1.2. A p-value < 0.05 means that the empirical synchronicity degrees of the feature-based analysis are significantly higher than the empirical synchronicity degrees of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature > file)
feature > file
ξ U p-value Cliff's
Delta
BusyBox
1 15544.5 0.6187063 -0.0200
5 45281.5 0.3691557 0.0164
10 57475 0.4313845 0.0080
15 62482 0.5603974 0.0488
OpenSSL
1 47968.5 0.0001996878 0.1622
5 202911 9.488344e-12 0.2073
10 298943.5 2.572438e-11 0.1837
15 332521 1.326538e-12 0.1902
QEMU
1 681839.5 3.757765e-40 0.2984
5 3949898 4.786402e-210 0.4257
10 8290824 0 0.4864
15 12049580 0 0.5135
Hypothesis H1+H1.3. Comparing abstraction levels regarding H1.3. A p-value < 0.05 means that the code growth ΔL in the synchronous commits of the feature-based analysis is significantly higher than the code growth ΔL in the synchronous commits of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature > file)
feature > file
ξ U p-value Cliff's
Delta
BusyBox
1 114081 0.1967877 0.0608
5 689731.5 0.02884461 0.0589
10 1350738 0.0001153576 0.0836
15 1890074 1.298514e-05 0.0871
OpenSSL
1 310536 0.9539756 -0.0029
5 1658454 0.9885068 -0.0112
10 3070373 0.989393 -0.0026
15 4241030 0.9916779 0.0053
QEMU
1 7273482 1 -0.0366
5 29462292 1 -0.0089
10 48855197 1 0.0036
15 61288970 0.3023295 0.0110
Hypothesis H1+H1.4. Comparing abstraction levels regarding H1.4. A p-value < 0.05 means that the code effort ΔW in the synchronous commits of the feature-based analysis is significantly lower than the code effort ΔW in the synchronous commits of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature < file)
feature < file
ξ U p-value Cliff's
Delta
BusyBox
1 123691 0.903862 -0.0183
5 714858.5 0.4960634 0.0246
10 1393609 0.0154136 0.0545
15 1969254 0.01952964 0.0488
OpenSSL
1 320524 0.98942 -0.0352
5 1699802 0.9999997 -0.0364
10 3186181 1 -0.0405
15 4384574 1 -0.0284
QEMU
1 7013639 1 0.0004
5 28321505 1 0.0301
10 46651924 1.161575e-07 0.0485
15 58353021 1 0.0584

Hypothesis 2

(Jump to data regarding BusyBox | OpenSSL | QEMU)

BusyBox

Message-based • lower-bound approach
Conversation-based • lower-bound approach
Message-based • upper-bound approach
Conversation-based • upper-bound approach

OpenSSL

Message-based • lower-bound approach
Conversation-based • lower-bound approach
Message-based • upper-bound approach
Conversation-based • upper-bound approach

QEMU

Message-based • lower-bound approach
Conversation-based • lower-bound approach
Message-based • upper-bound approach
Conversation-based • upper-bound approach

Hypothesis H3

(Jump to results of H3 using smoothing parameter and band-window size 1ξ | 2ξ | 3ξ)

Smoothing parameter and band-window size: 1ξ

(Jump to comparison regarding artifact abstraction | mail abstraction)
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 1ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the feature-based analysis are significantly smaller than the corresponding empirical DTW distances of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature < file)
feature < file
ξ mailing-list abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 message 29.5 0.1907301 -0.4048
5 message 139.5 0.1811315 -0.3413
10 message 266.5 0.1820859 -0.2690
15 message 346 0.339109 -0.1829
1 conversation 17 0.7147429 0.1905
5 conversation 141 0.2041459 -0.3558
10 conversation 209 0.3007771 -0.2667
15 conversation 287.5 0.5288958 -0.1735
OpenSSL • lower-bound approach
1 message -- -- --
5 message 0 0.9772499 1.0000
10 message 5 0.8067619 0.1667
15 message 8 0.5 -0.0667
1 conversation -- -- --
5 conversation -- -- --
10 conversation -- -- --
15 conversation -- -- --
QEMU • lower-bound approach
1 message 114530.5 0.03992912 -0.0697
5 message 1036377.0 3.509142e-12 -0.1572
10 message 2513288 2.837048e-07 -0.0901
15 message 4050886 4.012332e-08 -0.0856
1 conversation 21600.0 0.155811 -0.0634
5 conversation 90444.5 0.05896802 -0.0681
10 conversation 189474.0 0.1613079 -0.0352
15 conversation 273238.5 0.1803632 -0.0296
BusyBox • upper-bound approach
1 message 1762.5 0.2869297 -0.0633
5 message 3149.5 0.912163 0.1229
10 message 4326.5 0.9658731 0.1517
15 message 5293.5 0.97921 0.1608
1 conversation 1160.5 0.6973924 0.0622
5 conversation 2415 0.9615439 0.1692
10 conversation 2930.5 0.9852391 0.1967
15 conversation 3547.5 0.9883342 0.1956
OpenSSL • upper-bound approach
1 message 2372.5 0.3346254 -0.0429
5 message 9334 0.9078902 0.0911
10 message 14814.5 0.8931269 0.0762
15 message 20733 0.9114269 0.0761
1 conversation 966.5 0.3451975 -0.0505
5 conversation 2662.5 0.7336254 0.0589
10 conversation 3956.5 0.8085630 0.0744
15 conversation 5465.5 0.6788902 0.0368
QEMU • upper-bound approach
1 message 360931.0 0.46343033 -0.0027
5 message 2117556 3.646642e-01 -0.0063
10 message 4843092 9.326571e-01 0.0221
15 message 7656084 9.376598e-01 0.0202
1 conversation 196781.0 0.26515260 -0.0215
5 conversation 883089.0 0.935644957 0.0348
10 conversation 1731948 0.99999715 0.0862
15 conversation 2424468 1.000000 0.1040
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 1ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the conversation-based analysis are significantly smaller than the corresponding empirical DTW distances of the message-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: conversation < message)
conversation < message
ξ artifact abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 file 102.5 0.5910117 -0.0459
5 file 334 0.4744592 0.0118
10 file 527 0.5131239 -0.0038
15 file 696 0.5602421 -0.0198
1 feature 6.5 0.8658571 -0.4444
5 feature 31 0.4790301 0.0312
10 feature 72 0.655512 -0.0909
15 feature 108.5 0.5693267 -0.0333
OpenSSL • lower-bound approach
1 file -- -- --
5 file 1.5 0.9213504 -0.5000
10 file 2 0.6381632 0.0000
15 file 3 0.7209077 -0.2000
1 feature -- -- --
5 feature -- -- --
10 feature -- -- --
15 feature -- -- --
QEMU • lower-bound approach
1 file 168075 1 -0.2738
5 file 916742.5 1 -0.3503
10 file 2064022 1 -0.3727
15 file 3145634 1 -0.3829
1 feature 21100 0.9999935 -0.2801
5 feature 162799.5 1 -0.4574
10 feature 407885 1 -0.4534
15 feature 637096 1 -0.4635
BusyBox • upper-bound approach
1 file 3100.5 0.3837009 0.0273
5 file 4784 0.01238102 0.1772
10 file 6646.5 0.02616465 0.1426
15 file 8053.5 0.01127874 0.1590
1 feature 722 0.8140288 -0.1220
5 feature 1496 0.05805552 0.1668
10 feature 2116 0.115457 0.1183
15 feature 2556 0.1009288 0.1201
OpenSSL • upper-bound approach
1 file 1416 0.1373853 0.1205
5 file 5844 0.0545929 0.1255
10 file 8401 0.002500195 0.1964
15 file 11141 0.0007705685 0.2065
1 feature 1145 0.1539802 0.1192
5 feature 3498 0.01217159 0.1953
10 feature 5140 0.002939712 0.2161
15 feature 6770 0.0002303006 0.2535
QEMU • upper-bound approach
1 file 525906 0.0001413022 0.0907
5 file 1532476 5.566383e-120 0.4019
10 file 3059821 3.80131e-199 0.4328
15 file 4468298 1.332894e-250 0.4415
1 feature 107555 0.002878516 0.1029
5 feature 494158 1.329499e-47 0.3422
10 feature 1127558 1.01005e-74 0.3520
15 feature 1749509 5.353178e-84 0.3380

Smoothing parameter and band-window size: 2ξ

(Jump to comparison regarding artifact abstraction | mail abstraction)
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 2ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the feature-based analysis are significantly smaller than the corresponding empirical DTW distances of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature < file)
feature < file
ξ mailing-list abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 message 31.5 0.2344606 -0.50000000
5 message 143 0.123247 -0.37500000
10 message 262.5 0.15337 -0.25000000
15 message 339.0 0.2807124 -0.15897436
1 conversation 31 0.2586764 -0.47619048
5 conversation 141 0.1599226 -0.35576923
10 conversation 218.5 0.1701726 -0.32424242
15 conversation 302.0 0.3166841 -0.23265306
OpenSSL • lower-bound approach
1 message -- -- --
5 message 0 0.9772499 1
10 message 4 0.8116204 0.3333333
15 message 7 0.6172028 0.06666667
1 conversation -- -- --
5 conversation -- -- --
10 conversation -- -- --
15 conversation -- -- --
QEMU • lower-bound approach
1 message 126005.5 4.470506e-06 -0.1768681
5 message 980110.0 1.921303e-05 -0.09437354
10 message 2418520 0.003257534 -0.04902169
15 message 3864428 0.01271414 -0.03565929
1 conversation 21549.5 0.1656193 -0.06087235
5 conversation 89128.5 0.1138065 -0.05253307
10 conversation 185452.5 0.3550445 -0.01323277
15 conversation 266484.0 0.4489555 -0.00415818
BusyBox • upper-bound approach
1 message 1674 0.4656921 -0.009954751
5 message 3125.0 0.9235347 0.12976887
10 message 4318.0 0.9673609 0.15333333
15 message 5306.5 0.9778691 0.15876665
1 conversation 1209 0.576684 0.023030303
5 conversation 2402.5 0.965145 0.17354661
10 conversation 2915 0.9869064 0.200932018
15 conversation 3579.5 0.9855139 0.188321995
OpenSSL • upper-bound approach
1 message 2277 0.4973648 -0.0008791209
5 message 9292 0.9173539 0.09514071
10 message 14704 0.9123975 0.08312028
15 message 20543.5 0.9331116 0.08451426
1 conversation 920.0 0.5017270 2.902951e-18
5 conversation 2663.5 0.7323969 0.058501237
10 conversation 3945.5 0.8166709 0.076968066
15 conversation 5267.0 0.8172146 0.07181249
QEMU • upper-bound approach
1 message 370583.0 0.15409065 -0.02946871
5 message 2103508 5.071624e-01 0.0003291037
10 message 4815039 9.699130e-01 0.02775097
15 message 7600899 9.808686e-01 0.02727014
1 conversation 197184.5 0.245545342 -0.02362252
5 conversation 876425 0.966939181 0.04203912
10 conversation 1709122 0.9999999 0.09822544
15 conversation 2384348 1.0000000 0.118875992
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 2ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the conversation-based analysis are significantly smaller than the corresponding empirical DTW distances of the message-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: conversation < message)
conversation < message
ξ artifact abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 file 94.5 0.4451449 0.0357
5 file 335 0.4817489 0.0089
10 file 549.5 0.6289225 -0.0467
15 file 715.5 0.6415967 -0.0484
1 feature 3.5 0.412389 0.2222
5 feature 32 0.5209699 0.0000
10 feature 61 0.3908018 0.0758
15 feature 109.5 0.5864227 -0.0429
OpenSSL • lower-bound approach
1 file -- -- --
5 file 1.5 0.9213504 -0.5000
10 file 2 0.6381632 0.0000
15 file 3 0.7209077 -0.2000
1 feature -- -- --
5 feature -- -- --
10 feature -- -- --
15 feature -- -- --
QEMU • lower-bound approach
1 file 176881 1 -0.3406
5 file 938406.5 1 -0.3822
10 file 2074267 1 -0.3795
15 file 3166391 1 -0.3920
1 feature 23460.5 1 -0.4233
5 feature 160108.5 1 -0.4333
10 feature 408942 1 -0.4572
15 feature 630181 1 -0.4476
BusyBox • upper-bound approach
1 file 2846.5 0.1221561 0.1070
5 file 4960.5 0.03142918 0.1468
10 file 6873.5 0.06156174 0.1133
15 file 8514 0.05583297 0.1109
1 feature 582 0.2452866 0.0956
5 feature 1575 0.1238102 0.1228
10 feature 2227 0.2330175 0.0721
15 feature 2712 0.2404133 0.0664
OpenSSL • upper-bound approach
1 file 1313.5 0.04740422 0.1842
5 file 6017 0.1017992 0.0997
10 file 8839 0.01361712 0.1545
15 file 11710 0.00546155 0.1660
1 feature 1060 0.05701201 0.1846
5 feature 3575 0.02031265 0.1776
10 feature 5414 0.01315417 0.1743
15 feature 7339 0.004197322 0.1908
QEMU • upper-bound approach
1 file 399017 1.153492e-35 0.3101
5 file 1573106 7.068196e-111 0.3861
10 file 3239574 5.150017e-170 0.3995
15 file 4809009 4.894826e-205 0.3989
1 feature 85737.5 1.049393e-14 0.2849
5 feature 506895.5 3.343159e-43 0.3253
10 feature 1205634 2.149876e-57 0.3071
15 feature 1894614 1.435514e-59 0.2831

Smoothing parameter and band-window size: 3ξ

(Jump to comparison regarding artifact abstraction | mail abstraction)
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 3ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the feature-based analysis are significantly smaller than the corresponding empirical DTW distances of the file-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: feature < file)
feature < file
ξ mailing-list abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 message 30.5 0.2344606 -0.4524
5 message 149 0.1023692 -0.4327
10 message 266 0.1317595 -0.2667
15 message 342.5 0.2543151 -0.1709
1 conversation 31.5 0.1890857 -0.5000
5 conversation 143 0.1469495 -0.3750
10 conversation 227 0.1054986 -0.3758
15 conversation 277.0 0.4089362 -0.1306
OpenSSL • lower-bound approach
1 message -- -- --
5 message 0 0.9772499 1.0000
10 message 4 0.8116204 0.3333
15 message 7 0.6172028 0.0667
1 conversation -- -- --
5 conversation -- -- --
10 conversation -- -- --
15 conversation -- -- --
QEMU • lower-bound approach
1 message 127734.5 6.275401e-07 -0.1930
5 message 964266.0 0.0004111805 -0.0767
10 message 2394268 0.01630389 -0.0385
15 message 3899014 0.002433373 -0.0449
1 conversation 21481.5 0.1792627 -0.0575
5 conversation 87867.5 0.1936604 -0.0376
10 conversation 183296.0 0.4837749 -0.0015
15 conversation 265513.5 0.4938516 -0.0005
BusyBox • upper-bound approach
1 message 1662.0 0.4914131 -0.0027
5 message 3099 0.9343399 0.1370
10 message 4238 0.9789136 0.1690
15 message 5314.5 0.977008 0.1575
1 conversation 1199.5 0.6013256 0.0307
5 conversation 2386.5 0.9693453 0.1791
10 conversation 2965.0 0.980861 0.1872
15 conversation 3557.5 0.9875089 0.1933
OpenSSL • upper-bound approach
1 message 2271 0.5079052 0.0018
5 message 9205.0 0.9346033 0.1036
10 message 14523 0.9381803 0.0944
15 message 20332.5 0.9521578 0.0939
1 conversation 908.0 0.5430922 0.0130
5 conversation 2649.5 0.7493272 0.0634
10 conversation 3858.5 0.8730878 0.0973
15 conversation 5278.0 0.8106805 0.0699
QEMU • upper-bound approach
1 message 373791.5 0.092207855 0.0922
5 message 2096468 5.793862e-01 0.0037
10 message 4801986 9.802064e-01 0.0304
15 message 7571306 9.908571e-01 0.0311
1 conversation 196211.5 0.294141462 -0.0186
5 conversation 872148.5 0.979420225 0.0467
10 conversation 1696236 0.99999998 0.1050
15 conversation 2362834 1.000000 0.1268
Hypothesis H3+H3.1 (Smoothing parameter and band-window size: 3ξ). Testing H3.1 for different abstraction levels. A p-value < 0.05 means that the empirical DTW distances of the conversation-based analysis are significantly smaller than the corresponding empirical DTW distances of the message-based analysis. (One-tailed Mann-Whitney U test with FDR correction, alternative hypothesis: conversation < message)
conversation < message
ξ artifact abstraction U p-value Cliff's
Delta
BusyBox • lower-bound approach
1 file 99.5 0.5366444 -0.0153
5 file 335.5 0.4853985 0.0074
10 file 557 0.6655742 -0.0610
15 file 710.5 0.6211794 -0.0410
1 feature 2.5 0.2532776 0.4444
5 feature 35 0.6436064 -0.0938
10 feature 70.5 0.6210005 -0.0682
15 feature 127 0.8370049 -0.2095
OpenSSL • lower-bound approach
1 file -- -- --
5 file 1 0.7298543 0.0000
10 file 2 0.6381632 0.0000
15 file 4 0.8792167 -0.6000
1 feature -- -- --
5 feature -- -- --
10 feature -- -- --
15 feature -- -- --
QEMU • lower-bound approach
1 file 172707.5 1 -0.3090
5 file 944118.5 1 -0.3907
10 file 2090894 1 -0.3905
15 file 3184026 1 -0.3998
1 feature 22989 1 -0.3947
5 feature 162067.5 1 -0.4508
10 feature 411195.5 1 -0.4652
15 feature 632020.5 1 -0.4518
BusyBox • upper-bound approach
1 file 2801.5 0.09372704 0.1211
5 file 4973.5 0.03348719 0.1446
10 file 6990.5 0.09071909 0.0982
15 file 8710 0.09731117 0.0904
1 feature 563.5 0.1844637 0.1243
5 feature 1579 0.1281613 0.1206
10 feature 2201 0.2007864 0.0829
15 feature 2756 0.2932769 0.0513
OpenSSL • upper-bound approach
1 file 1315 0.04824589 0.1832
5 file 6074 0.1225216 0.0911
10 file 8966 0.02095059 0.1424
15 file 11868 0.008833537 0.1547
1 feature 1078 0.07192717 0.1708
5 feature 3623.5 0.02751554 0.1664
10 feature 5535 0.02349773 0.1559
15 feature 7417 0.005919076 0.1822
QEMU • upper-bound approach
1 file 364234.5 5.724665e-50 0.3703
5 file 1574941 1.785894e-110 0.3854
10 file 3268462 1.501843e-165 0.3941
15 file 4902068 2.157015e-193 0.3873
1 feature 80305 3.989165e-19 0.3302
5 feature 508927.5 1.605788e-42 0.3226
10 feature 1227856 6.853704e-53 0.2943
15 feature 1944635 3.833429e-52 0.2642

Statistical Tests for Sub-Hypotheses

Hypothesis H1.1

(Jump to data for BusyBox | OpenSSL | QEMU,
BusyBox (only pairs having a C-burst) | OpenSSL (only pairs having a C-burst) | QEMU (only pairs having a C-burst))
Hypothesis H1.1. Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for BusyBox. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
BusyBox
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 0.01 ± 0.20 0.00 ± 0.06 1321 8.597e-10 -0.0594
5 0.02 ± 0.38 0.01 ± 0.18 2927 3.166e-10 -0.0609
10 0.02 ± 0.39 0.01 ± 0.24 4511 6.111e-08 -0.0521
15 0.02 ± 0.40 0.01 ± 0.28 5780.5 3.24e-07 -0.0491
file
1 0.01 ± 0.34 0.00 ± 0.13 6213 < 2.2e-16 -0.0531
5 0.02 ± 0.48 0.01 ± 0.33 9034 2.656e-16 -0.0500
10 0.02 ± 0.47 0.01 ± 0.36 11769 1.728e-13 -0.0449
15 0.02 ± 0.40 0.01 ± 0.35 13247 4.773e-10 -0.0377
Hypothesis H1.1. Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for OpenSSL. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
OpenSSL
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 0.06 ± 0.72 0.02 ± 0.30 7216 1.353e-07 -0.0716
5 0.13 ± 1.33 0.07 ± 0.88 25084 2.183e-07 -0.0704
10 0.15 ± 1.54 0.11 ± 1.18 45655 0.0002217 -0.0489
15 0.16 ± 1.52 0.14 ± 1.29 61138 0.001974 -0.0401
file
1 0.03 ± 0.57 0.01 ± 0.20 4983 2.614e-16 -0.0685
5 0.06 ± 0.96 0.03 ± 0.58 21636 < 2.2e-16 -0.0903
10 0.07 ± 1.05 0.05 ± 0.78 24502 4.008e-15 -0.0656
15 0.07 ± 1.03 0.06 ± 0.86 26400 3.943e-11 -0.0549
Hypothesis H1.1. Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for QEMU. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
QEMU
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 0.01 ± 0.16 0.00 ± 0.14 257870 < 2.2e-16 -0.0395
5 0.02 ± 0.30 0.02 ± 0.30 1908400 < 2.2e-16 -0.0380
10 0.04 ± 0.34 0.03 ± 0.35 4973200 < 2.2e-16 -0.0360
15 0.04 ± 0.37 0.04 ± 0.38 8208900 < 2.2e-16 -0.0324
file
1 0.00 ± 0.12 0.00 ± 0.08 706200 2.2e-16 -0.0405
5 0.01 ± 0.24 0.01 ± 0.21 2948700 < 2.2e-16 -0.0448
10 0.02 ± 0.31 0.01 ± 0.28 5925700 < 2.2e-16 -0.0470
15 0.02 ± 0.34 0.02 ± 0.31 8491000 < 2.2e-16 -0.0475
Hypothesis H1.1 (only pairs having, at least, one C-burst). Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for BusyBox. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
BusyBox
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 1.96 ± 2.05 0.26 ± 0.80 1225 1.058e-10 -0.899
5 2.48 ± 3.59 0.74 ± 1.81 2927 1.094e-12 -0.785
10 2.34 ± 3.30 0.98 ± 2.27 3572 1.335e-12 -0.714
15 2.30 ± 3.20 1.05 ± 2.48 4347.5 6.879e-14 -0.722
file
1 2.80 ± 4.51 0.61 ± 1.92 5974 < 2.2e-16 -0.857
5 3.21 ± 5.65 1.54 ± 4.09 8276 < 2.2e-16 -0.758
10 2.99 ± 4.95 1.71 ± 4.09 9845.5 < 2.2e-16 -0.672
15 2.63 ± 3.89 1.64 ± 3.64 9721.5 < 2.2e-16 -0.599
Hypothesis H1.1 (only pairs having, at least, one C-burst). Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for OpenSSL. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
OpenSSL
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 2.95 ± 4.12 0.21 ± 0.78 5269 < 2.2e-16 -0.867
5 3.27 ± 6.01 0.58 ± 1.94 16974 < 2.2e-16 -0.812
10 3.05 ± 6.22 0.76 ± 2.59 26853 < 2.2e-16 -0.780
15 2.78 ± 5.71 0.83 ± 2.82 35681 < 2.2e-16 -0.766
file
1 3.52 ± 5.60 0.93 ± 2.04 4465 < 2.2e-16 -0.841
5 3.30 ± 6.58 1.62 ± 4.13 19662 < 2.2e-16 -0.767
10 3.02 ± 6.38 1.93 ± 4.86 19694 < 2.2e-16 -0.614
15 2.77 ± 5.86 1.98 ± 4.97 20038 < 2.2e-16 -0.522
Hypothesis H1.1 (only pairs having, at least, one C-burst). Paired, one-tailed Wilcoxon signed rank test for comparing empirical and simulated number of C-bursts per developer pair for QEMU. A p-value < 0.05 means that the number of empirical C-bursts per developer pair is significantly higher than the corresponding number of simulated C-bursts per developer pair.
QEMU
ξ mean ± std. dev. of
the empirical number of
C-bursts per developer pair		 mean ± std. dev. of
the simulated number of
C-bursts per developer pair		 V p-value r
feature
1 1.46 ± 1.59 0.27 ± 1.49 204900 < 2.2e-16 -0.888
5 1.58 ± 1.88 0.25 ± 1.68 1302800 < 2.2e-16 -0.857
10 1.56 ± 1.67 0.19 ± 1.24 3165600 < 2.2e-16 -0.850
15 1.53 ± 1.60 0.18 ± 1.05 4968700 < 2.2e-16 -0.852
file
1 1.58 ± 1.51 0.47 ± 1.35 660460 < 2.2e-16 -0.822
5 1.86 ± 2.35 0.99 ± 2.49 2533500 < 2.2e-16 -0.705
10 1.90 ± 2.56 1.13 ± 2.65 4927400 < 2.2e-16 -0.659
15 1.89 ± 2.56 1.17 ± 2.61 6903100 < 2.2e-16 -0.642

Hypothesis H1.2

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H1.2. One-tailed Mann-Whitney U test for comparing empirical and simulated synchronicity degrees for BusyBox. A p-value < 0.05 means that the empirical synchronicity degrees are significantly higher than the simulated synchronicity degrees.
BusyBox
ξ mean ± std. dev. of
empirical degsync		 mean ± std. dev. of
simulated degsync		 U p-value Cliff's
Delta
feature
1 0.60 ± 0.32 0.31 ± 0.26 199030 < 2.2e-16 0.5128
5 0.47 ± 0.33 0.26 ± 0.24 1267300 < 2.2e-16 0.3854
10 0.42 ± 0.31 0.24 ± 0.23 2182500 < 2.2e-16 0.3721
15 0.39 ± 0.31 0.23 ± 0.23 2776200 < 2.2e-16 0.3387
file
1 0.61 ± 0.35 0.32 ± 0.30 2416800 < 2.2e-16 0.4502
5 0.47 ± 0.35 0.25 ± 0.25 9497500 < 2.2e-16 0.3873
10 0.43 ± 0.34 0.23 ± 0.24 13510000 < 2.2e-16 0.3570
15 0.41 ± 0.34 0.22 ± 0.24 14568000 < 2.2e-16 0.3265
Hypothesis H1.2. One-tailed Mann-Whitney U test for comparing empirical and simulated synchronicity degrees for OpenSSL. A p-value < 0.05 means that the empirical synchronicity degrees are significantly higher than the simulated synchronicity degrees.
OpenSSL
ξ mean ± std. dev. of
empirical degsync		 mean ± std. dev. of
simulated degsync		 U p-value Cliff's
Delta
feature
1 0.51 ± 0.33 0.39 ± 0.29 2635200 3.578e-09 0.1927
5 0.43 ± 0.29 0.35 ± 0.27 17328000 2.19e-14 0.1729
10 0.38 ± 0.27 0.32 ± 0.25 29429000 3.81e-12 0.1419
15 0.37 ± 0.27 0.31 ± 0.25 36351000 1.819e-13 0.1467
file
1 0.42 ± 0.34 0.30 ± 0.28 3616500 2.146e-12 0.2095
5 0.34 ± 0.30 0.25 ± 0.24 24001000 2.2e-16 0.1797
10 0.32 ± 0.29 0.23 ± 0.23 40996000 < 2.2e-16 0.1829
15 0.30 ± 0.28 0.22 ± 0.22 51711000 < 2.2e-16 0.1833
Hypothesis H1.2. One-tailed Mann-Whitney U test for comparing empirical and simulated synchronicity degrees for QEMU. A p-value < 0.05 means that the empirical synchronicity degrees are significantly higher than the simulated synchronicity degrees.
QEMU
ξ mean ± std. dev. of
empirical degsync		 mean ± std. dev. of
simulated degsync		 U p-value Cliff's
Delta
feature
1 0.64 ± 0.31 0.60 ± 0.31 47697000 2.255e-05 0.0763
5 0.61 ± 0.31 0.58 ± 0.31 418200000 9.187e-06 0.0481
10 0.60 ± 0.31 0.57 ± 0.31 981100000 1.061e-09 0.0542
15 0.59 ± 0.31 0.57 ± 0.31 1509200000 2.926e-09 0.0473
file
1 0.47 ± 0.35 0.34 ± 0.31 169020000 < 2.2e-16 0.2071
5 0.37 ± 0.33 0.30 ± 0.28 1403200000 < 2.2e-16 0.1232
10 0.33 ± 0.31 0.27 ± 0.27 3296700000 < 2.2e-16 0.1098
15 0.31 ± 0.30 0.25 ± 0.26 5078600000 < 2.2e-16 0.0969

Hypothesis H1.3

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H1.3. One-tailed Mann-Whitney U test for comparing code growth ΔL of synchronous and non-synchronous commits for BusyBox. A p-value < 0.05 means that the code growth ΔL in synchronous commits is significantly higher than the code growth ΔL in non-synchronous commits.
BusyBox
ξ ΔL (mean ± std. dev.) in
synchronous commits		 ΔL (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 40.71 ± 217.16 72.90 ± 866.91 621380 0.1606 0.0366
5 152.54 ± 1,436.99 57.94 ± 702.23 1515300 0.1113 0.0287
10 143.84 ± 1,297.45 53.26 ± 688.30 2027500 0.01392 0.0452
15 136.44 ± 1,228.08 51.96 ± 692.07 2251200 0.006921 0.0480
file
1 34.28 ± 210.01 47.88 ± 683.84 4371200 0.3396 0.0081
5 47.07 ± 546.76 46.47 ± 679.66 8416800 0.5163 -0.0006
10 45.97 ± 497.10 46.87 ± 710.35 10410000 0.9621 -0.0222
15 54.17 ± 688.27 42.26 ± 633.61 11448000 0.9911 -0.0282
Hypothesis H1.3. One-tailed Mann-Whitney U test for comparing code growth ΔL of synchronous and non-synchronous commits for OpenSSL. A p-value < 0.05 means that the code growth ΔL in synchronous commits is significantly higher than the code growth ΔL in non-synchronous commits.
OpenSSL
ξ ΔL (mean ± std. dev.) in
synchronous commits		 ΔL (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 52.89 ± 331.81 83.37 ± 1,357.07 1148900 0.9497 -0.0395
5 69.66 ± 408.72 83.27 ± 1,525.61 1968200 0.9165 -0.0256
10 65.75 ± 400.89 90.82 ± 1,710.95 2164700 0.859 -0.0190
15 112.25 ± 1,638.92 32.05 ± 129.24 2146900 0.5877 -0.0040
file
1 68.61 ± 578.95 64.72 ± 1,258.09 3095900 0.9043 -0.0263
5 57.87 ± 418.90 68.00 ± 1,388.58 6277300 0.4181 0.0030
10 111.12 ± 1,941.32 37.67 ± 203.15 7286000 0.484 0.0005
15 98.60 ± 1,781.98 38.37 ± 209.83 7679800 0.5056 -0.0002
Hypothesis H1.3. One-tailed Mann-Whitney U test for comparing code growth ΔL of synchronous and non-synchronous commits for QEMU. A p-value < 0.05 means that the code growth ΔL in synchronous commits is significantly higher than the code growth ΔL in non-synchronous commits.
QEMU
ξ ΔL (mean ± std. dev.) in
synchronous commits		 ΔL (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 53.36 ± 310.37 78.06 ± 366.91 6548900 1 -0.0615
5 47.65 ± 269.63 96.74 ± 422.17 7994600 1 -0.1042
10 48.25 ± 262.21 108.91 ± 460.59 7356300 1 -0.1287
15 50.72 ± 267.30 111.97 ± 474.95 6895300 1 -0.1388
file
1 42.65 ± 732.06 39.24 ± 263.36 96963000 1.381e-13 0.0586
5 34.99 ± 515.49 42.77 ± 285.68 152310000 0.0001606 0.0226
10 34.23 ± 448.14 46.10 ± 308.76 158750000 0.2157 0.0048
15 32.98 ± 415.29 51.16 ± 339.91 147620000 0.9134 -0.0086

Hypothesis H1.4

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H1.4. One-tailed Mann-Whitney U test for comparing code effort ΔW of synchronous and non-synchronous commits for BusyBox. A p-value < 0.05 means that the code effort ΔW in synchronous commits is significantly lower than the code effort ΔW in non-synchronous commits.
BusyBox
ξ ΔW (mean ± std. dev.) in
synchronous commits		 ΔW (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 110.19 ± 256.55 189.86 ± 1,097.23 582280 0.2201 -0.0286
5 253.02 ± 1,470.09 174.73 ± 989.08 1496200 0.7474 0.0157
10 263.91 ± 1,403.50 166.38 ± 969.15 2024200 0.9826 0.0435
15 259.48 ± 1,335.70 163.93 ± 976.87 2215300 0.9452 0.0313
file
1 127.37 ± 362.98 140.35 ± 982.85 4655300 0.9999 0.0736
5 165.36 ± 1,366.64 132.18 ± 792.28 9114400 1 0.0823
10 162.38 ± 1,179.11 129.25 ± 820.71 11488000 1 0.0790
15 172.25 ± 1,222.08 120.15 ± 734.43 12707000 1 0.0786
Hypothesis H1.4. One-tailed Mann-Whitney U test for comparing code effort ΔW of synchronous and non-synchronous commits for OpenSSL. A p-value < 0.05 means that the code effort ΔW in synchronous commits is significantly lower than the code effort ΔW in non-synchronous commits.
OpenSSL
ξ ΔW (mean ± std. dev.) in
synchronous commits		 ΔW (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 706.69 ± 14,256.52 121.61 ± 1,391.34 1335300 1 0.1164
5 400.57 ± 9,638.69 114.93 ± 1,560.62 2227900 1 0.1030
10 315.29 ± 8,170.89 119.53 ± 1,749.45 2440600 1 0.1060
15 333.69 ± 7,722.08 54.91 ± 147.70 2422700 1 0.1240
file
1 743.60 ± 16,919.80 107.34 ± 1,326.78 3667200 1 0.1533
5 382.26 ± 10,903.57 102.87 ± 1,432.18 7201600 1 0.1507
10 369.61 ± 9,619.07 67.96 ± 242.24 8425100 1 0.1570
15 323.46 ± 8,825.31 66.63 ± 242.61 8851400 1 0.1523
Hypothesis H1.4. One-tailed Mann-Whitney U test for comparing code effort ΔW of synchronous and non-synchronous commits for QEMU. A p-value < 0.05 means that the code effort ΔW in synchronous commits is significantly lower than the code effort ΔW in non-synchronous commits.
QEMU
ξ ΔW (mean ± std. dev.) in
synchronous commits		 ΔW (mean ± std. dev.) in
non-synchronous commits		 U p-value Cliff's
Delta
feature
1 118.55 ± 500.31 130.35 ± 438.24 6727800 0.005738 -0.0359
5 109.04 ± 423.50 146.46 ± 486.60 8453900 1.358e-05 -0.0527
10 107.70 ± 401.94 158.62 ± 529.27 7941400 2.102e-06 -0.0594
15 109.36 ± 400.10 162.00 ± 546.24 7518300 2.157e-06 -0.0609
file
1 87.90 ± 764.97 74.34 ± 318.52 99455000 1 0.0858
5 78.64 ± 563.80 75.55 ± 326.28 158850000 1 0.0666
10 75.28 ± 493.61 78.33 ± 349.17 166410000 1 0.0533
15 72.55 ± 460.57 83.60 ± 378.55 155570000 1 0.0448

Hypothesis H3.1

(Jump to results of H3.1 using smoothing parameter and band-window size 1ξ | 2ξ | 3ξ)

Smoothing parameter and band-window size: 1ξ

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for BusyBox. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 1ξ
BusyBox
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.25 ± 0.19 0.55 ± 0.32 31.5 0.02809 -0.6613
5 message 0.12 ± 0.09 0.80 ± 0.41 12 8.058e-07 -0.9917
10 message 0.38 ± 1.01 0.87 ± 0.69 1079 3.432e-06 -0.7563
15 message 0.37 ± 0.88 0.94 ± 0.83 2369 2.147e-06 -0.6906
1 conversation 0.55 ± 0.27 0.66 ± 0.38 61 0.356 -0.1348
5 conversation 0.11 ± 0.07 0.80 ± 0.42 5 8.594e-07 -0.9949
10 conversation 0.42 ± 1.05 0.89 ± 0.64 491 2.697e-06 -0.8025
15 conversation 0.40 ± 0.91 0.97 ± 0.83 1190 1.547e-06 -0.7284
file • lower-bound approach
1 message 0.43 ± 0.32 0.88 ± 0.72 1369 0.004908 -0.4074
5 message 0.72 ± 1.25 1.90 ± 2.40 8089 2.363e-06 -0.5236
10 message 1.13 ± 2.94 1.90 ± 2.80 21368 7.74e-07 -0.4726
15 message 0.88 ± 2.19 1.95 ± 2.95 27716 8.219e-09 -0.5254
1 conversation 0.43 ± 0.38 1.07 ± 0.84 789.5 0.0007932 -0.5031
5 conversation 0.73 ± 1.29 2.47 ± 2.82 4256 8.945e-08 -0.6008
10 conversation 0.76 ± 1.38 2.64 ± 3.33 7673 8.53e-09 -0.6016
15 conversation 0.63 ± 1.08 2.67 ± 3.52 9958.5 1.836e-11 -0.6528
feature • upper-bound approach
1 message 6.74 ± 11.50 12.14 ± 17.06 23419 0.2708 -0.0573
5 message 13.31 ± 14.91 15.51 ± 18.25 109990 0.321 -0.0341
10 message 11.33 ± 13.44 11.97 ± 14.50 199510 0.3757 -0.0213
15 message 9.25 ± 10.89 9.98 ± 12.01 266430 0.2336 -0.0464
1 conversation 8.27 ± 11.70 17.53 ± 17.45 6978 0.001603 -0.3044
5 conversation 9.47 ± 12.11 18.89 ± 19.64 28781 1.49e-05 -0.3254
10 conversation 8.73 ± 11.90 16.52 ± 17.73 45411 4.634e-06 -0.3249
15 conversation 7.18 ± 9.64 14.65 ± 15.98 58123 6.51e-07 -0.3386
file • upper-bound approach
1 message 9.01 ± 14.76 10.51 ± 15.62 150470 0.4657 -0.0055
5 message 11.32 ± 14.89 12.78 ± 16.69 459040 0.06565 -0.0821
10 message 8.56 ± 11.70 9.43 ± 12.63 792920 0.01302 -0.1108
15 message 6.92 ± 9.38 7.77 ± 10.21 1052700 0.001046 -0.1448
1 conversation 7.54 ± 11.13 14.81 ± 17.31 43772 3.587e-05 -0.2709
5 conversation 7.91 ± 12.51 15.03 ± 18.24 114840 7.036e-09 -0.3292
10 conversation 6.37 ± 10.08 12.17 ± 15.47 175430 1.313e-09 -0.3260
15 conversation 5.08 ± 8.11 10.45 ± 13.54 220680 1.603e-11 -0.3454
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for OpenSSL. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 1ξ
OpenSSL
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message -- ± -- -- ± -- -- -- --
5 message 1.49 ± -- 1.24 ± 0.94 95 0.8289 0.5447
10 message 1.39 ± 1.08 1.53 ± 1.30 328 0.5391 0.0314
15 message 1.11 ± 1.07 1.69 ± 1.69 333 0.2536 -0.2238
1 conversation -- ± -- -- ± -- -- -- --
5 conversation -- ± -- -- ± -- -- -- --
10 conversation -- ± -- -- ± -- -- -- --
15 conversation -- ± -- -- ± -- -- -- --
file • lower-bound approach
1 message 0.51 ± -- 0.74 ± 0.28 6 0.1481 -0.6471
5 message 0.57 ± 0.57 1.18 ± 0.72 104 0.156 -0.4190
10 message 1.00 ± 0.71 1.48 ± 1.02 462 0.1607 -0.2892
15 message 0.89 ± 0.72 1.56 ± 1.13 738 0.09502 -0.3411
1 conversation -- ± -- -- ± -- -- -- --
5 conversation 1.14 ± -- 1.11 ± 0.53 27 0.7179 0.4211
10 conversation 1.17 ± -- 1.41 ± 0.75 28 0.477 -0.0508
15 conversation 1.36 ± -- 1.44 ± 0.73 44 0.5671 0.0864
feature • upper-bound approach
1 message 6.63 ± 8.34 9.46 ± 12.61 168510 0.1669 -0.0696
5 message 9.77 ± 12.15 11.72 ± 14.87 739630 0.1316 -0.0579
10 message 7.98 ± 9.89 9.34 ± 11.90 1266200 0.1006 -0.0590
15 message 6.82 ± 8.30 7.88 ± 9.90 1805200 0.1412 -0.0456
1 conversation 3.42 ± 3.51 3.60 ± 3.78 58068 0.3198 -0.0430
5 conversation 4.52 ± 4.54 5.12 ± 5.11 191410 0.1981 -0.0593
10 conversation 3.97 ± 4.57 4.76 ± 5.08 284320 0.0441 -0.1086
15 conversation 3.45 ± 4.23 4.56 ± 4.93 351890 0.001909 -0.1705
file • upper-bound approach
1 message 6.79 ± 8.56 9.20 ± 12.30 206560 0.3156 -0.0333
5 message 8.14 ± 11.32 10.83 ± 14.20 1012300 0.004428 -0.1191
10 message 6.68 ± 9.11 8.61 ± 11.42 1765600 0.006031 -0.1023
15 message 5.77 ± 7.50 7.16 ± 9.36 2620000 0.01207 -0.0845
1 conversation 3.69 ± 3.66 3.80 ± 3.65 71590 0.2943 -0.0464
5 conversation 4.30 ± 4.81 5.15 ± 5.31 231950 0.02911 -0.1215
10 conversation 3.53 ± 4.67 4.86 ± 5.35 341420 0.0001486 -0.2071
15 conversation 3.25 ± 4.42 4.59 ± 5.23 445400 3.199e-05 -0.2147
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for QEMU. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 1ξ
QEMU
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.25 ± 0.28 0.45 ± 0.42 13366 1.36e-08 -0.3333
5 message 0.31 ± 0.47 1.20 ± 1.28 69318 < 2.2e-16 -0.7585
10 message 0.30 ± 0.51 1.38 ± 1.51 186200 < 2.2e-16 -0.7823
15 message 0.25 ± 0.45 1.43 ± 1.56 288960 < 2.2e-16 -0.8110
1 conversation 0.44 ± 0.42 0.59 ± 0.49 4628.5 0.004488 -0.2057
5 conversation 0.86 ± 1.00 1.52 ± 1.52 24226 6.055e-16 -0.3929
10 conversation 0.88 ± 1.10 1.77 ± 1.83 51951 < 2.2e-16 -0.4394
15 conversation 0.86 ± 1.12 1.90 ± 1.93 73603 < 2.2e-16 -0.4855
file • lower-bound approach
1 message 0.29 ± 0.34 0.52 ± 0.52 92955 < 2.2e-16 -0.3160
5 message 0.50 ± 0.79 1.65 ± 2.10 437570 < 2.2e-16 -0.6608
10 message 0.43 ± 0.79 1.89 ± 2.44 866550 < 2.2e-16 -0.7391
15 message 0.37 ± 0.69 1.91 ± 2.39 1278900 < 2.2e-16 -0.7704
1 conversation 0.52 ± 0.55 0.74 ± 0.63 43613 5.402e-09 -0.2529
5 conversation 1.07 ± 1.30 2.01 ± 2.38 220690 2.2e-16 -0.3936
10 conversation 1.07 ± 1.43 2.34 ± 2.81 424970 < 2.2e-16 -0.4470
15 conversation 1.05 ± 1.45 2.39 ± 2.78 605500 < 2.2e-16 -0.4708
feature • upper-bound approach
1 message 10.66 ± 18.13 8.70 ± 15.16 376520 0.998 0.0836
5 message 15.65 ± 15.90 19.40 ± 20.44 2548600 1.796e-06 -0.0811
10 message 11.80 ± 11.30 14.50 ± 14.64 6174500 2.218e-09 -0.0821
15 message 9.40 ± 8.88 11.83 ± 11.65 9783500 < 2.2e-16 -0.1013
1 conversation 7.15 ± 14.36 8.49 ± 15.43 137800 0.02016 -0.0734
5 conversation 9.44 ± 15.92 10.91 ± 18.81 894930 0.0106 -0.0524
10 conversation 7.09 ± 11.63 8.60 ± 14.55 1839100 4.059e-05 -0.0743
15 conversation 5.80 ± 9.16 7.38 ± 11.87 2566300 1.468e-10 -0.1083
file • upper-bound approach
1 message 10.52 ± 16.85 8.92 ± 14.64 1118600 0.9975 0.0608
5 message 15.45 ± 15.18 20.02 ± 20.04 6038900 < 2.2e-16 -0.1127
10 message 11.12 ± 10.60 14.74 ± 14.26 13431000 < 2.2e-16 -0.1306
15 message 8.85 ± 8.22 11.75 ± 11.12 21168000 < 2.2e-16 -0.1365
1 conversation 6.39 ± 12.13 8.77 ± 15.89 556040 0.0001101 -0.0912
5 conversation 7.67 ± 12.94 9.92 ± 16.73 2976900 9.035e-10 -0.0989
10 conversation 5.22 ± 8.76 7.42 ± 12.11 5537900 < 2.2e-16 -0.1569
15 conversation 4.03 ± 6.53 6.05 ± 9.51 7637000 < 2.2e-16 -0.1863

Smoothing parameter and band-window size: 2ξ

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for BusyBox. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 2ξ
BusyBox
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.15 ± 0.13 0.56 ± 0.21 9 0.004511 -0.9032
5 message 0.06 ± 0.08 0.67 ± 0.39 10 7.792e-07 -0.9931
10 message 0.26 ± 0.70 0.72 ± 0.61 1090.5 3.691e-06 -0.7537
15 message 0.25 ± 0.64 0.76 ± 0.72 1897.5 2.772e-07 -0.7522
1 conversation 0.14 ± 0.13 0.58 ± 0.20 6 0.00445 -0.9149
5 conversation 0.06 ± 0.08 0.66 ± 0.38 11 9.953e-07 -0.9887
10 conversation 0.28 ± 0.72 0.74 ± 0.57 515 3.493e-06 -0.7928
15 conversation 0.26 ± 0.66 0.78 ± 0.73 1006 4.04e-07 -0.7704
file • lower-bound approach
1 message 0.43 ± 0.38 1.11 ± 1.03 1080 0.0003705 -0.5325
5 message 0.56 ± 1.02 1.60 ± 2.11 7514 5.504e-07 -0.5574
10 message 0.85 ± 2.27 1.53 ± 2.34 21183 6.131e-07 -0.4771
15 message 0.66 ± 1.75 1.53 ± 2.39 25236 5.234e-10 -0.5679
1 conversation 0.43 ± 0.39 1.33 ± 1.15 615.5 6.06e-05 -0.6126
5 conversation 0.56 ± 1.06 2.08 ± 2.49 4044 3.467e-08 -0.6206
10 conversation 0.56 ± 0.89 2.13 ± 2.78 7878.5 1.517e-08 -0.5909
15 conversation 0.44 ± 0.82 2.11 ± 2.85 8823 1.125e-12 -0.6924
feature • upper-bound approach
1 message 11.24 ± 16.02 20.90 ± 23.85 19416 0.009991 -0.2185
5 message 10.32 ± 11.54 12.77 ± 14.81 105930 0.1709 -0.0697
10 message 7.74 ± 9.16 8.78 ± 10.30 190510 0.1648 -0.0654
15 message 5.89 ± 6.94 6.84 ± 7.89 251590 0.05942 -0.0995
1 conversation 8.93 ± 12.08 18.79 ± 18.55 6776.5 0.0008398 -0.3245
5 conversation 7.77 ± 9.86 15.76 ± 16.14 28038 5.451e-06 -0.3428
10 conversation 6.67 ± 8.91 13.11 ± 13.77 43935 1.106e-06 -0.3468
15 conversation 5.31 ± 7.17 11.06 ± 11.80 56581 1.789e-07 -0.3562
file • upper-bound approach
1 message 12.51 ± 17.60 17.86 ± 22.42 127270 0.006106 -0.1588
5 message 8.64 ± 10.47 11.35 ± 13.50 438340 0.01162 -0.1235
10 message 5.81 ± 8.01 6.94 ± 9.03 749250 0.000664 -0.1597
15 message 4.40 ± 6.01 5.39 ± 6.79 989630 1.55e-05 -0.1960
1 conversation 9.42 ± 14.18 16.12 ± 18.44 43538 2.816e-05 -0.2748
5 conversation 6.31 ± 9.88 12.42 ± 14.94 112770 2.04e-09 -0.3413
10 conversation 4.77 ± 7.58 9.54 ± 11.95 169080 7.836e-11 -0.3503
15 conversation 3.71 ± 5.99 7.81 ± 9.95 212860 7.075e-13 -0.3686
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for OpenSSL. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 2ξ
OpenSSL
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message -- ± -- -- ± -- -- -- --
5 message 1.42 ± -- 1.16 ± 0.91 93 0.8143 0.5122
10 message 1.48 ± 1.10 1.37 ± 1.20 361 0.6578 0.1352
15 message 0.94 ± 0.91 1.49 ± 1.49 336 0.2603 -0.2168
1 conversation -- ± -- -- ± -- -- -- --
5 conversation -- ± -- -- ± -- -- -- --
10 conversation -- ± -- -- ± -- -- -- --
15 conversation -- ± -- -- ± -- -- -- --
file • lower-bound approach
1 message 0.03 ± -- 0.67 ± 0.27 0 0.05063 -1.0000
5 message 0.55 ± 0.55 0.99 ± 0.63 117 0.202 -0.3464
10 message 0.86 ± 0.69 1.18 ± 0.88 502 0.2177 -0.2277
15 message 0.57 ± 0.49 1.22 ± 0.95 547 0.02462 -0.5116
1 conversation -- ± -- -- ± -- -- -- --
5 conversation 1.11 ± -- 0.96 ± 0.46 29 0.7692 0.5263
10 conversation 1.18 ± -- 1.18 ± 0.67 35 0.6355 0.1864
15 conversation 1.09 ± -- 1.15 ± 0.54 41 0.5168 0.0123
feature • upper-bound approach
1 message 9.65 ± 10.98 14.65 ± 16.33 153510 0.01714 -0.1525
5 message 9.99 ± 7.72 12.60 ± 9.50 717970 0.04919 -0.0854
10 message 5.61 ± 6.91 7.28 ± 9.10 1219600 0.02131 -0.0936
15 message 4.53 ± 5.50 5.80 ± 7.02 1721300 0.01697 -0.0900
1 conversation 4.55 ± 4.20 4.04 ± 4.03 64523 0.7547 0.0633
5 conversation 3.80 ± 3.93 4.46 ± 4.42 185520 0.1033 -0.0883
10 conversation 3.17 ± 3.79 3.98 ± 4.20 274710 0.01472 -0.1388
15 conversation 2.74 ± 3.51 3.68 ± 3.89 342900 0.0005729 -0.1917
file • upper-bound approach
1 message 9.55 ± 10.96 13.97 ± 15.97 186460 0.03323 -0.1274
5 message 6.36 ± 8.73 9.17 ± 12.00 983850 0.000782 -0.1438
10 message 4.63 ± 6.26 6.66 ± 8.69 1689600 0.0002704 -0.1409
15 message 3.79 ± 4.95 5.22 ± 6.58 2488300 0.000248 -0.1305
1 conversation 4.62 ± 4.38 4.20 ± 3.94 76566 0.5918 0.0199
5 conversation 3.48 ± 3.91 4.39 ± 4.52 227090 0.0146 -0.1399
10 conversation 2.78 ± 3.75 3.99 ± 4.35 332640 3.53e-05 -0.2275
15 conversation 2.44 ± 3.49 3.63 ± 4.07 420070 6.842e-07 -0.2593
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for QEMU. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 2ξ
QEMU
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.29 ± 0.36 0.65 ± 0.51 7842 < 2.2e-16 -0.6088
5 message 0.20 ± 0.35 1.10 ± 1.24 52735 < 2.2e-16 -0.8162
10 message 0.16 ± 0.33 1.16 ± 1.30 130300 < 2.2e-16 -0.8477
15 message 0.13 ± 0.28 1.14 ± 1.28 226660 < 2.2e-16 -0.8518
1 conversation 0.66 ± 0.66 0.79 ± 0.60 4543 0.002579 -0.2204
5 conversation 0.66 ± 0.85 1.37 ± 1.44 22011 < 2.2e-16 -0.4484
10 conversation 0.62 ± 0.83 1.51 ± 1.55 43588 < 2.2e-16 -0.5296
15 conversation 0.61 ± 0.87 1.56 ± 1.58 65972 < 2.2e-16 -0.5388
file • lower-bound approach
1 message 0.40 ± 0.48 0.89 ± 0.84 60276 < 2.2e-16 -0.5565
5 message 0.30 ± 0.56 1.46 ± 1.91 327120 < 2.2e-16 -0.7464
10 message 0.22 ± 0.46 1.53 ± 1.93 656910 < 2.2e-16 -0.8023
15 message 0.17 ± 0.39 1.45 ± 1.77 950620 < 2.2e-16 -0.8293
1 conversation 0.72 ± 0.73 1.05 ± 0.90 40476 2.124e-12 -0.3066
5 conversation 0.80 ± 1.05 1.74 ± 2.12 201110 < 2.2e-16 -0.4474
10 conversation 0.73 ± 1.05 1.88 ± 2.20 375170 < 2.2e-16 -0.5118
15 conversation 0.68 ± 1.01 1.83 ± 2.07 530830 < 2.2e-16 -0.5361
feature • upper-bound approach
1 message 20.97 ± 26.46 28.03 ± 32.23 300040 1.34e-06 -0.1365
5 message 12.61 ± 12.51 16.11 ± 16.27 2482000 9.51e-10 -0.1051
10 message 8.36 ± 7.89 10.51 ± 9.98 5960800 < 2.2e-16 -0.1139
15 message 6.19 ± 5.82 7.98 ± 7.36 9399200 < 2.2e-16 -0.1366
1 conversation 11.44 ± 20.58 13.90 ± 22.75 135800 0.007634 -0.0868
5 conversation 7.85 ± 12.93 9.23 ± 15.42 884480 0.00263 -0.0635
10 conversation 5.44 ± 8.50 6.83 ± 10.90 1794100 1.35e-07 -0.0969
15 conversation 4.17 ± 6.20 5.54 ± 8.25 2480600 4.747e-16 -0.1381
file • upper-bound approach
1 message 21.41 ± 25.35 30.52 ± 32.38 881640 1.634e-14 -0.1639
5 message 12.31 ± 11.86 16.49 ± 15.86 5858400 < 2.2e-16 -0.1392
10 message 7.77 ± 7.30 10.55 ± 9.61 12954000 < 2.2e-16 -0.1614
15 message 5.74 ± 5.32 7.82 ± 6.93 20268000 < 2.2e-16 -0.1732
1 conversation 10.22 ± 17.17 14.95 ± 23.59 532990 8.911e-08 -0.1289
5 conversation 6.27 ± 10.29 8.32 ± 13.54 2927900 2.34e-12 -0.1138
10 conversation 3.88 ± 6.17 5.80 ± 8.83 5349600 2.2e-16 -0.1856
15 conversation 2.79 ± 4.23 4.46 ± 6.41 7273000 < 2.2e-16 -0.2251

Smoothing parameter and band-window size: 3ξ

(Jump to data for BusyBox | OpenSSL | QEMU)
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for BusyBox. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 3ξ
BusyBox
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.06 ± 0.04 0.47 ± 0.16 0 0.001911 -1.0000
5 message 0.05 ± 0.08 0.62 ± 0.39 14 8.333e-07 -0.9903
10 message 0.22 ± 0.59 0.65 ± 0.58 1177 6.321e-06 -0.7342
15 message 0.21 ± 0.54 0.68 ± 0.65 2144 8.25e-07 -0.7200
1 conversation 0.06 ± 0.04 0.48 ± 0.16 0 0.002111 -1.0000
5 conversation 0.05 ± 0.08 0.62 ± 0.37 14.5 1.084e-06 -0.9851
10 conversation 0.24 ± 0.62 0.67 ± 0.55 583 7.156e-06 -0.7655
15 conversation 0.23 ± 0.56 0.71 ± 0.69 1284 2.99e-06 -0.7070
file • lower-bound approach
1 message 0.35 ± 0.35 0.98 ± 0.97 1111 0.0005033 -0.5190
5 message 0.51 ± 0.90 1.46 ± 1.94 7544 5.951e-07 -0.5557
10 message 0.73 ± 1.93 1.36 ± 2.07 19760 9.511e-08 -0.5123
15 message 0.59 ± 1.59 1.34 ± 2.08 24325 1.81e-10 -0.5835
1 conversation 0.37 ± 0.34 1.18 ± 1.07 634 8.147e-05 -0.6010
5 conversation 0.52 ± 0.92 1.90 ± 2.29 4121 4.908e-08 -0.6134
10 conversation 0.49 ± 0.87 1.89 ± 2.47 7063 1.457e-09 -0.6333
15 conversation 0.39 ± 0.78 1.84 ± 2.49 8829.5 1.143e-12 -0.6922
feature • upper-bound approach
1 message 9.78 ± 13.93 18.42 ± 20.98 19317 0.00892 -0.2224
5 message 9.15 ± 10.53 11.72 ± 13.71 104260 0.1249 -0.0844
10 message 6.54 ± 7.76 7.73 ± 9.17 186130 0.09768 -0.0869
15 message 4.79 ± 5.71 5.87 ± 6.82 245590 0.02895 -0.1209
1 conversation 7.35 ± 10.01 15.75 ± 15.23 6521.5 0.0003522 -0.3499
5 conversation 6.69 ± 8.37 14.09 ± 14.26 26922 1.1e-06 -0.3690
10 conversation 5.47 ± 7.30 11.13 ± 11.51 42819 3.534e-07 -0.3634
15 conversation 4.30 ± 5.79 9.06 ± 9.42 55525 7.129e-08 -0.3682
file • upper-bound approach
1 message 10.61 ± 14.88 15.67 ± 19.66 125580 0.003659 -0.1700
5 message 7.53 ± 10.08 9.52 ± 12.38 429660 0.00482 -0.1409
10 message 4.84 ± 6.78 6.06 ± 7.99 721850 6.48e-05 -0.1905
15 message 3.49 ± 4.83 4.59 ± 5.86 963810 1.997e-06 -0.2170
1 conversation 7.64 ± 11.46 13.50 ± 15.19 42628 1.07e-05 -0.2900
5 conversation 5.34 ± 8.20 11.03 ± 13.12 110460 4.858e-10 -0.3548
10 conversation 3.91 ± 6.21 8.07 ± 9.95 163490 5.552e-12 -0.3718
15 conversation 2.96 ± 4.75 6.41 ± 8.02 210340 2.481e-13 -0.3761
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for OpenSSL. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 3ξ
OpenSSL
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message -- ± -- -- ± -- -- -- --
5 message 1.61 ± -- 1.12 ± 0.89 103 0.8797 0.6748
10 message 1.29 ± 0.97 1.28 ± 1.14 346 0.605 0.0881
15 message 0.83 ± 0.78 1.37 ± 1.39 323 0.2319 -0.2471
1 conversation -- ± -- -- ± -- -- -- --
5 conversation -- ± -- -- ± -- -- -- --
10 conversation -- ± -- -- ± -- -- -- --
15 conversation -- ± -- -- ± -- -- -- --
file • lower-bound approach
1 message 0.00 ± -- 0.56 ± 0.20 0 0.05094 -1.0000
5 message 0.53 ± 0.53 0.90 ± 0.60 122 0.2216 -0.3184
10 message 0.70 ± 0.58 1.04 ± 0.79 453 0.1493 -0.3031
15 message 0.52 ± 0.46 1.04 ± 0.84 584 0.03292 -0.4786
1 conversation -- ± -- -- ± -- -- -- --
5 conversation 1.06 ± -- 0.89 ± 0.48 31 0.8205 0.6316
10 conversation 1.02 ± -- 1.04 ± 0.54 32 0.5688 0.0847
15 conversation 1.13 ± -- 1.01 ± 0.47 53 0.7086 0.3086
feature • upper-bound approach
1 message 8.31 ± 9.27 12.61 ± 13.89 153900 0.01845 -0.1503
5 message 6.83 ± 8.41 9.05 ± 11.33 706680 0.02675 -0.0998
10 message 4.72 ± 5.80 6.34 ± 7.83 1197400 0.008515 -0.1101
15 message 3.79 ± 4.62 4.98 ± 5.96 1689900 0.005997 -0.1066
1 conversation 4.05 ± 3.63 3.61 ± 3.47 64187 0.7353 0.0578
5 conversation 4.13 ± 3.39 4.04 ± 3.53 181240 0.05902 -0.1093
10 conversation 2.86 ± 3.48 3.55 ± 3.66 271270 0.009464 -0.1495
15 conversation 2.35 ± 3.15 3.22 ± 3.31 329440 7.515e-05 -0.2234
file • upper-bound approach
1 message 8.20 ± 9.26 11.99 ± 13.58 186460 0.03326 -0.1274
5 message 5.52 ± 7.65 8.28 ± 10.80 961030 0.0001596 -0.1637
10 message 3.86 ± 5.27 5.78 ± 7.47 1645000 2.962e-05 -0.1636
15 message 3.11 ± 4.07 4.46 ± 5.55 2419400 1.848e-05 -0.1545
1 conversation 4.03 ± 3.82 3.68 ± 3.40 76082 0.5624 0.0135
5 conversation 3.10 ± 3.49 4.02 ± 4.10 223210 0.007976 -0.1546
10 conversation 2.39 ± 3.31 3.52 ± 3.77 319020 2.998e-06 -0.2591
15 conversation 2.04 ± 3.02 3.14 ± 3.45 400920 2.407e-08 -0.2931
Hypothesis H3.1. One-tailed Mann-Whitney U test for comparing DTW distances for empirical and simulated bursts for QEMU. A p-value < 0.05 means that the empirical DTW distances are significantly smaller than the simulated DTW distances. Smoothing parameter and band-window size: 3ξ
QEMU
ξ mailing-list abstraction mean ± std. dev. of
empirical DTW distances		 mean ± std. dev. of
simulated DTW distances		 U p-value Cliff's
Delta
feature • lower-bound approach
1 message 0.26 ± 0.38 0.65 ± 0.52 7026 < 2.2e-16 -0.6495
5 message 0.15 ± 0.29 1.04 ± 1.17 45847 < 2.2e-16 -0.8402
10 message 0.11 ± 0.27 1.04 ± 1.14 118750 < 2.2e-16 -0.8612
15 message 0.09 ± 0.22 1.00 ± 1.11 221620 < 2.2e-16 -0.8551
1 conversation 0.60 ± 0.65 0.75 ± 0.59 4427.5 0.00115 -0.2402
5 conversation 0.55 ± 0.76 1.29 ± 1.34 19858 < 2.2e-16 -0.5023
10 conversation 0.52 ± 0.75 1.36 ± 1.37 40950 < 2.2e-16 -0.5581
15 conversation 0.49 ± 0.71 1.37 ± 1.37 61506 < 2.2e-16 -0.5700
file • lower-bound approach
1 message 0.38 ± 0.49 0.86 ± 0.85 59664 < 2.2e-16 -0.5610
5 message 0.22 ± 0.45 1.34 ± 1.76 279790 < 2.2e-16 -0.7831
10 message 0.15 ± 0.36 1.35 ± 1.67 570080 < 2.2e-16 -0.8284
15 message 0.11 ± 0.28 1.24 ± 1.48 854640 < 2.2e-16 -0.8466
1 conversation 0.66 ± 0.70 0.95 ± 0.86 40636 3.276e-12 -0.3039
5 conversation 0.66 ± 0.92 1.58 ± 1.90 185550 < 2.2e-16 -0.4901
10 conversation 0.59 ± 0.88 1.65 ± 1.88 348860 < 2.2e-16 -0.5460
15 conversation 0.53 ± 0.84 1.56 ± 1.71 494030 < 2.2e-16 -0.5682
feature • upper-bound approach
1 message 21.35 ± 25.74 30.48 ± 34.41 295000 1.039e-07 -0.1510
5 message 11.37 ± 11.26 15.04 ± 15.19 2434000 1.333e-12 -0.1224
10 message 7.04 ± 6.66 9.16 ± 8.58 5810100 < 2.2e-16 -0.1363
15 message 4.95 ± 4.67 6.68 ± 6.09 9085400 < 2.2e-16 -0.1654
1 conversation 10.36 ± 18.17 12.21 ± 19.66 137750 0.0197 -0.0737
5 conversation 6.82 ± 10.85 8.22 ± 13.20 871040 0.0003181 -0.0777
10 conversation 4.45 ± 6.63 5.75 ± 8.64 1744500 4.952e-11 -0.1219
15 conversation 3.27 ± 4.63 4.53 ± 6.31 2405800 < 2.2e-16 -0.1641
file • upper-bound approach
1 message 22.23 ± 25.10 33.60 ± 35.03 866330 < 2.2e-16 -0.1785
5 message 11.03 ± 10.64 15.36 ± 14.77 5732400 < 2.2e-16 -0.1577
10 message 6.51 ± 6.13 9.15 ± 8.25 12612000 < 2.2e-16 -0.1836
15 message 4.55 ± 4.26 6.51 ± 5.70 19533000 < 2.2e-16 -0.2032
1 conversation 9.27 ± 15.23 13.11 ± 20.43 541740 1.734e-06 -0.1146
5 conversation 5.39 ± 8.51 7.37 ± 11.48 2871600 9.239e-16 -0.1308
10 conversation 3.12 ± 4.73 4.86 ± 6.94 5174200 < 2.2e-16 -0.2123
15 conversation 2.14 ± 3.09 3.63 ± 4.87 6975000 < 2.2e-16 -0.2569