https://launchpad.net/ubuntu/+archive/test-rebuild-20220617-kinetic/+build/24000232 RUN: /usr/share/launchpad-buildd/bin/builder-prep Kernel version: Linux lgw01-amd64-015 5.4.0-120-generic #136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022 x86_64 Buildd toolchain package versions: launchpad-buildd_215~563~ubuntu20.04.1 python3-lpbuildd_215~563~ubuntu20.04.1 sbuild_0.79.0-1ubuntu1 git-build-recipe_0.3.6 git_1:2.25.1-1ubuntu3.4 dpkg-dev_1.19.7ubuntu3.2 python3-debian_0.1.36ubuntu1. Syncing the system clock with the buildd NTP service... 22 Jun 14:51:56 ntpdate[1830]: adjust time server 10.211.37.1 offset -0.001101 sec RUN: /usr/share/launchpad-buildd/bin/in-target unpack-chroot --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 --image-type chroot /home/buildd/filecache-default/4d18961cd05542d5def75b14497015b674c37821 Creating target for build PACKAGEBUILD-24000232 RUN: /usr/share/launchpad-buildd/bin/in-target mount-chroot --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 Starting target for build PACKAGEBUILD-24000232 RUN: /usr/share/launchpad-buildd/bin/in-target override-sources-list --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 'deb http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic main' 'deb http://ftpmaster.internal/ubuntu kinetic main universe' Overriding sources.list in build-PACKAGEBUILD-24000232 RUN: /usr/share/launchpad-buildd/bin/in-target add-trusted-keys --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 Adding trusted keys to build-PACKAGEBUILD-24000232 pub rsa1024/1E9377A2BA9EF27F 2009-10-22 [SC] Key fingerprint = 60C3 1780 3A41 BA51 845E 371A 1E93 77A2 BA9E F27F uid Launchpad Toolchain builds RUN: /usr/share/launchpad-buildd/bin/in-target update-debian-chroot --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 Updating target for build PACKAGEBUILD-24000232 Get:1 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic InRelease [23.8 kB] Get:2 http://ftpmaster.internal/ubuntu kinetic InRelease [267 kB] Get:3 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 Packages [9632 B] Get:4 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main Translation-en [4132 B] Get:5 http://ftpmaster.internal/ubuntu kinetic/main amd64 Packages [1398 kB] Get:6 http://ftpmaster.internal/ubuntu kinetic/main Translation-en [510 kB] Get:7 http://ftpmaster.internal/ubuntu kinetic/universe amd64 Packages [13.8 MB] Get:8 http://ftpmaster.internal/ubuntu kinetic/universe Translation-en [5728 kB] Fetched 21.8 MB in 4s (6131 kB/s) Reading package lists... Reading package lists... Building dependency tree... Reading state information... Calculating upgrade... The following NEW packages will be installed: libgprofng0 util-linux-extra The following packages will be upgraded: adduser apt base-files binutils binutils-common binutils-x86-64-linux-gnu bsdutils cpp-11 dpkg dpkg-dev e2fsprogs fakeroot g++-11 gcc-11 gcc-11-base gcc-12-base init init-system-helpers libapparmor1 libapt-pkg6.0 libasan6 libatomic1 libbinutils libblkid1 libcap-ng0 libcc1-0 libcom-err2 libctf-nobfd0 libctf0 libdb5.3 libdpkg-perl libext2fs2 libfakeroot libgcc-11-dev libgcc-s1 libgcrypt20 libgnutls30 libgomp1 libgpg-error0 libip4tc2 libitm1 libkeyutils1 liblsan0 liblzma5 libmount1 libncurses6 libncursesw6 libpng16-16 libquadmath0 libreadline8 libselinux1 libsemanage-common libsemanage2 libsepol2 libsmartcols1 libsqlite3-0 libss2 libssl3 libstdc++-11-dev libstdc++6 libsystemd0 libtinfo6 libtsan0 libubsan1 libudev1 libuuid1 libzstd1 login logsave lto-disabled-list mawk mount ncurses-base ncurses-bin openssl passwd pinentry-curses policyrcd-script-zg2 readline-common systemd systemd-sysv util-linux xz-utils 83 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 79.7 MB of archives. After this operation, 5304 kB of additional disk space will be used. Get:1 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 libctf0 amd64 2.38.50.20220615-4ubuntu1 [103 kB] Get:2 http://ftpmaster.internal/ubuntu kinetic/main amd64 base-files amd64 12.2ubuntu1 [62.6 kB] Get:3 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 libctf-nobfd0 amd64 2.38.50.20220615-4ubuntu1 [107 kB] Get:4 http://ftpmaster.internal/ubuntu kinetic/main amd64 bsdutils amd64 1:2.38-4ubuntu1 [80.7 kB] Get:5 http://ftpmaster.internal/ubuntu kinetic/main amd64 libzstd1 amd64 1.5.2+dfsg-1 [270 kB] Get:6 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 binutils-x86-64-linux-gnu amd64 2.38.50.20220615-4ubuntu1 [2476 kB] Get:7 http://ftpmaster.internal/ubuntu kinetic/main amd64 libatomic1 amd64 12.1.0-2ubuntu1 [10.4 kB] Get:8 http://ftpmaster.internal/ubuntu kinetic/main amd64 libubsan1 amd64 12.1.0-2ubuntu1 [969 kB] Get:9 http://ftpmaster.internal/ubuntu kinetic/main amd64 libquadmath0 amd64 12.1.0-2ubuntu1 [152 kB] Get:10 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 libbinutils amd64 2.38.50.20220615-4ubuntu1 [655 kB] Get:11 http://ftpmaster.internal/ubuntu kinetic/main amd64 liblsan0 amd64 12.1.0-2ubuntu1 [1060 kB] Get:12 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 binutils amd64 2.38.50.20220615-4ubuntu1 [3292 B] Get:13 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 binutils-common amd64 2.38.50.20220615-4ubuntu1 [224 kB] Get:14 http://ppa.launchpadcontent.net/ubuntu-toolchain-r/binutils/ubuntu kinetic/main amd64 libgprofng0 amd64 2.38.50.20220615-4ubuntu1 [963 kB] Get:15 http://ftpmaster.internal/ubuntu kinetic/main amd64 libitm1 amd64 12.1.0-2ubuntu1 [29.5 kB] Get:16 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgomp1 amd64 12.1.0-2ubuntu1 [125 kB] Get:17 http://ftpmaster.internal/ubuntu kinetic/main amd64 gcc-12-base amd64 12.1.0-2ubuntu1 [18.8 kB] Get:18 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgcc-s1 amd64 12.1.0-2ubuntu1 [54.2 kB] Get:19 http://ftpmaster.internal/ubuntu kinetic/main amd64 libcc1-0 amd64 12.1.0-2ubuntu1 [46.6 kB] Get:20 http://ftpmaster.internal/ubuntu kinetic/main amd64 libstdc++6 amd64 12.1.0-2ubuntu1 [679 kB] Get:21 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgpg-error0 amd64 1.45-2 [69.0 kB] Get:22 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgcrypt20 amd64 1.10.1-2ubuntu1 [524 kB] Get:23 http://ftpmaster.internal/ubuntu kinetic/main amd64 liblzma5 amd64 5.2.5-2.1 [99.2 kB] Get:24 http://ftpmaster.internal/ubuntu kinetic/main amd64 systemd-sysv amd64 249.11-0ubuntu4 [10.2 kB] Get:25 http://ftpmaster.internal/ubuntu kinetic/main amd64 libapparmor1 amd64 3.0.4-2ubuntu3 [38.0 kB] Get:26 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgnutls30 amd64 3.7.6-2ubuntu1 [968 kB] Get:27 http://ftpmaster.internal/ubuntu kinetic/main amd64 libip4tc2 amd64 1.8.7-1ubuntu6 [19.7 kB] Get:28 http://ftpmaster.internal/ubuntu kinetic/main amd64 libblkid1 amd64 2.38-4ubuntu1 [103 kB] Get:29 http://ftpmaster.internal/ubuntu kinetic/main amd64 libselinux1 amd64 3.4-1 [77.6 kB] Get:30 http://ftpmaster.internal/ubuntu kinetic/main amd64 libmount1 amd64 2.38-4ubuntu1 [121 kB] Get:31 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsmartcols1 amd64 2.38-4ubuntu1 [50.8 kB] Get:32 http://ftpmaster.internal/ubuntu kinetic/main amd64 login amd64 1:4.11.1+dfsg1-2ubuntu1 [183 kB] Get:33 http://ftpmaster.internal/ubuntu kinetic/main amd64 util-linux-extra amd64 2.38-4ubuntu1 [83.8 kB] Get:34 http://ftpmaster.internal/ubuntu kinetic/main amd64 util-linux amd64 2.38-4ubuntu1 [1069 kB] Get:35 http://ftpmaster.internal/ubuntu kinetic/main amd64 mount amd64 2.38-4ubuntu1 [114 kB] Get:36 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsemanage-common all 3.4-1 [9828 B] Get:37 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsepol2 amd64 3.4-2 [294 kB] Get:38 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsemanage2 amd64 3.4-1 [91.8 kB] Get:39 http://ftpmaster.internal/ubuntu kinetic/main amd64 passwd amd64 1:4.11.1+dfsg1-2ubuntu1 [765 kB] Get:40 http://ftpmaster.internal/ubuntu kinetic/main amd64 adduser all 3.121ubuntu1 [150 kB] Get:41 http://ftpmaster.internal/ubuntu kinetic/main amd64 systemd amd64 249.11-0ubuntu4 [4517 kB] Get:42 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsystemd0 amd64 249.11-0ubuntu4 [314 kB] Get:43 http://ftpmaster.internal/ubuntu kinetic/main amd64 libudev1 amd64 249.11-0ubuntu4 [76.1 kB] Get:44 http://ftpmaster.internal/ubuntu kinetic/main amd64 libapt-pkg6.0 amd64 2.5.0 [896 kB] Get:45 http://ftpmaster.internal/ubuntu kinetic/main amd64 dpkg amd64 1.21.8ubuntu1 [1333 kB] Get:46 http://ftpmaster.internal/ubuntu kinetic/main amd64 ncurses-bin amd64 6.3+20220423-2 [183 kB] Get:47 http://ftpmaster.internal/ubuntu kinetic/main amd64 init-system-helpers all 1.63 [38.7 kB] Get:48 http://ftpmaster.internal/ubuntu kinetic/main amd64 ncurses-base all 6.3+20220423-2 [21.2 kB] Get:49 http://ftpmaster.internal/ubuntu kinetic/main amd64 apt amd64 2.5.0 [1370 kB] Get:50 http://ftpmaster.internal/ubuntu kinetic/main amd64 logsave amd64 1.46.5-2ubuntu2 [9914 B] Get:51 http://ftpmaster.internal/ubuntu kinetic/main amd64 libext2fs2 amd64 1.46.5-2ubuntu2 [205 kB] Get:52 http://ftpmaster.internal/ubuntu kinetic/main amd64 e2fsprogs amd64 1.46.5-2ubuntu2 [585 kB] Get:53 http://ftpmaster.internal/ubuntu kinetic/main amd64 init amd64 1.63 [5674 B] Get:54 http://ftpmaster.internal/ubuntu kinetic/main amd64 libcap-ng0 amd64 0.7.9-4 [12.1 kB] Get:55 http://ftpmaster.internal/ubuntu kinetic/main amd64 libdb5.3 amd64 5.3.28+dfsg1-0.9 [717 kB] Get:56 http://ftpmaster.internal/ubuntu kinetic/main amd64 libncurses6 amd64 6.3+20220423-2 [110 kB] Get:57 http://ftpmaster.internal/ubuntu kinetic/main amd64 libncursesw6 amd64 6.3+20220423-2 [146 kB] Get:58 http://ftpmaster.internal/ubuntu kinetic/main amd64 libtinfo6 amd64 6.3+20220423-2 [99.6 kB] Get:59 http://ftpmaster.internal/ubuntu kinetic/main amd64 libuuid1 amd64 2.38-4ubuntu1 [23.2 kB] Get:60 http://ftpmaster.internal/ubuntu kinetic/main amd64 libcom-err2 amd64 1.46.5-2ubuntu2 [9074 B] Get:61 http://ftpmaster.internal/ubuntu kinetic/main amd64 libss2 amd64 1.46.5-2ubuntu2 [12.3 kB] Get:62 http://ftpmaster.internal/ubuntu kinetic/main amd64 mawk amd64 1.3.4.20200120-3.1 [109 kB] Get:63 http://ftpmaster.internal/ubuntu kinetic/main amd64 libkeyutils1 amd64 1.6.1-3ubuntu1 [10.1 kB] Get:64 http://ftpmaster.internal/ubuntu kinetic/main amd64 readline-common all 8.1.2-1.2 [53.6 kB] Get:65 http://ftpmaster.internal/ubuntu kinetic/main amd64 libreadline8 amd64 8.1.2-1.2 [153 kB] Get:66 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsqlite3-0 amd64 3.38.5-1 [646 kB] Get:67 http://ftpmaster.internal/ubuntu kinetic/main amd64 libssl3 amd64 3.0.3-5ubuntu3 [1897 kB] Get:68 http://ftpmaster.internal/ubuntu kinetic/main amd64 openssl amd64 3.0.3-5ubuntu3 [1175 kB] Get:69 http://ftpmaster.internal/ubuntu kinetic/main amd64 libpng16-16 amd64 1.6.37-5 [187 kB] Get:70 http://ftpmaster.internal/ubuntu kinetic/main amd64 xz-utils amd64 5.2.5-2.1 [83.8 kB] Get:71 http://ftpmaster.internal/ubuntu kinetic/main amd64 g++-11 amd64 11.3.0-3ubuntu1 [11.3 MB] Get:72 http://ftpmaster.internal/ubuntu kinetic/main amd64 libstdc++-11-dev amd64 11.3.0-3ubuntu1 [2074 kB] Get:73 http://ftpmaster.internal/ubuntu kinetic/main amd64 libasan6 amd64 11.3.0-3ubuntu1 [2265 kB] Get:74 http://ftpmaster.internal/ubuntu kinetic/main amd64 gcc-11 amd64 11.3.0-3ubuntu1 [20.0 MB] Get:75 http://ftpmaster.internal/ubuntu kinetic/main amd64 libgcc-11-dev amd64 11.3.0-3ubuntu1 [2484 kB] Get:76 http://ftpmaster.internal/ubuntu kinetic/main amd64 libtsan0 amd64 11.3.0-3ubuntu1 [2241 kB] Get:77 http://ftpmaster.internal/ubuntu kinetic/main amd64 cpp-11 amd64 11.3.0-3ubuntu1 [9910 kB] Get:78 http://ftpmaster.internal/ubuntu kinetic/main amd64 gcc-11-base amd64 11.3.0-3ubuntu1 [20.8 kB] Get:79 http://ftpmaster.internal/ubuntu kinetic/main amd64 dpkg-dev all 1.21.8ubuntu1 [1069 kB] Get:80 http://ftpmaster.internal/ubuntu kinetic/main amd64 libdpkg-perl all 1.21.8ubuntu1 [237 kB] Get:81 http://ftpmaster.internal/ubuntu kinetic/main amd64 lto-disabled-list all 27 [12.4 kB] Get:82 http://ftpmaster.internal/ubuntu kinetic/main amd64 libfakeroot amd64 1.29-1ubuntu1 [31.1 kB] Get:83 http://ftpmaster.internal/ubuntu kinetic/main amd64 fakeroot amd64 1.29-1ubuntu1 [60.0 kB] Get:84 http://ftpmaster.internal/ubuntu kinetic/main amd64 pinentry-curses amd64 1.2.0-1ubuntu1 [34.5 kB] Get:85 http://ftpmaster.internal/ubuntu kinetic/main amd64 policyrcd-script-zg2 all 0.1-3.1 [5730 B] debconf: delaying package configuration, since apt-utils is not installed Fetched 79.7 MB in 3s (23.3 MB/s) (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../base-files_12.2ubuntu1_amd64.deb ... Unpacking base-files (12.2ubuntu1) over (12ubuntu5) ... Setting up base-files (12.2ubuntu1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../bsdutils_1%3a2.38-4ubuntu1_amd64.deb ... Unpacking bsdutils (1:2.38-4ubuntu1) over (1:2.37.2-4ubuntu3) ... Setting up bsdutils (1:2.38-4ubuntu1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../libzstd1_1.5.2+dfsg-1_amd64.deb ... Unpacking libzstd1:amd64 (1.5.2+dfsg-1) over (1.4.8+dfsg-3build1) ... Setting up libzstd1:amd64 (1.5.2+dfsg-1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../0-libatomic1_12.1.0-2ubuntu1_amd64.deb ... Unpacking libatomic1:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../1-libubsan1_12.1.0-2ubuntu1_amd64.deb ... Unpacking libubsan1:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../2-libquadmath0_12.1.0-2ubuntu1_amd64.deb ... Unpacking libquadmath0:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../3-liblsan0_12.1.0-2ubuntu1_amd64.deb ... Unpacking liblsan0:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../4-libitm1_12.1.0-2ubuntu1_amd64.deb ... Unpacking libitm1:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../5-libgomp1_12.1.0-2ubuntu1_amd64.deb ... Unpacking libgomp1:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../6-gcc-12-base_12.1.0-2ubuntu1_amd64.deb ... Unpacking gcc-12-base:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Setting up gcc-12-base:amd64 (12.1.0-2ubuntu1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../libgcc-s1_12.1.0-2ubuntu1_amd64.deb ... Unpacking libgcc-s1:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Setting up libgcc-s1:amd64 (12.1.0-2ubuntu1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../libcc1-0_12.1.0-2ubuntu1_amd64.deb ... Unpacking libcc1-0:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Preparing to unpack .../libstdc++6_12.1.0-2ubuntu1_amd64.deb ... Unpacking libstdc++6:amd64 (12.1.0-2ubuntu1) over (12-20220428-1ubuntu1) ... Setting up libstdc++6:amd64 (12.1.0-2ubuntu1) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../libgpg-error0_1.45-2_amd64.deb ... Unpacking libgpg-error0:amd64 (1.45-2) over (1.43-3) ... Setting up libgpg-error0:amd64 (1.45-2) ... (Reading database ... 13449 files and directories currently installed.) Preparing to unpack .../libgcrypt20_1.10.1-2ubuntu1_amd64.deb ... Unpacking libgcrypt20:amd64 (1.10.1-2ubuntu1) over (1.9.4-3ubuntu3) ... Setting up libgcrypt20:amd64 (1.10.1-2ubuntu1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../liblzma5_5.2.5-2.1_amd64.deb ... Unpacking liblzma5:amd64 (5.2.5-2.1) over (5.2.5-2ubuntu1) ... Setting up liblzma5:amd64 (5.2.5-2.1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../systemd-sysv_249.11-0ubuntu4_amd64.deb ... Unpacking systemd-sysv (249.11-0ubuntu4) over (249.11-0ubuntu3.1) ... Preparing to unpack .../libapparmor1_3.0.4-2ubuntu3_amd64.deb ... Unpacking libapparmor1:amd64 (3.0.4-2ubuntu3) over (3.0.4-2ubuntu2) ... Preparing to unpack .../libgnutls30_3.7.6-2ubuntu1_amd64.deb ... Unpacking libgnutls30:amd64 (3.7.6-2ubuntu1) over (3.7.4-2ubuntu1) ... Setting up libgnutls30:amd64 (3.7.6-2ubuntu1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../libip4tc2_1.8.7-1ubuntu6_amd64.deb ... Unpacking libip4tc2:amd64 (1.8.7-1ubuntu6) over (1.8.7-1ubuntu5) ... Preparing to unpack .../libblkid1_2.38-4ubuntu1_amd64.deb ... Unpacking libblkid1:amd64 (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... Setting up libblkid1:amd64 (2.38-4ubuntu1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../libselinux1_3.4-1_amd64.deb ... Unpacking libselinux1:amd64 (3.4-1) over (3.3-1build2) ... Setting up libselinux1:amd64 (3.4-1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../libmount1_2.38-4ubuntu1_amd64.deb ... Unpacking libmount1:amd64 (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... Setting up libmount1:amd64 (2.38-4ubuntu1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../libsmartcols1_2.38-4ubuntu1_amd64.deb ... Unpacking libsmartcols1:amd64 (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... Setting up libsmartcols1:amd64 (2.38-4ubuntu1) ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../login_1%3a4.11.1+dfsg1-2ubuntu1_amd64.deb ... Unpacking login (1:4.11.1+dfsg1-2ubuntu1) over (1:4.8.1-2ubuntu2) ... Setting up login (1:4.11.1+dfsg1-2ubuntu1) ... Installing new version of config file /etc/login.defs ... (Reading database ... 13451 files and directories currently installed.) Preparing to unpack .../util-linux_2.38-4ubuntu1_amd64.deb ... Unpacking util-linux (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... dpkg: warning: unable to delete old directory '/usr/lib/udev': Directory not empty Selecting previously unselected package util-linux-extra. Preparing to unpack .../util-linux-extra_2.38-4ubuntu1_amd64.deb ... Unpacking util-linux-extra (2.38-4ubuntu1) ... Setting up util-linux-extra (2.38-4ubuntu1) ... (Reading database ... 13463 files and directories currently installed.) Preparing to unpack .../mount_2.38-4ubuntu1_amd64.deb ... Unpacking mount (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... Preparing to unpack .../libsemanage-common_3.4-1_all.deb ... Unpacking libsemanage-common (3.4-1) over (3.3-1build2) ... Setting up libsemanage-common (3.4-1) ... Installing new version of config file /etc/selinux/semanage.conf ... (Reading database ... 13463 files and directories currently installed.) Preparing to unpack .../libsepol2_3.4-2_amd64.deb ... Unpacking libsepol2:amd64 (3.4-2) over (3.3-1build1) ... Setting up libsepol2:amd64 (3.4-2) ... (Reading database ... 13463 files and directories currently installed.) Preparing to unpack .../libsemanage2_3.4-1_amd64.deb ... Unpacking libsemanage2:amd64 (3.4-1) over (3.3-1build2) ... Setting up libsemanage2:amd64 (3.4-1) ... (Reading database ... 13463 files and directories currently installed.) Preparing to unpack .../passwd_1%3a4.11.1+dfsg1-2ubuntu1_amd64.deb ... Unpacking passwd (1:4.11.1+dfsg1-2ubuntu1) over (1:4.8.1-2ubuntu2) ... Setting up passwd (1:4.11.1+dfsg1-2ubuntu1) ... Installing new version of config file /etc/default/useradd ... (Reading database ... 13463 files and directories currently installed.) Preparing to unpack .../adduser_3.121ubuntu1_all.deb ... Unpacking adduser (3.121ubuntu1) over (3.118ubuntu5) ... Setting up adduser (3.121ubuntu1) ... Installing new version of config file /etc/deluser.conf ... (Reading database ... 13460 files and directories currently installed.) Preparing to unpack .../systemd_249.11-0ubuntu4_amd64.deb ... Unpacking systemd (249.11-0ubuntu4) over (249.11-0ubuntu3.1) ... Preparing to unpack .../libsystemd0_249.11-0ubuntu4_amd64.deb ... Unpacking libsystemd0:amd64 (249.11-0ubuntu4) over (249.11-0ubuntu3.1) ... Setting up libsystemd0:amd64 (249.11-0ubuntu4) ... (Reading database ... 13460 files and directories currently installed.) Preparing to unpack .../libudev1_249.11-0ubuntu4_amd64.deb ... Unpacking libudev1:amd64 (249.11-0ubuntu4) over (249.11-0ubuntu3.1) ... Setting up libudev1:amd64 (249.11-0ubuntu4) ... (Reading database ... 13460 files and directories currently installed.) Preparing to unpack .../libapt-pkg6.0_2.5.0_amd64.deb ... Unpacking libapt-pkg6.0:amd64 (2.5.0) over (2.4.5) ... Setting up libapt-pkg6.0:amd64 (2.5.0) ... (Reading database ... 13460 files and directories currently installed.) Preparing to unpack .../dpkg_1.21.8ubuntu1_amd64.deb ... Unpacking dpkg (1.21.8ubuntu1) over (1.21.1ubuntu2) ... Setting up dpkg (1.21.8ubuntu1) ... (Reading database ... 13464 files and directories currently installed.) Preparing to unpack .../ncurses-bin_6.3+20220423-2_amd64.deb ... Unpacking ncurses-bin (6.3+20220423-2) over (6.3-2) ... Setting up ncurses-bin (6.3+20220423-2) ... (Reading database ... 13464 files and directories currently installed.) Preparing to unpack .../init-system-helpers_1.63_all.deb ... Unpacking init-system-helpers (1.63) over (1.62) ... Setting up init-system-helpers (1.63) ... (Reading database ... 13464 files and directories currently installed.) Preparing to unpack .../ncurses-base_6.3+20220423-2_all.deb ... Unpacking ncurses-base (6.3+20220423-2) over (6.3-2) ... Setting up ncurses-base (6.3+20220423-2) ... (Reading database ... 13465 files and directories currently installed.) Preparing to unpack .../archives/apt_2.5.0_amd64.deb ... Unpacking apt (2.5.0) over (2.4.5) ... Setting up apt (2.5.0) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../logsave_1.46.5-2ubuntu2_amd64.deb ... Unpacking logsave (1.46.5-2ubuntu2) over (1.46.5-2ubuntu1) ... Preparing to unpack .../libext2fs2_1.46.5-2ubuntu2_amd64.deb ... Unpacking libext2fs2:amd64 (1.46.5-2ubuntu2) over (1.46.5-2ubuntu1) ... Setting up libext2fs2:amd64 (1.46.5-2ubuntu2) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../e2fsprogs_1.46.5-2ubuntu2_amd64.deb ... Unpacking e2fsprogs (1.46.5-2ubuntu2) over (1.46.5-2ubuntu1) ... Setting up libapparmor1:amd64 (3.0.4-2ubuntu3) ... Setting up libip4tc2:amd64 (1.8.7-1ubuntu6) ... Setting up util-linux (2.38-4ubuntu1) ... Setting up mount (2.38-4ubuntu1) ... Setting up systemd (249.11-0ubuntu4) ... Initializing machine ID from random generator. Setting up systemd-sysv (249.11-0ubuntu4) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../archives/init_1.63_amd64.deb ... Unpacking init (1.63) over (1.62) ... Preparing to unpack .../libcap-ng0_0.7.9-4_amd64.deb ... Unpacking libcap-ng0:amd64 (0.7.9-4) over (0.7.9-2.2build3) ... Setting up libcap-ng0:amd64 (0.7.9-4) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../libdb5.3_5.3.28+dfsg1-0.9_amd64.deb ... Unpacking libdb5.3:amd64 (5.3.28+dfsg1-0.9) over (5.3.28+dfsg1-0.8ubuntu3) ... Setting up libdb5.3:amd64 (5.3.28+dfsg1-0.9) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../libncurses6_6.3+20220423-2_amd64.deb ... Unpacking libncurses6:amd64 (6.3+20220423-2) over (6.3-2) ... Preparing to unpack .../libncursesw6_6.3+20220423-2_amd64.deb ... Unpacking libncursesw6:amd64 (6.3+20220423-2) over (6.3-2) ... Preparing to unpack .../libtinfo6_6.3+20220423-2_amd64.deb ... Unpacking libtinfo6:amd64 (6.3+20220423-2) over (6.3-2) ... Setting up libtinfo6:amd64 (6.3+20220423-2) ... (Reading database ... 13465 files and directories currently installed.) Preparing to unpack .../libuuid1_2.38-4ubuntu1_amd64.deb ... Unpacking libuuid1:amd64 (2.38-4ubuntu1) over (2.37.2-4ubuntu3) ... Setting up libuuid1:amd64 (2.38-4ubuntu1) ... (Reading database ... 13466 files and directories currently installed.) Preparing to unpack .../00-libcom-err2_1.46.5-2ubuntu2_amd64.deb ... Unpacking libcom-err2:amd64 (1.46.5-2ubuntu2) over (1.46.5-2ubuntu1) ... Preparing to unpack .../01-libss2_1.46.5-2ubuntu2_amd64.deb ... Unpacking libss2:amd64 (1.46.5-2ubuntu2) over (1.46.5-2ubuntu1) ... Preparing to unpack .../02-mawk_1.3.4.20200120-3.1_amd64.deb ... Unpacking mawk (1.3.4.20200120-3.1) over (1.3.4.20200120-3) ... Preparing to unpack .../03-libkeyutils1_1.6.1-3ubuntu1_amd64.deb ... Unpacking libkeyutils1:amd64 (1.6.1-3ubuntu1) over (1.6.1-2ubuntu3) ... Preparing to unpack .../04-readline-common_8.1.2-1.2_all.deb ... Unpacking readline-common (8.1.2-1.2) over (8.1.2-1) ... Preparing to unpack .../05-libreadline8_8.1.2-1.2_amd64.deb ... Unpacking libreadline8:amd64 (8.1.2-1.2) over (8.1.2-1) ... Preparing to unpack .../06-libsqlite3-0_3.38.5-1_amd64.deb ... Unpacking libsqlite3-0:amd64 (3.38.5-1) over (3.37.2-2) ... Preparing to unpack .../07-libssl3_3.0.3-5ubuntu3_amd64.deb ... Unpacking libssl3:amd64 (3.0.3-5ubuntu3) over (3.0.2-0ubuntu1) ... Preparing to unpack .../08-openssl_3.0.3-5ubuntu3_amd64.deb ... Unpacking openssl (3.0.3-5ubuntu3) over (3.0.2-0ubuntu1) ... Preparing to unpack .../09-libpng16-16_1.6.37-5_amd64.deb ... Unpacking libpng16-16:amd64 (1.6.37-5) over (1.6.37-3build5) ... Preparing to unpack .../10-xz-utils_5.2.5-2.1_amd64.deb ... Unpacking xz-utils (5.2.5-2.1) over (5.2.5-2ubuntu1) ... Preparing to unpack .../11-libctf0_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking libctf0:amd64 (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Preparing to unpack .../12-libctf-nobfd0_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking libctf-nobfd0:amd64 (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Preparing to unpack .../13-binutils-x86-64-linux-gnu_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking binutils-x86-64-linux-gnu (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Preparing to unpack .../14-libbinutils_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking libbinutils:amd64 (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Preparing to unpack .../15-binutils_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking binutils (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Preparing to unpack .../16-binutils-common_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking binutils-common:amd64 (2.38.50.20220615-4ubuntu1) over (2.38-3ubuntu1) ... Selecting previously unselected package libgprofng0:amd64. Preparing to unpack .../17-libgprofng0_2.38.50.20220615-4ubuntu1_amd64.deb ... Unpacking libgprofng0:amd64 (2.38.50.20220615-4ubuntu1) ... Preparing to unpack .../18-g++-11_11.3.0-3ubuntu1_amd64.deb ... Unpacking g++-11 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../19-libstdc++-11-dev_11.3.0-3ubuntu1_amd64.deb ... Unpacking libstdc++-11-dev:amd64 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../20-libasan6_11.3.0-3ubuntu1_amd64.deb ... Unpacking libasan6:amd64 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../21-gcc-11_11.3.0-3ubuntu1_amd64.deb ... Unpacking gcc-11 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../22-libgcc-11-dev_11.3.0-3ubuntu1_amd64.deb ... Unpacking libgcc-11-dev:amd64 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../23-libtsan0_11.3.0-3ubuntu1_amd64.deb ... Unpacking libtsan0:amd64 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../24-cpp-11_11.3.0-3ubuntu1_amd64.deb ... Unpacking cpp-11 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../25-gcc-11-base_11.3.0-3ubuntu1_amd64.deb ... Unpacking gcc-11-base:amd64 (11.3.0-3ubuntu1) over (11.3.0-1ubuntu1) ... Preparing to unpack .../26-dpkg-dev_1.21.8ubuntu1_all.deb ... Unpacking dpkg-dev (1.21.8ubuntu1) over (1.21.1ubuntu2) ... Preparing to unpack .../27-libdpkg-perl_1.21.8ubuntu1_all.deb ... Unpacking libdpkg-perl (1.21.8ubuntu1) over (1.21.1ubuntu2) ... Preparing to unpack .../28-lto-disabled-list_27_all.deb ... Unpacking lto-disabled-list (27) over (25) ... Preparing to unpack .../29-libfakeroot_1.29-1ubuntu1_amd64.deb ... Unpacking libfakeroot:amd64 (1.29-1ubuntu1) over (1.28-1ubuntu1) ... Preparing to unpack .../30-fakeroot_1.29-1ubuntu1_amd64.deb ... Unpacking fakeroot (1.29-1ubuntu1) over (1.28-1ubuntu1) ... Preparing to unpack .../31-pinentry-curses_1.2.0-1ubuntu1_amd64.deb ... Unpacking pinentry-curses (1.2.0-1ubuntu1) over (1.1.1-1build2) ... Preparing to unpack .../32-policyrcd-script-zg2_0.1-3.1_all.deb ... Unpacking policyrcd-script-zg2 (0.1-3.1) over (0.1-3) ... Setting up gcc-11-base:amd64 (11.3.0-3ubuntu1) ... Setting up libkeyutils1:amd64 (1.6.1-3ubuntu1) ... Setting up lto-disabled-list (27) ... Setting up init (1.63) ... Setting up libsqlite3-0:amd64 (3.38.5-1) ... Setting up binutils-common:amd64 (2.38.50.20220615-4ubuntu1) ... Setting up libssl3:amd64 (3.0.3-5ubuntu3) ... Setting up libctf-nobfd0:amd64 (2.38.50.20220615-4ubuntu1) ... Setting up libcom-err2:amd64 (1.46.5-2ubuntu2) ... Setting up libgomp1:amd64 (12.1.0-2ubuntu1) ... Setting up libfakeroot:amd64 (1.29-1ubuntu1) ... Setting up libasan6:amd64 (11.3.0-3ubuntu1) ... Setting up fakeroot (1.29-1ubuntu1) ... Setting up policyrcd-script-zg2 (0.1-3.1) ... Setting up libncurses6:amd64 (6.3+20220423-2) ... Setting up xz-utils (5.2.5-2.1) ... Setting up libquadmath0:amd64 (12.1.0-2ubuntu1) ... Setting up libpng16-16:amd64 (1.6.37-5) ... Setting up libatomic1:amd64 (12.1.0-2ubuntu1) ... Setting up libss2:amd64 (1.46.5-2ubuntu2) ... Setting up libncursesw6:amd64 (6.3+20220423-2) ... Setting up libdpkg-perl (1.21.8ubuntu1) ... Setting up logsave (1.46.5-2ubuntu2) ... Setting up libubsan1:amd64 (12.1.0-2ubuntu1) ... Setting up mawk (1.3.4.20200120-3.1) ... Setting up libbinutils:amd64 (2.38.50.20220615-4ubuntu1) ... Setting up openssl (3.0.3-5ubuntu3) ... Installing new version of config file /etc/ssl/openssl.cnf ... Setting up readline-common (8.1.2-1.2) ... Setting up libcc1-0:amd64 (12.1.0-2ubuntu1) ... Setting up liblsan0:amd64 (12.1.0-2ubuntu1) ... Setting up libitm1:amd64 (12.1.0-2ubuntu1) ... Setting up libtsan0:amd64 (11.3.0-3ubuntu1) ... Setting up libctf0:amd64 (2.38.50.20220615-4ubuntu1) ... Setting up pinentry-curses (1.2.0-1ubuntu1) ... Setting up cpp-11 (11.3.0-3ubuntu1) ... Setting up libreadline8:amd64 (8.1.2-1.2) ... Setting up e2fsprogs (1.46.5-2ubuntu2) ... Setting up libgprofng0:amd64 (2.38.50.20220615-4ubuntu1) ... Setting up libgcc-11-dev:amd64 (11.3.0-3ubuntu1) ... Setting up binutils-x86-64-linux-gnu (2.38.50.20220615-4ubuntu1) ... Setting up binutils (2.38.50.20220615-4ubuntu1) ... Setting up dpkg-dev (1.21.8ubuntu1) ... Setting up libstdc++-11-dev:amd64 (11.3.0-3ubuntu1) ... Setting up gcc-11 (11.3.0-3ubuntu1) ... Setting up g++-11 (11.3.0-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3) ... RUN: /usr/share/launchpad-buildd/bin/sbuild-package PACKAGEBUILD-24000232 amd64 kinetic -c chroot:build-PACKAGEBUILD-24000232 --arch=amd64 --dist=kinetic --nolog -A golang-github-hashicorp-raft_1.1.2-1.dsc Initiating build PACKAGEBUILD-24000232 with 4 jobs across 4 processor cores. Kernel reported to sbuild: 5.4.0-120-generic #136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022 x86_64 sbuild (Debian sbuild) 0.79.0 (05 February 2020) on lgw01-amd64-015.buildd +==============================================================================+ | golang-github-hashicorp-raft 1.1.2-1 (amd64) Wed, 22 Jun 2022 14:52:19 +0000 | +==============================================================================+ Package: golang-github-hashicorp-raft Version: 1.1.2-1 Source Version: 1.1.2-1 Distribution: kinetic Machine Architecture: amd64 Host Architecture: amd64 Build Architecture: amd64 Build Type: binary I: NOTICE: Log filtering will replace 'home/buildd/build-PACKAGEBUILD-24000232/chroot-autobuild' with '<>' I: NOTICE: Log filtering will replace 'build/golang-github-hashicorp-raft-A9OwIg/resolver-ssKwKf' with '<>' +------------------------------------------------------------------------------+ | Fetch source files | +------------------------------------------------------------------------------+ Local sources ------------- golang-github-hashicorp-raft_1.1.2-1.dsc exists in .; copying to chroot I: NOTICE: Log filtering will replace 'build/golang-github-hashicorp-raft-A9OwIg/golang-github-hashicorp-raft-1.1.2' with '<>' I: NOTICE: Log filtering will replace 'build/golang-github-hashicorp-raft-A9OwIg' with '<>' +------------------------------------------------------------------------------+ | Install package build dependencies | +------------------------------------------------------------------------------+ Setup apt archive ----------------- Merged Build-Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~), build-essential, fakeroot Filtered Build-Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~), build-essential, fakeroot dpkg-deb: building package 'sbuild-build-depends-main-dummy' in '/<>/apt_archive/sbuild-build-depends-main-dummy.deb'. Ign:1 copy:/<>/apt_archive ./ InRelease Get:2 copy:/<>/apt_archive ./ Release [957 B] Ign:3 copy:/<>/apt_archive ./ Release.gpg Get:4 copy:/<>/apt_archive ./ Sources [436 B] Get:5 copy:/<>/apt_archive ./ Packages [519 B] Fetched 1912 B in 0s (139 kB/s) Reading package lists... Reading package lists... Install main build dependencies (apt-based resolver) ---------------------------------------------------- Installing build dependencies Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: autoconf automake autopoint autotools-dev bsdextrautils debhelper debugedit dh-autoreconf dh-golang dh-strip-nondeterminism dwz file gettext gettext-base golang-1.18-go golang-1.18-src golang-any golang-github-alecthomas-units-dev golang-github-armon-go-metrics-dev golang-github-aws-aws-sdk-go-dev golang-github-beorn7-perks-dev golang-github-cespare-xxhash-dev golang-github-circonus-labs-circonus-gometrics-dev golang-github-circonus-labs-circonusllhist-dev golang-github-creack-pty-dev golang-github-datadog-datadog-go-dev golang-github-davecgh-go-spew-dev golang-github-fatih-color-dev golang-github-go-kit-log-dev golang-github-go-logfmt-logfmt-dev golang-github-hashicorp-go-cleanhttp-dev golang-github-hashicorp-go-hclog-dev golang-github-hashicorp-go-immutable-radix-dev golang-github-hashicorp-go-msgpack-dev golang-github-hashicorp-go-retryablehttp-dev golang-github-hashicorp-go-uuid-dev golang-github-hashicorp-golang-lru-dev golang-github-jmespath-go-jmespath-dev golang-github-jpillora-backoff-dev golang-github-json-iterator-go-dev golang-github-julienschmidt-httprouter-dev golang-github-kr-pretty-dev golang-github-kr-text-dev golang-github-mattn-go-colorable-dev golang-github-mattn-go-isatty-dev golang-github-modern-go-concurrent-dev golang-github-modern-go-reflect2-dev golang-github-mwitkow-go-conntrack-dev golang-github-pascaldekloe-goe-dev golang-github-pkg-diff-dev golang-github-pkg-errors-dev golang-github-pmezard-go-difflib-dev golang-github-prometheus-client-golang-dev golang-github-prometheus-client-model-dev golang-github-prometheus-common-dev golang-github-prometheus-procfs-dev golang-github-rogpeppe-go-internal-dev golang-github-stretchr-objx-dev golang-github-stretchr-testify-dev golang-github-tv42-httpunix-dev golang-github-vmihailenco-tagparser-dev golang-go golang-golang-x-net-dev golang-golang-x-oauth2-dev golang-golang-x-sync-dev golang-golang-x-sys-dev golang-golang-x-text-dev golang-gopkg-alecthomas-kingpin.v2-dev golang-gopkg-check.v1-dev golang-gopkg-errgo.v2-dev golang-gopkg-mgo.v2-dev golang-gopkg-tomb.v2-dev golang-gopkg-vmihailenco-msgpack.v2-dev golang-gopkg-yaml.v2-dev golang-gopkg-yaml.v3-dev golang-goprotobuf-dev golang-protobuf-extensions-dev golang-src groff-base intltool-debian libarchive-zip-perl libdebhelper-perl libdw1 libelf1 libfile-stripnondeterminism-perl libicu71 libmagic-mgc libmagic1 libpipeline1 libprotobuf-dev libprotobuf-lite23 libprotobuf23 libprotoc23 libsasl2-2 libsasl2-dev libsasl2-modules-db libsigsegv2 libsub-override-perl libtool libuchardet0 libxml2 m4 man-db po-debconf protobuf-compiler zlib1g-dev Suggested packages: autoconf-archive gnu-standards autoconf-doc dh-make gettext-doc libasprintf-dev libgettextpo-dev bzr | brz git mercurial subversion groff libtool-doc gfortran | fortran95-compiler gcj-jdk m4-doc apparmor less www-browser libmail-box-perl protobuf-mode-el Recommended packages: curl | wget | lynx pkg-config libarchive-cpio-perl libsasl2-modules libltdl-dev libmail-sendmail-perl The following NEW packages will be installed: autoconf automake autopoint autotools-dev bsdextrautils debhelper debugedit dh-autoreconf dh-golang dh-strip-nondeterminism dwz file gettext gettext-base golang-1.18-go golang-1.18-src golang-any golang-github-alecthomas-units-dev golang-github-armon-go-metrics-dev golang-github-aws-aws-sdk-go-dev golang-github-beorn7-perks-dev golang-github-cespare-xxhash-dev golang-github-circonus-labs-circonus-gometrics-dev golang-github-circonus-labs-circonusllhist-dev golang-github-creack-pty-dev golang-github-datadog-datadog-go-dev golang-github-davecgh-go-spew-dev golang-github-fatih-color-dev golang-github-go-kit-log-dev golang-github-go-logfmt-logfmt-dev golang-github-hashicorp-go-cleanhttp-dev golang-github-hashicorp-go-hclog-dev golang-github-hashicorp-go-immutable-radix-dev golang-github-hashicorp-go-msgpack-dev golang-github-hashicorp-go-retryablehttp-dev golang-github-hashicorp-go-uuid-dev golang-github-hashicorp-golang-lru-dev golang-github-jmespath-go-jmespath-dev golang-github-jpillora-backoff-dev golang-github-json-iterator-go-dev golang-github-julienschmidt-httprouter-dev golang-github-kr-pretty-dev golang-github-kr-text-dev golang-github-mattn-go-colorable-dev golang-github-mattn-go-isatty-dev golang-github-modern-go-concurrent-dev golang-github-modern-go-reflect2-dev golang-github-mwitkow-go-conntrack-dev golang-github-pascaldekloe-goe-dev golang-github-pkg-diff-dev golang-github-pkg-errors-dev golang-github-pmezard-go-difflib-dev golang-github-prometheus-client-golang-dev golang-github-prometheus-client-model-dev golang-github-prometheus-common-dev golang-github-prometheus-procfs-dev golang-github-rogpeppe-go-internal-dev golang-github-stretchr-objx-dev golang-github-stretchr-testify-dev golang-github-tv42-httpunix-dev golang-github-vmihailenco-tagparser-dev golang-go golang-golang-x-net-dev golang-golang-x-oauth2-dev golang-golang-x-sync-dev golang-golang-x-sys-dev golang-golang-x-text-dev golang-gopkg-alecthomas-kingpin.v2-dev golang-gopkg-check.v1-dev golang-gopkg-errgo.v2-dev golang-gopkg-mgo.v2-dev golang-gopkg-tomb.v2-dev golang-gopkg-vmihailenco-msgpack.v2-dev golang-gopkg-yaml.v2-dev golang-gopkg-yaml.v3-dev golang-goprotobuf-dev golang-protobuf-extensions-dev golang-src groff-base intltool-debian libarchive-zip-perl libdebhelper-perl libdw1 libelf1 libfile-stripnondeterminism-perl libicu71 libmagic-mgc libmagic1 libpipeline1 libprotobuf-dev libprotobuf-lite23 libprotobuf23 libprotoc23 libsasl2-2 libsasl2-dev libsasl2-modules-db libsigsegv2 libsub-override-perl libtool libuchardet0 libxml2 m4 man-db po-debconf protobuf-compiler sbuild-build-depends-main-dummy zlib1g-dev 0 upgraded, 107 newly installed, 0 to remove and 0 not upgraded. Need to get 121 MB of archives. After this operation, 739 MB of additional disk space will be used. Get:1 copy:/<>/apt_archive ./ sbuild-build-depends-main-dummy 0.invalid.0 [736 B] Get:2 http://ftpmaster.internal/ubuntu kinetic/main amd64 libelf1 amd64 0.187-1 [51.3 kB] Get:3 http://ftpmaster.internal/ubuntu kinetic/main amd64 libicu71 amd64 71.1-3 [10.6 MB] Get:4 http://ftpmaster.internal/ubuntu kinetic/main amd64 libxml2 amd64 2.9.14+dfsg-1 [756 kB] Get:5 http://ftpmaster.internal/ubuntu kinetic/main amd64 bsdextrautils amd64 2.38-4ubuntu1 [71.1 kB] Get:6 http://ftpmaster.internal/ubuntu kinetic/main amd64 libmagic-mgc amd64 1:5.41-4 [257 kB] Get:7 http://ftpmaster.internal/ubuntu kinetic/main amd64 libmagic1 amd64 1:5.41-4 [87.3 kB] Get:8 http://ftpmaster.internal/ubuntu kinetic/main amd64 file amd64 1:5.41-4 [21.5 kB] Get:9 http://ftpmaster.internal/ubuntu kinetic/main amd64 gettext-base amd64 0.21-6 [37.9 kB] Get:10 http://ftpmaster.internal/ubuntu kinetic/main amd64 libuchardet0 amd64 0.0.7-1build2 [76.9 kB] Get:11 http://ftpmaster.internal/ubuntu kinetic/main amd64 groff-base amd64 1.22.4-8build1 [955 kB] Get:12 http://ftpmaster.internal/ubuntu kinetic/main amd64 libpipeline1 amd64 1.5.6-1 [23.1 kB] Get:13 http://ftpmaster.internal/ubuntu kinetic/main amd64 man-db amd64 2.10.2-1 [1187 kB] Get:14 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsigsegv2 amd64 2.13-1ubuntu3 [14.6 kB] Get:15 http://ftpmaster.internal/ubuntu kinetic/main amd64 m4 amd64 1.4.18-5ubuntu2 [199 kB] Get:16 http://ftpmaster.internal/ubuntu kinetic/main amd64 autoconf all 2.71-2 [338 kB] Get:17 http://ftpmaster.internal/ubuntu kinetic/main amd64 autotools-dev all 20220109.1 [44.9 kB] Get:18 http://ftpmaster.internal/ubuntu kinetic/main amd64 automake all 1:1.16.5-1.3 [558 kB] Get:19 http://ftpmaster.internal/ubuntu kinetic/main amd64 autopoint all 0.21-6 [421 kB] Get:20 http://ftpmaster.internal/ubuntu kinetic/main amd64 libdebhelper-perl all 13.7.1ubuntu1 [66.9 kB] Get:21 http://ftpmaster.internal/ubuntu kinetic/main amd64 libtool all 2.4.7-4 [166 kB] Get:22 http://ftpmaster.internal/ubuntu kinetic/main amd64 dh-autoreconf all 20 [16.1 kB] Get:23 http://ftpmaster.internal/ubuntu kinetic/main amd64 libarchive-zip-perl all 1.68-1 [90.2 kB] Get:24 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsub-override-perl all 0.09-3 [9756 B] Get:25 http://ftpmaster.internal/ubuntu kinetic/main amd64 libfile-stripnondeterminism-perl all 1.13.0-1 [18.1 kB] Get:26 http://ftpmaster.internal/ubuntu kinetic/main amd64 dh-strip-nondeterminism all 1.13.0-1 [5344 B] Get:27 http://ftpmaster.internal/ubuntu kinetic/main amd64 libdw1 amd64 0.187-1 [247 kB] Get:28 http://ftpmaster.internal/ubuntu kinetic/main amd64 debugedit amd64 1:5.0-4build1 [47.2 kB] Get:29 http://ftpmaster.internal/ubuntu kinetic/main amd64 dwz amd64 0.14-1build2 [105 kB] Get:30 http://ftpmaster.internal/ubuntu kinetic/main amd64 gettext amd64 0.21-6 [862 kB] Get:31 http://ftpmaster.internal/ubuntu kinetic/main amd64 intltool-debian all 0.35.0+20060710.5 [24.9 kB] Get:32 http://ftpmaster.internal/ubuntu kinetic/main amd64 po-debconf all 1.0.21+nmu1 [233 kB] Get:33 http://ftpmaster.internal/ubuntu kinetic/main amd64 debhelper all 13.7.1ubuntu1 [940 kB] Get:34 http://ftpmaster.internal/ubuntu kinetic/main amd64 golang-1.18-src all 1.18.3-1ubuntu1 [16.2 MB] Get:35 http://ftpmaster.internal/ubuntu kinetic/main amd64 golang-1.18-go amd64 1.18.3-1ubuntu1 [65.2 MB] Get:36 http://ftpmaster.internal/ubuntu kinetic/main amd64 golang-src all 2:1.18~0ubuntu2 [4438 B] Get:37 http://ftpmaster.internal/ubuntu kinetic/main amd64 golang-go amd64 2:1.18~0ubuntu2 [41.8 kB] Get:38 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-any amd64 2:1.18~0ubuntu2 [2850 B] Get:39 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-creack-pty-dev all 1.1.18-1 [10.0 kB] Get:40 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-go-logfmt-logfmt-dev all 0.5.0-2 [12.8 kB] Get:41 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-go-kit-log-dev all 0.2.0-3 [26.3 kB] Get:42 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-golang-x-sys-dev all 0.0~git20220520.bc2c85a-1 [404 kB] Get:43 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-mattn-go-isatty-dev all 0.0.14-1 [6200 B] Get:44 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-mattn-go-colorable-dev all 0.1.12-1 [10.7 kB] Get:45 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-fatih-color-dev all 1.7.0-1 [11.4 kB] Get:46 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-davecgh-go-spew-dev all 1.1.1-2 [29.7 kB] Get:47 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-pmezard-go-difflib-dev all 1.0.0-3 [12.4 kB] Get:48 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-stretchr-objx-dev all 0.3.0-1 [25.4 kB] Get:49 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-yaml.v3-dev all 3.0.1-1 [75.9 kB] Get:50 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-stretchr-testify-dev all 1.7.2-1 [66.2 kB] Get:51 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-hclog-dev all 0.11.0-1 [18.0 kB] Get:52 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-jpillora-backoff-dev all 1.0.0-1.1 [3744 B] Get:53 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-modern-go-concurrent-dev all 1.0.3-1.1 [4708 B] Get:54 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-modern-go-reflect2-dev all 1.0.2-2 [11.0 kB] Get:55 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-json-iterator-go-dev all 1.1.12-1 [63.9 kB] Get:56 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-beorn7-perks-dev all 1.0.1-1 [12.2 kB] Get:57 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-cespare-xxhash-dev all 2.1.1-2 [9052 B] Get:58 http://ftpmaster.internal/ubuntu kinetic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-2ubuntu9 [164 kB] Get:59 http://ftpmaster.internal/ubuntu kinetic/main amd64 libprotobuf23 amd64 3.12.4-1ubuntu7 [876 kB] Get:60 http://ftpmaster.internal/ubuntu kinetic/main amd64 libprotobuf-lite23 amd64 3.12.4-1ubuntu7 [208 kB] Get:61 http://ftpmaster.internal/ubuntu kinetic/main amd64 libprotobuf-dev amd64 3.12.4-1ubuntu7 [1346 kB] Get:62 http://ftpmaster.internal/ubuntu kinetic/main amd64 libprotoc23 amd64 3.12.4-1ubuntu7 [663 kB] Get:63 http://ftpmaster.internal/ubuntu kinetic/universe amd64 protobuf-compiler amd64 3.12.4-1ubuntu7 [29.2 kB] Get:64 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-goprotobuf-dev amd64 1.3.4-2 [1628 kB] Get:65 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-prometheus-client-model-dev all 0.2.0-3 [8812 B] Get:66 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-yaml.v2-dev all 2.4.0-3 [62.6 kB] Get:67 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-jmespath-go-jmespath-dev all 0.4.0-2 [93.9 kB] Get:68 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-pkg-errors-dev all 0.9.1-2 [13.4 kB] Get:69 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-golang-x-text-dev all 0.3.7-1 [4555 kB] Get:70 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-golang-x-net-dev all 1:0.0+git20211209.491a49a+dfsg-1 [702 kB] Get:71 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-aws-aws-sdk-go-dev all 1.41.14-1ubuntu1 [8574 kB] Get:72 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-julienschmidt-httprouter-dev all 1.3.0-1 [19.4 kB] Get:73 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-golang-x-oauth2-dev all 0.0~git20211104.d3ed0bb-1 [46.3 kB] Get:74 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-alecthomas-units-dev all 0.0~git20211218.b94a6e3-1 [8966 B] Get:75 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-alecthomas-kingpin.v2-dev all 2.2.6-3 [42.9 kB] Get:76 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-protobuf-extensions-dev all 1.0.1-1 [29.6 kB] Get:77 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-prometheus-common-dev all 0.32.1-8 [106 kB] Get:78 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-golang-x-sync-dev all 0.0~git20210220.036812b-1 [19.2 kB] Get:79 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-prometheus-procfs-dev all 0.7.3-2 [116 kB] Get:80 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-prometheus-client-golang-dev all 1.11.1-1 [119 kB] Get:81 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-mwitkow-go-conntrack-dev all 0.0~git20190716.2f06839-2 [14.7 kB] Get:82 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-pascaldekloe-goe-dev all 0.1.0-3 [22.4 kB] Get:83 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-pkg-diff-dev all 0.0~git20210226.20ebb0f-1 [92.2 kB] Get:84 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-kr-text-dev all 0.2.0-1 [11.1 kB] Get:85 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-kr-pretty-dev all 0.3.0-1 [15.3 kB] Get:86 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-check.v1-dev all 0.0+git20200902.038fdea-1 [31.7 kB] Get:87 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-errgo.v2-dev all 2.1.0-2 [9024 B] Get:88 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-rogpeppe-go-internal-dev all 1.8.1-1 [118 kB] Get:89 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-tv42-httpunix-dev all 0.0~git20150427.b75d861-2 [3784 B] Get:90 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-vmihailenco-tagparser-dev all 0.1.1-3 [4572 B] Get:91 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg-6ubuntu1 [20.5 kB] Get:92 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsasl2-2 amd64 2.1.28+dfsg-6ubuntu1 [55.9 kB] Get:93 http://ftpmaster.internal/ubuntu kinetic/main amd64 libsasl2-dev amd64 2.1.28+dfsg-6ubuntu1 [245 kB] Get:94 http://ftpmaster.internal/ubuntu kinetic/main amd64 dh-golang all 1.56 [22.1 kB] Get:95 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-circonus-labs-circonusllhist-dev all 0.0~git20191022.ec08cde-1 [11.2 kB] Get:96 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-cleanhttp-dev all 0.5.2-1 [10.9 kB] Get:97 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-retryablehttp-dev all 0.7.0-1 [20.7 kB] Get:98 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-circonus-labs-circonus-gometrics-dev all 2.3.1-3 [64.8 kB] Get:99 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-datadog-datadog-go-dev all 2.1.0-3 [13.5 kB] Get:100 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-uuid-dev all 1.0.2-1 [8812 B] Get:101 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-golang-lru-dev all 0.5.4-2 [14.8 kB] Get:102 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-immutable-radix-dev all 1.3.1-1 [26.9 kB] Get:103 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-armon-go-metrics-dev all 0.3.4-1 [30.0 kB] Get:104 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-vmihailenco-msgpack.v2-dev all 4.3.1-2 [26.6 kB] Get:105 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-tomb.v2-dev all 0.0~git20161208.d5d1b58-3 [6884 B] Get:106 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-gopkg-mgo.v2-dev all 2016.08.01-7 [316 kB] Get:107 http://ftpmaster.internal/ubuntu kinetic/universe amd64 golang-github-hashicorp-go-msgpack-dev all 0.5.5-1 [43.3 kB] debconf: delaying package configuration, since apt-utils is not installed Fetched 121 MB in 3s (34.9 MB/s) Selecting previously unselected package libelf1:amd64. (Reading database ... 13495 files and directories currently installed.) Preparing to unpack .../000-libelf1_0.187-1_amd64.deb ... Unpacking libelf1:amd64 (0.187-1) ... Selecting previously unselected package libicu71:amd64. Preparing to unpack .../001-libicu71_71.1-3_amd64.deb ... Unpacking libicu71:amd64 (71.1-3) ... Selecting previously unselected package libxml2:amd64. Preparing to unpack .../002-libxml2_2.9.14+dfsg-1_amd64.deb ... Unpacking libxml2:amd64 (2.9.14+dfsg-1) ... Selecting previously unselected package bsdextrautils. Preparing to unpack .../003-bsdextrautils_2.38-4ubuntu1_amd64.deb ... Unpacking bsdextrautils (2.38-4ubuntu1) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../004-libmagic-mgc_1%3a5.41-4_amd64.deb ... Unpacking libmagic-mgc (1:5.41-4) ... Selecting previously unselected package libmagic1:amd64. Preparing to unpack .../005-libmagic1_1%3a5.41-4_amd64.deb ... Unpacking libmagic1:amd64 (1:5.41-4) ... Selecting previously unselected package file. Preparing to unpack .../006-file_1%3a5.41-4_amd64.deb ... Unpacking file (1:5.41-4) ... Selecting previously unselected package gettext-base. Preparing to unpack .../007-gettext-base_0.21-6_amd64.deb ... Unpacking gettext-base (0.21-6) ... Selecting previously unselected package libuchardet0:amd64. Preparing to unpack .../008-libuchardet0_0.0.7-1build2_amd64.deb ... Unpacking libuchardet0:amd64 (0.0.7-1build2) ... Selecting previously unselected package groff-base. Preparing to unpack .../009-groff-base_1.22.4-8build1_amd64.deb ... Unpacking groff-base (1.22.4-8build1) ... Selecting previously unselected package libpipeline1:amd64. Preparing to unpack .../010-libpipeline1_1.5.6-1_amd64.deb ... Unpacking libpipeline1:amd64 (1.5.6-1) ... Selecting previously unselected package man-db. Preparing to unpack .../011-man-db_2.10.2-1_amd64.deb ... Unpacking man-db (2.10.2-1) ... Selecting previously unselected package libsigsegv2:amd64. Preparing to unpack .../012-libsigsegv2_2.13-1ubuntu3_amd64.deb ... Unpacking libsigsegv2:amd64 (2.13-1ubuntu3) ... Selecting previously unselected package m4. Preparing to unpack .../013-m4_1.4.18-5ubuntu2_amd64.deb ... Unpacking m4 (1.4.18-5ubuntu2) ... Selecting previously unselected package autoconf. Preparing to unpack .../014-autoconf_2.71-2_all.deb ... Unpacking autoconf (2.71-2) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../015-autotools-dev_20220109.1_all.deb ... Unpacking autotools-dev (20220109.1) ... Selecting previously unselected package automake. Preparing to unpack .../016-automake_1%3a1.16.5-1.3_all.deb ... Unpacking automake (1:1.16.5-1.3) ... Selecting previously unselected package autopoint. Preparing to unpack .../017-autopoint_0.21-6_all.deb ... Unpacking autopoint (0.21-6) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../018-libdebhelper-perl_13.7.1ubuntu1_all.deb ... Unpacking libdebhelper-perl (13.7.1ubuntu1) ... Selecting previously unselected package libtool. Preparing to unpack .../019-libtool_2.4.7-4_all.deb ... Unpacking libtool (2.4.7-4) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../020-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../021-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libsub-override-perl. Preparing to unpack .../022-libsub-override-perl_0.09-3_all.deb ... Unpacking libsub-override-perl (0.09-3) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../023-libfile-stripnondeterminism-perl_1.13.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.13.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../024-dh-strip-nondeterminism_1.13.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.13.0-1) ... Selecting previously unselected package libdw1:amd64. Preparing to unpack .../025-libdw1_0.187-1_amd64.deb ... Unpacking libdw1:amd64 (0.187-1) ... Selecting previously unselected package debugedit. Preparing to unpack .../026-debugedit_1%3a5.0-4build1_amd64.deb ... Unpacking debugedit (1:5.0-4build1) ... Selecting previously unselected package dwz. Preparing to unpack .../027-dwz_0.14-1build2_amd64.deb ... Unpacking dwz (0.14-1build2) ... Selecting previously unselected package gettext. Preparing to unpack .../028-gettext_0.21-6_amd64.deb ... Unpacking gettext (0.21-6) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../029-intltool-debian_0.35.0+20060710.5_all.deb ... Unpacking intltool-debian (0.35.0+20060710.5) ... Selecting previously unselected package po-debconf. Preparing to unpack .../030-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../031-debhelper_13.7.1ubuntu1_all.deb ... Unpacking debhelper (13.7.1ubuntu1) ... Selecting previously unselected package golang-1.18-src. Preparing to unpack .../032-golang-1.18-src_1.18.3-1ubuntu1_all.deb ... Unpacking golang-1.18-src (1.18.3-1ubuntu1) ... Selecting previously unselected package golang-1.18-go. Preparing to unpack .../033-golang-1.18-go_1.18.3-1ubuntu1_amd64.deb ... Unpacking golang-1.18-go (1.18.3-1ubuntu1) ... Selecting previously unselected package golang-src. Preparing to unpack .../034-golang-src_2%3a1.18~0ubuntu2_all.deb ... Unpacking golang-src (2:1.18~0ubuntu2) ... Selecting previously unselected package golang-go:amd64. Preparing to unpack .../035-golang-go_2%3a1.18~0ubuntu2_amd64.deb ... Unpacking golang-go:amd64 (2:1.18~0ubuntu2) ... Selecting previously unselected package golang-any:amd64. Preparing to unpack .../036-golang-any_2%3a1.18~0ubuntu2_amd64.deb ... Unpacking golang-any:amd64 (2:1.18~0ubuntu2) ... Selecting previously unselected package golang-github-creack-pty-dev. Preparing to unpack .../037-golang-github-creack-pty-dev_1.1.18-1_all.deb ... Unpacking golang-github-creack-pty-dev (1.1.18-1) ... Selecting previously unselected package golang-github-go-logfmt-logfmt-dev. Preparing to unpack .../038-golang-github-go-logfmt-logfmt-dev_0.5.0-2_all.deb ... Unpacking golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Selecting previously unselected package golang-github-go-kit-log-dev. Preparing to unpack .../039-golang-github-go-kit-log-dev_0.2.0-3_all.deb ... Unpacking golang-github-go-kit-log-dev (0.2.0-3) ... Selecting previously unselected package golang-golang-x-sys-dev. Preparing to unpack .../040-golang-golang-x-sys-dev_0.0~git20220520.bc2c85a-1_all.deb ... Unpacking golang-golang-x-sys-dev (0.0~git20220520.bc2c85a-1) ... Selecting previously unselected package golang-github-mattn-go-isatty-dev. Preparing to unpack .../041-golang-github-mattn-go-isatty-dev_0.0.14-1_all.deb ... Unpacking golang-github-mattn-go-isatty-dev (0.0.14-1) ... Selecting previously unselected package golang-github-mattn-go-colorable-dev. Preparing to unpack .../042-golang-github-mattn-go-colorable-dev_0.1.12-1_all.deb ... Unpacking golang-github-mattn-go-colorable-dev (0.1.12-1) ... Selecting previously unselected package golang-github-fatih-color-dev. Preparing to unpack .../043-golang-github-fatih-color-dev_1.7.0-1_all.deb ... Unpacking golang-github-fatih-color-dev (1.7.0-1) ... Selecting previously unselected package golang-github-davecgh-go-spew-dev. Preparing to unpack .../044-golang-github-davecgh-go-spew-dev_1.1.1-2_all.deb ... Unpacking golang-github-davecgh-go-spew-dev (1.1.1-2) ... Selecting previously unselected package golang-github-pmezard-go-difflib-dev. Preparing to unpack .../045-golang-github-pmezard-go-difflib-dev_1.0.0-3_all.deb ... Unpacking golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Selecting previously unselected package golang-github-stretchr-objx-dev. Preparing to unpack .../046-golang-github-stretchr-objx-dev_0.3.0-1_all.deb ... Unpacking golang-github-stretchr-objx-dev (0.3.0-1) ... Selecting previously unselected package golang-gopkg-yaml.v3-dev. Preparing to unpack .../047-golang-gopkg-yaml.v3-dev_3.0.1-1_all.deb ... Unpacking golang-gopkg-yaml.v3-dev (3.0.1-1) ... Selecting previously unselected package golang-github-stretchr-testify-dev. Preparing to unpack .../048-golang-github-stretchr-testify-dev_1.7.2-1_all.deb ... Unpacking golang-github-stretchr-testify-dev (1.7.2-1) ... Selecting previously unselected package golang-github-hashicorp-go-hclog-dev. Preparing to unpack .../049-golang-github-hashicorp-go-hclog-dev_0.11.0-1_all.deb ... Unpacking golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Selecting previously unselected package golang-github-jpillora-backoff-dev. Preparing to unpack .../050-golang-github-jpillora-backoff-dev_1.0.0-1.1_all.deb ... Unpacking golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Selecting previously unselected package golang-github-modern-go-concurrent-dev. Preparing to unpack .../051-golang-github-modern-go-concurrent-dev_1.0.3-1.1_all.deb ... Unpacking golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Selecting previously unselected package golang-github-modern-go-reflect2-dev. Preparing to unpack .../052-golang-github-modern-go-reflect2-dev_1.0.2-2_all.deb ... Unpacking golang-github-modern-go-reflect2-dev (1.0.2-2) ... Selecting previously unselected package golang-github-json-iterator-go-dev. Preparing to unpack .../053-golang-github-json-iterator-go-dev_1.1.12-1_all.deb ... Unpacking golang-github-json-iterator-go-dev (1.1.12-1) ... Selecting previously unselected package golang-github-beorn7-perks-dev. Preparing to unpack .../054-golang-github-beorn7-perks-dev_1.0.1-1_all.deb ... Unpacking golang-github-beorn7-perks-dev (1.0.1-1) ... Selecting previously unselected package golang-github-cespare-xxhash-dev. Preparing to unpack .../055-golang-github-cespare-xxhash-dev_2.1.1-2_all.deb ... Unpacking golang-github-cespare-xxhash-dev (2.1.1-2) ... Selecting previously unselected package zlib1g-dev:amd64. Preparing to unpack .../056-zlib1g-dev_1%3a1.2.11.dfsg-2ubuntu9_amd64.deb ... Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-2ubuntu9) ... Selecting previously unselected package libprotobuf23:amd64. Preparing to unpack .../057-libprotobuf23_3.12.4-1ubuntu7_amd64.deb ... Unpacking libprotobuf23:amd64 (3.12.4-1ubuntu7) ... Selecting previously unselected package libprotobuf-lite23:amd64. Preparing to unpack .../058-libprotobuf-lite23_3.12.4-1ubuntu7_amd64.deb ... Unpacking libprotobuf-lite23:amd64 (3.12.4-1ubuntu7) ... Selecting previously unselected package libprotobuf-dev:amd64. Preparing to unpack .../059-libprotobuf-dev_3.12.4-1ubuntu7_amd64.deb ... Unpacking libprotobuf-dev:amd64 (3.12.4-1ubuntu7) ... Selecting previously unselected package libprotoc23:amd64. Preparing to unpack .../060-libprotoc23_3.12.4-1ubuntu7_amd64.deb ... Unpacking libprotoc23:amd64 (3.12.4-1ubuntu7) ... Selecting previously unselected package protobuf-compiler. Preparing to unpack .../061-protobuf-compiler_3.12.4-1ubuntu7_amd64.deb ... Unpacking protobuf-compiler (3.12.4-1ubuntu7) ... Selecting previously unselected package golang-goprotobuf-dev. Preparing to unpack .../062-golang-goprotobuf-dev_1.3.4-2_amd64.deb ... Unpacking golang-goprotobuf-dev (1.3.4-2) ... Selecting previously unselected package golang-github-prometheus-client-model-dev. Preparing to unpack .../063-golang-github-prometheus-client-model-dev_0.2.0-3_all.deb ... Unpacking golang-github-prometheus-client-model-dev (0.2.0-3) ... Selecting previously unselected package golang-gopkg-yaml.v2-dev. Preparing to unpack .../064-golang-gopkg-yaml.v2-dev_2.4.0-3_all.deb ... Unpacking golang-gopkg-yaml.v2-dev (2.4.0-3) ... Selecting previously unselected package golang-github-jmespath-go-jmespath-dev. Preparing to unpack .../065-golang-github-jmespath-go-jmespath-dev_0.4.0-2_all.deb ... Unpacking golang-github-jmespath-go-jmespath-dev (0.4.0-2) ... Selecting previously unselected package golang-github-pkg-errors-dev. Preparing to unpack .../066-golang-github-pkg-errors-dev_0.9.1-2_all.deb ... Unpacking golang-github-pkg-errors-dev (0.9.1-2) ... Selecting previously unselected package golang-golang-x-text-dev. Preparing to unpack .../067-golang-golang-x-text-dev_0.3.7-1_all.deb ... Unpacking golang-golang-x-text-dev (0.3.7-1) ... Selecting previously unselected package golang-golang-x-net-dev. Preparing to unpack .../068-golang-golang-x-net-dev_1%3a0.0+git20211209.491a49a+dfsg-1_all.deb ... Unpacking golang-golang-x-net-dev (1:0.0+git20211209.491a49a+dfsg-1) ... Selecting previously unselected package golang-github-aws-aws-sdk-go-dev. Preparing to unpack .../069-golang-github-aws-aws-sdk-go-dev_1.41.14-1ubuntu1_all.deb ... Unpacking golang-github-aws-aws-sdk-go-dev (1.41.14-1ubuntu1) ... Selecting previously unselected package golang-github-julienschmidt-httprouter-dev. Preparing to unpack .../070-golang-github-julienschmidt-httprouter-dev_1.3.0-1_all.deb ... Unpacking golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Selecting previously unselected package golang-golang-x-oauth2-dev. Preparing to unpack .../071-golang-golang-x-oauth2-dev_0.0~git20211104.d3ed0bb-1_all.deb ... Unpacking golang-golang-x-oauth2-dev (0.0~git20211104.d3ed0bb-1) ... Selecting previously unselected package golang-github-alecthomas-units-dev. Preparing to unpack .../072-golang-github-alecthomas-units-dev_0.0~git20211218.b94a6e3-1_all.deb ... Unpacking golang-github-alecthomas-units-dev (0.0~git20211218.b94a6e3-1) ... Selecting previously unselected package golang-gopkg-alecthomas-kingpin.v2-dev. Preparing to unpack .../073-golang-gopkg-alecthomas-kingpin.v2-dev_2.2.6-3_all.deb ... Unpacking golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Selecting previously unselected package golang-protobuf-extensions-dev. Preparing to unpack .../074-golang-protobuf-extensions-dev_1.0.1-1_all.deb ... Unpacking golang-protobuf-extensions-dev (1.0.1-1) ... Selecting previously unselected package golang-github-prometheus-common-dev. Preparing to unpack .../075-golang-github-prometheus-common-dev_0.32.1-8_all.deb ... Unpacking golang-github-prometheus-common-dev (0.32.1-8) ... Selecting previously unselected package golang-golang-x-sync-dev. Preparing to unpack .../076-golang-golang-x-sync-dev_0.0~git20210220.036812b-1_all.deb ... Unpacking golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Selecting previously unselected package golang-github-prometheus-procfs-dev. Preparing to unpack .../077-golang-github-prometheus-procfs-dev_0.7.3-2_all.deb ... Unpacking golang-github-prometheus-procfs-dev (0.7.3-2) ... Selecting previously unselected package golang-github-prometheus-client-golang-dev. Preparing to unpack .../078-golang-github-prometheus-client-golang-dev_1.11.1-1_all.deb ... Unpacking golang-github-prometheus-client-golang-dev (1.11.1-1) ... Selecting previously unselected package golang-github-mwitkow-go-conntrack-dev. Preparing to unpack .../079-golang-github-mwitkow-go-conntrack-dev_0.0~git20190716.2f06839-2_all.deb ... Unpacking golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Selecting previously unselected package golang-github-pascaldekloe-goe-dev. Preparing to unpack .../080-golang-github-pascaldekloe-goe-dev_0.1.0-3_all.deb ... Unpacking golang-github-pascaldekloe-goe-dev (0.1.0-3) ... Selecting previously unselected package golang-github-pkg-diff-dev. Preparing to unpack .../081-golang-github-pkg-diff-dev_0.0~git20210226.20ebb0f-1_all.deb ... Unpacking golang-github-pkg-diff-dev (0.0~git20210226.20ebb0f-1) ... Selecting previously unselected package golang-github-kr-text-dev. Preparing to unpack .../082-golang-github-kr-text-dev_0.2.0-1_all.deb ... Unpacking golang-github-kr-text-dev (0.2.0-1) ... Selecting previously unselected package golang-github-kr-pretty-dev. Preparing to unpack .../083-golang-github-kr-pretty-dev_0.3.0-1_all.deb ... Unpacking golang-github-kr-pretty-dev (0.3.0-1) ... Selecting previously unselected package golang-gopkg-check.v1-dev. Preparing to unpack .../084-golang-gopkg-check.v1-dev_0.0+git20200902.038fdea-1_all.deb ... Unpacking golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Selecting previously unselected package golang-gopkg-errgo.v2-dev. Preparing to unpack .../085-golang-gopkg-errgo.v2-dev_2.1.0-2_all.deb ... Unpacking golang-gopkg-errgo.v2-dev (2.1.0-2) ... Selecting previously unselected package golang-github-rogpeppe-go-internal-dev. Preparing to unpack .../086-golang-github-rogpeppe-go-internal-dev_1.8.1-1_all.deb ... Unpacking golang-github-rogpeppe-go-internal-dev (1.8.1-1) ... Selecting previously unselected package golang-github-tv42-httpunix-dev. Preparing to unpack .../087-golang-github-tv42-httpunix-dev_0.0~git20150427.b75d861-2_all.deb ... Unpacking golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Selecting previously unselected package golang-github-vmihailenco-tagparser-dev. Preparing to unpack .../088-golang-github-vmihailenco-tagparser-dev_0.1.1-3_all.deb ... Unpacking golang-github-vmihailenco-tagparser-dev (0.1.1-3) ... Selecting previously unselected package libsasl2-modules-db:amd64. Preparing to unpack .../089-libsasl2-modules-db_2.1.28+dfsg-6ubuntu1_amd64.deb ... Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg-6ubuntu1) ... Selecting previously unselected package libsasl2-2:amd64. Preparing to unpack .../090-libsasl2-2_2.1.28+dfsg-6ubuntu1_amd64.deb ... Unpacking libsasl2-2:amd64 (2.1.28+dfsg-6ubuntu1) ... Selecting previously unselected package libsasl2-dev. Preparing to unpack .../091-libsasl2-dev_2.1.28+dfsg-6ubuntu1_amd64.deb ... Unpacking libsasl2-dev (2.1.28+dfsg-6ubuntu1) ... Selecting previously unselected package dh-golang. Preparing to unpack .../092-dh-golang_1.56_all.deb ... Unpacking dh-golang (1.56) ... Selecting previously unselected package golang-github-circonus-labs-circonusllhist-dev. Preparing to unpack .../093-golang-github-circonus-labs-circonusllhist-dev_0.0~git20191022.ec08cde-1_all.deb ... Unpacking golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Selecting previously unselected package golang-github-hashicorp-go-cleanhttp-dev. Preparing to unpack .../094-golang-github-hashicorp-go-cleanhttp-dev_0.5.2-1_all.deb ... Unpacking golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Selecting previously unselected package golang-github-hashicorp-go-retryablehttp-dev. Preparing to unpack .../095-golang-github-hashicorp-go-retryablehttp-dev_0.7.0-1_all.deb ... Unpacking golang-github-hashicorp-go-retryablehttp-dev (0.7.0-1) ... Selecting previously unselected package golang-github-circonus-labs-circonus-gometrics-dev. Preparing to unpack .../096-golang-github-circonus-labs-circonus-gometrics-dev_2.3.1-3_all.deb ... Unpacking golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Selecting previously unselected package golang-github-datadog-datadog-go-dev. Preparing to unpack .../097-golang-github-datadog-datadog-go-dev_2.1.0-3_all.deb ... Unpacking golang-github-datadog-datadog-go-dev (2.1.0-3) ... Selecting previously unselected package golang-github-hashicorp-go-uuid-dev. Preparing to unpack .../098-golang-github-hashicorp-go-uuid-dev_1.0.2-1_all.deb ... Unpacking golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Selecting previously unselected package golang-github-hashicorp-golang-lru-dev. Preparing to unpack .../099-golang-github-hashicorp-golang-lru-dev_0.5.4-2_all.deb ... Unpacking golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Selecting previously unselected package golang-github-hashicorp-go-immutable-radix-dev. Preparing to unpack .../100-golang-github-hashicorp-go-immutable-radix-dev_1.3.1-1_all.deb ... Unpacking golang-github-hashicorp-go-immutable-radix-dev (1.3.1-1) ... Selecting previously unselected package golang-github-armon-go-metrics-dev. Preparing to unpack .../101-golang-github-armon-go-metrics-dev_0.3.4-1_all.deb ... Unpacking golang-github-armon-go-metrics-dev (0.3.4-1) ... Selecting previously unselected package golang-gopkg-vmihailenco-msgpack.v2-dev. Preparing to unpack .../102-golang-gopkg-vmihailenco-msgpack.v2-dev_4.3.1-2_all.deb ... Unpacking golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Selecting previously unselected package golang-gopkg-tomb.v2-dev. Preparing to unpack .../103-golang-gopkg-tomb.v2-dev_0.0~git20161208.d5d1b58-3_all.deb ... Unpacking golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Selecting previously unselected package golang-gopkg-mgo.v2-dev. Preparing to unpack .../104-golang-gopkg-mgo.v2-dev_2016.08.01-7_all.deb ... Unpacking golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Selecting previously unselected package golang-github-hashicorp-go-msgpack-dev. Preparing to unpack .../105-golang-github-hashicorp-go-msgpack-dev_0.5.5-1_all.deb ... Unpacking golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Selecting previously unselected package sbuild-build-depends-main-dummy. Preparing to unpack .../106-sbuild-build-depends-main-dummy_0.invalid.0_amd64.deb ... Unpacking sbuild-build-depends-main-dummy (0.invalid.0) ... Setting up libpipeline1:amd64 (1.5.6-1) ... Setting up libicu71:amd64 (71.1-3) ... Setting up golang-gopkg-yaml.v2-dev (2.4.0-3) ... Setting up golang-github-hashicorp-go-uuid-dev (1.0.2-1) ... Setting up bsdextrautils (2.38-4ubuntu1) ... Setting up golang-github-creack-pty-dev (1.1.18-1) ... Setting up golang-github-pkg-diff-dev (0.0~git20210226.20ebb0f-1) ... Setting up golang-github-stretchr-objx-dev (0.3.0-1) ... Setting up libmagic-mgc (1:5.41-4) ... Setting up golang-github-pkg-errors-dev (0.9.1-2) ... Setting up golang-github-hashicorp-golang-lru-dev (0.5.4-2) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up golang-gopkg-yaml.v3-dev (3.0.1-1) ... Setting up libdebhelper-perl (13.7.1ubuntu1) ... Setting up golang-golang-x-sys-dev (0.0~git20220520.bc2c85a-1) ... Setting up libmagic1:amd64 (1:5.41-4) ... Setting up golang-github-pmezard-go-difflib-dev (1.0.0-3) ... Setting up golang-github-modern-go-concurrent-dev (1.0.3-1.1) ... Setting up gettext-base (0.21-6) ... Setting up golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ... Setting up libprotobuf23:amd64 (3.12.4-1ubuntu7) ... Setting up file (1:5.41-4) ... Setting up golang-github-beorn7-perks-dev (1.0.1-1) ... Setting up golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ... Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg-6ubuntu1) ... Setting up golang-github-cespare-xxhash-dev (2.1.1-2) ... Setting up golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ... Setting up golang-github-jpillora-backoff-dev (1.0.0-1.1) ... Setting up golang-github-davecgh-go-spew-dev (1.1.1-2) ... Setting up libprotobuf-lite23:amd64 (3.12.4-1ubuntu7) ... Setting up autotools-dev (20220109.1) ... Setting up golang-github-pascaldekloe-goe-dev (0.1.0-3) ... Setting up golang-github-go-logfmt-logfmt-dev (0.5.0-2) ... Setting up golang-github-kr-text-dev (0.2.0-1) ... Setting up golang-github-stretchr-testify-dev (1.7.2-1) ... Setting up golang-github-hashicorp-go-immutable-radix-dev (1.3.1-1) ... Setting up libprotoc23:amd64 (3.12.4-1ubuntu7) ... Setting up libsigsegv2:amd64 (2.13-1ubuntu3) ... Setting up golang-golang-x-sync-dev (0.0~git20210220.036812b-1) ... Setting up autopoint (0.21-6) ... Setting up golang-1.18-src (1.18.3-1ubuntu1) ... Setting up libsasl2-2:amd64 (2.1.28+dfsg-6ubuntu1) ... Setting up golang-github-julienschmidt-httprouter-dev (1.3.0-1) ... Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-2ubuntu9) ... Setting up golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-2) ... Setting up golang-golang-x-text-dev (0.3.7-1) ... Setting up libuchardet0:amd64 (0.0.7-1build2) ... Setting up golang-github-prometheus-procfs-dev (0.7.3-2) ... Setting up golang-github-alecthomas-units-dev (0.0~git20211218.b94a6e3-1) ... Setting up golang-github-modern-go-reflect2-dev (1.0.2-2) ... Setting up libsub-override-perl (0.09-3) ... Setting up golang-github-datadog-datadog-go-dev (2.1.0-3) ... Setting up golang-src (2:1.18~0ubuntu2) ... Setting up libelf1:amd64 (0.187-1) ... Setting up golang-github-vmihailenco-tagparser-dev (0.1.1-3) ... Setting up libxml2:amd64 (2.9.14+dfsg-1) ... Setting up golang-github-jmespath-go-jmespath-dev (0.4.0-2) ... Setting up libfile-stripnondeterminism-perl (1.13.0-1) ... Setting up golang-github-mattn-go-isatty-dev (0.0.14-1) ... Setting up golang-github-go-kit-log-dev (0.2.0-3) ... Setting up libdw1:amd64 (0.187-1) ... Setting up gettext (0.21-6) ... Setting up protobuf-compiler (3.12.4-1ubuntu7) ... Setting up golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ... Setting up golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-3) ... Setting up libtool (2.4.7-4) ... Setting up golang-github-mattn-go-colorable-dev (0.1.12-1) ... Setting up golang-1.18-go (1.18.3-1ubuntu1) ... Setting up m4 (1.4.18-5ubuntu2) ... Setting up golang-github-json-iterator-go-dev (1.1.12-1) ... Setting up intltool-debian (0.35.0+20060710.5) ... Setting up libprotobuf-dev:amd64 (3.12.4-1ubuntu7) ... Setting up libsasl2-dev (2.1.28+dfsg-6ubuntu1) ... Setting up golang-goprotobuf-dev (1.3.4-2) ... Setting up golang-golang-x-net-dev (1:0.0+git20211209.491a49a+dfsg-1) ... Setting up autoconf (2.71-2) ... Setting up dh-strip-nondeterminism (1.13.0-1) ... Setting up dwz (0.14-1build2) ... Setting up groff-base (1.22.4-8build1) ... Setting up debugedit (1:5.0-4build1) ... Setting up golang-github-prometheus-client-model-dev (0.2.0-3) ... Setting up golang-golang-x-oauth2-dev (0.0~git20211104.d3ed0bb-1) ... Setting up golang-github-aws-aws-sdk-go-dev (1.41.14-1ubuntu1) ... Setting up automake (1:1.16.5-1.3) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up golang-github-fatih-color-dev (1.7.0-1) ... Setting up golang-go:amd64 (2:1.18~0ubuntu2) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up golang-protobuf-extensions-dev (1.0.1-1) ... Setting up man-db (2.10.2-1) ... Not building database; man-db/auto-update is not 'true'. Created symlink /etc/systemd/system/timers.target.wants/man-db.timer → /lib/systemd/system/man-db.timer. Setting up golang-any:amd64 (2:1.18~0ubuntu2) ... Setting up dh-autoreconf (20) ... Setting up golang-github-hashicorp-go-hclog-dev (0.11.0-1) ... Setting up debhelper (13.7.1ubuntu1) ... Setting up golang-github-hashicorp-go-retryablehttp-dev (0.7.0-1) ... Setting up dh-golang (1.56) ... Setting up golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-3) ... Setting up golang-github-rogpeppe-go-internal-dev (1.8.1-1) ... Setting up golang-github-kr-pretty-dev (0.3.0-1) ... Setting up golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-2) ... Setting up golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ... Setting up golang-gopkg-mgo.v2-dev (2016.08.01-7) ... Setting up golang-github-prometheus-client-golang-dev (1.11.1-1) ... Setting up golang-github-armon-go-metrics-dev (0.3.4-1) ... Setting up golang-gopkg-errgo.v2-dev (2.1.0-2) ... Setting up golang-github-prometheus-common-dev (0.32.1-8) ... Setting up golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ... Setting up sbuild-build-depends-main-dummy (0.invalid.0) ... Processing triggers for libc-bin (2.35-0ubuntu3) ... +------------------------------------------------------------------------------+ | Check architectures | +------------------------------------------------------------------------------+ Arch check ok (amd64 included in all) +------------------------------------------------------------------------------+ | Build environment | +------------------------------------------------------------------------------+ Kernel: Linux 5.4.0-120-generic #136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022 amd64 (x86_64) Toolchain package versions: binutils_2.38.50.20220615-4ubuntu1 dpkg-dev_1.21.8ubuntu1 g++-11_11.3.0-3ubuntu1 gcc-11_11.3.0-3ubuntu1 libc6-dev_2.35-0ubuntu3 libstdc++-11-dev_11.3.0-3ubuntu1 libstdc++6_12.1.0-2ubuntu1 linux-libc-dev_5.15.0-27.28 Package versions: adduser_3.121ubuntu1 advancecomp_2.1-2.1ubuntu2 apt_2.5.0 autoconf_2.71-2 automake_1:1.16.5-1.3 autopoint_0.21-6 autotools-dev_20220109.1 base-files_12.2ubuntu1 base-passwd_3.5.52build1 bash_5.1-6ubuntu1 binutils_2.38.50.20220615-4ubuntu1 binutils-common_2.38.50.20220615-4ubuntu1 binutils-x86-64-linux-gnu_2.38.50.20220615-4ubuntu1 bsdextrautils_2.38-4ubuntu1 bsdutils_1:2.38-4ubuntu1 build-essential_12.9ubuntu3 bzip2_1.0.8-5build1 ca-certificates_20211016 coreutils_8.32-4.1ubuntu1 cpp_4:11.2.0-1ubuntu1 cpp-11_11.3.0-3ubuntu1 dash_0.5.11+git20210903+057cd650a4ed-3build1 debconf_1.5.79ubuntu1 debhelper_13.7.1ubuntu1 debianutils_5.7-0.2 debugedit_1:5.0-4build1 dh-autoreconf_20 dh-golang_1.56 dh-strip-nondeterminism_1.13.0-1 diffutils_1:3.8-0ubuntu2 dpkg_1.21.8ubuntu1 dpkg-dev_1.21.8ubuntu1 dwz_0.14-1build2 e2fsprogs_1.46.5-2ubuntu2 fakeroot_1.29-1ubuntu1 file_1:5.41-4 findutils_4.8.0-1ubuntu3 g++_4:11.2.0-1ubuntu1 g++-11_11.3.0-3ubuntu1 gcc_4:11.2.0-1ubuntu1 gcc-11_11.3.0-3ubuntu1 gcc-11-base_11.3.0-3ubuntu1 gcc-12-base_12.1.0-2ubuntu1 gettext_0.21-6 gettext-base_0.21-6 golang-1.18-go_1.18.3-1ubuntu1 golang-1.18-src_1.18.3-1ubuntu1 golang-any_2:1.18~0ubuntu2 golang-github-alecthomas-units-dev_0.0~git20211218.b94a6e3-1 golang-github-armon-go-metrics-dev_0.3.4-1 golang-github-aws-aws-sdk-go-dev_1.41.14-1ubuntu1 golang-github-beorn7-perks-dev_1.0.1-1 golang-github-cespare-xxhash-dev_2.1.1-2 golang-github-circonus-labs-circonus-gometrics-dev_2.3.1-3 golang-github-circonus-labs-circonusllhist-dev_0.0~git20191022.ec08cde-1 golang-github-creack-pty-dev_1.1.18-1 golang-github-datadog-datadog-go-dev_2.1.0-3 golang-github-davecgh-go-spew-dev_1.1.1-2 golang-github-fatih-color-dev_1.7.0-1 golang-github-go-kit-log-dev_0.2.0-3 golang-github-go-logfmt-logfmt-dev_0.5.0-2 golang-github-hashicorp-go-cleanhttp-dev_0.5.2-1 golang-github-hashicorp-go-hclog-dev_0.11.0-1 golang-github-hashicorp-go-immutable-radix-dev_1.3.1-1 golang-github-hashicorp-go-msgpack-dev_0.5.5-1 golang-github-hashicorp-go-retryablehttp-dev_0.7.0-1 golang-github-hashicorp-go-uuid-dev_1.0.2-1 golang-github-hashicorp-golang-lru-dev_0.5.4-2 golang-github-jmespath-go-jmespath-dev_0.4.0-2 golang-github-jpillora-backoff-dev_1.0.0-1.1 golang-github-json-iterator-go-dev_1.1.12-1 golang-github-julienschmidt-httprouter-dev_1.3.0-1 golang-github-kr-pretty-dev_0.3.0-1 golang-github-kr-text-dev_0.2.0-1 golang-github-mattn-go-colorable-dev_0.1.12-1 golang-github-mattn-go-isatty-dev_0.0.14-1 golang-github-modern-go-concurrent-dev_1.0.3-1.1 golang-github-modern-go-reflect2-dev_1.0.2-2 golang-github-mwitkow-go-conntrack-dev_0.0~git20190716.2f06839-2 golang-github-pascaldekloe-goe-dev_0.1.0-3 golang-github-pkg-diff-dev_0.0~git20210226.20ebb0f-1 golang-github-pkg-errors-dev_0.9.1-2 golang-github-pmezard-go-difflib-dev_1.0.0-3 golang-github-prometheus-client-golang-dev_1.11.1-1 golang-github-prometheus-client-model-dev_0.2.0-3 golang-github-prometheus-common-dev_0.32.1-8 golang-github-prometheus-procfs-dev_0.7.3-2 golang-github-rogpeppe-go-internal-dev_1.8.1-1 golang-github-stretchr-objx-dev_0.3.0-1 golang-github-stretchr-testify-dev_1.7.2-1 golang-github-tv42-httpunix-dev_0.0~git20150427.b75d861-2 golang-github-vmihailenco-tagparser-dev_0.1.1-3 golang-go_2:1.18~0ubuntu2 golang-golang-x-net-dev_1:0.0+git20211209.491a49a+dfsg-1 golang-golang-x-oauth2-dev_0.0~git20211104.d3ed0bb-1 golang-golang-x-sync-dev_0.0~git20210220.036812b-1 golang-golang-x-sys-dev_0.0~git20220520.bc2c85a-1 golang-golang-x-text-dev_0.3.7-1 golang-gopkg-alecthomas-kingpin.v2-dev_2.2.6-3 golang-gopkg-check.v1-dev_0.0+git20200902.038fdea-1 golang-gopkg-errgo.v2-dev_2.1.0-2 golang-gopkg-mgo.v2-dev_2016.08.01-7 golang-gopkg-tomb.v2-dev_0.0~git20161208.d5d1b58-3 golang-gopkg-vmihailenco-msgpack.v2-dev_4.3.1-2 golang-gopkg-yaml.v2-dev_2.4.0-3 golang-gopkg-yaml.v3-dev_3.0.1-1 golang-goprotobuf-dev_1.3.4-2 golang-protobuf-extensions-dev_1.0.1-1 golang-src_2:1.18~0ubuntu2 gpg_2.2.27-3ubuntu2 gpg-agent_2.2.27-3ubuntu2 gpgconf_2.2.27-3ubuntu2 gpgv_2.2.27-3ubuntu2 grep_3.7-1build1 groff-base_1.22.4-8build1 gzip_1.10-4ubuntu4 hostname_3.23ubuntu2 init_1.63 init-system-helpers_1.63 intltool-debian_0.35.0+20060710.5 libacl1_2.3.1-1 libapparmor1_3.0.4-2ubuntu3 libapt-pkg6.0_2.5.0 libarchive-zip-perl_1.68-1 libargon2-1_0~20171227-0.3 libasan6_11.3.0-3ubuntu1 libassuan0_2.5.5-3 libatomic1_12.1.0-2ubuntu1 libattr1_1:2.5.1-1build1 libaudit-common_1:3.0.7-1build1 libaudit1_1:3.0.7-1build1 libbinutils_2.38.50.20220615-4ubuntu1 libblkid1_2.38-4ubuntu1 libbz2-1.0_1.0.8-5build1 libc-bin_2.35-0ubuntu3 libc-dev-bin_2.35-0ubuntu3 libc6_2.35-0ubuntu3 libc6-dev_2.35-0ubuntu3 libcap-ng0_0.7.9-4 libcap2_1:2.44-1build3 libcc1-0_12.1.0-2ubuntu1 libcom-err2_1.46.5-2ubuntu2 libcrypt-dev_1:4.4.27-1 libcrypt1_1:4.4.27-1 libcryptsetup12_2:2.4.3-1ubuntu1 libctf-nobfd0_2.38.50.20220615-4ubuntu1 libctf0_2.38.50.20220615-4ubuntu1 libdb5.3_5.3.28+dfsg1-0.9 libdebconfclient0_0.261ubuntu1 libdebhelper-perl_13.7.1ubuntu1 libdevmapper1.02.1_2:1.02.175-2.1ubuntu4 libdpkg-perl_1.21.8ubuntu1 libdw1_0.187-1 libelf1_0.187-1 libext2fs2_1.46.5-2ubuntu2 libfakeroot_1.29-1ubuntu1 libffi8_3.4.2-4 libfile-stripnondeterminism-perl_1.13.0-1 libgcc-11-dev_11.3.0-3ubuntu1 libgcc-s1_12.1.0-2ubuntu1 libgcrypt20_1.10.1-2ubuntu1 libgdbm-compat4_1.23-1 libgdbm6_1.23-1 libgmp10_2:6.2.1+dfsg-3ubuntu1 libgnutls30_3.7.6-2ubuntu1 libgomp1_12.1.0-2ubuntu1 libgpg-error0_1.45-2 libgprofng0_2.38.50.20220615-4ubuntu1 libgssapi-krb5-2_1.19.2-2 libhogweed6_3.7.3-1build2 libicu71_71.1-3 libidn2-0_2.3.2-2build1 libip4tc2_1.8.7-1ubuntu6 libisl23_0.24-2build1 libitm1_12.1.0-2ubuntu1 libjson-c5_0.16-1 libk5crypto3_1.19.2-2 libkeyutils1_1.6.1-3ubuntu1 libkmod2_29-1ubuntu1 libkrb5-3_1.19.2-2 libkrb5support0_1.19.2-2 liblockfile-bin_1.17-1build2 liblockfile1_1.17-1build2 liblsan0_12.1.0-2ubuntu1 liblz4-1_1.9.3-2build2 liblzma5_5.2.5-2.1 libmagic-mgc_1:5.41-4 libmagic1_1:5.41-4 libmount1_2.38-4ubuntu1 libmpc3_1.2.1-2build1 libmpfr6_4.1.0-3build3 libncurses6_6.3+20220423-2 libncursesw6_6.3+20220423-2 libnettle8_3.7.3-1build2 libnpth0_1.6-3build2 libnsl-dev_1.3.0-2build2 libnsl2_1.3.0-2build2 libp11-kit0_0.24.1-1 libpam-modules_1.4.0-13ubuntu1 libpam-modules-bin_1.4.0-13ubuntu1 libpam-runtime_1.4.0-13ubuntu1 libpam0g_1.4.0-13ubuntu1 libpcre2-8-0_10.40-1 libpcre3_2:8.39-14 libperl5.34_5.34.0-3ubuntu1 libpipeline1_1.5.6-1 libpng16-16_1.6.37-5 libprocps8_2:3.3.17-6ubuntu2 libprotobuf-dev_3.12.4-1ubuntu7 libprotobuf-lite23_3.12.4-1ubuntu7 libprotobuf23_3.12.4-1ubuntu7 libprotoc23_3.12.4-1ubuntu7 libquadmath0_12.1.0-2ubuntu1 libreadline8_8.1.2-1.2 libsasl2-2_2.1.28+dfsg-6ubuntu1 libsasl2-dev_2.1.28+dfsg-6ubuntu1 libsasl2-modules-db_2.1.28+dfsg-6ubuntu1 libseccomp2_2.5.4-1ubuntu1 libselinux1_3.4-1 libsemanage-common_3.4-1 libsemanage2_3.4-1 libsepol2_3.4-2 libsigsegv2_2.13-1ubuntu3 libsmartcols1_2.38-4ubuntu1 libsqlite3-0_3.38.5-1 libss2_1.46.5-2ubuntu2 libssl3_3.0.3-5ubuntu3 libstdc++-11-dev_11.3.0-3ubuntu1 libstdc++6_12.1.0-2ubuntu1 libsub-override-perl_0.09-3 libsystemd0_249.11-0ubuntu4 libtasn1-6_4.18.0-4build1 libtinfo6_6.3+20220423-2 libtirpc-common_1.3.2-2build1 libtirpc-dev_1.3.2-2build1 libtirpc3_1.3.2-2build1 libtool_2.4.7-4 libtsan0_11.3.0-3ubuntu1 libubsan1_12.1.0-2ubuntu1 libuchardet0_0.0.7-1build2 libudev1_249.11-0ubuntu4 libunistring2_1.0-1 libuuid1_2.38-4ubuntu1 libxml2_2.9.14+dfsg-1 libxxhash0_0.8.1-1 libzstd1_1.5.2+dfsg-1 linux-libc-dev_5.15.0-27.28 lockfile-progs_0.1.19build1 login_1:4.11.1+dfsg1-2ubuntu1 logsave_1.46.5-2ubuntu2 lsb-base_11.1.0ubuntu4 lto-disabled-list_27 m4_1.4.18-5ubuntu2 make_4.3-4.1build1 man-db_2.10.2-1 mawk_1.3.4.20200120-3.1 mount_2.38-4ubuntu1 ncurses-base_6.3+20220423-2 ncurses-bin_6.3+20220423-2 openssl_3.0.3-5ubuntu3 optipng_0.7.7-2build1 passwd_1:4.11.1+dfsg1-2ubuntu1 patch_2.7.6-7build2 perl_5.34.0-3ubuntu1 perl-base_5.34.0-3ubuntu1 perl-modules-5.34_5.34.0-3ubuntu1 pinentry-curses_1.2.0-1ubuntu1 pkgbinarymangler_149 po-debconf_1.0.21+nmu1 policyrcd-script-zg2_0.1-3.1 procps_2:3.3.17-6ubuntu2 protobuf-compiler_3.12.4-1ubuntu7 readline-common_8.1.2-1.2 rpcsvc-proto_1.4.2-0ubuntu6 sbuild-build-depends-main-dummy_0.invalid.0 sed_4.8-1ubuntu2 sensible-utils_0.0.17 systemd_249.11-0ubuntu4 systemd-sysv_249.11-0ubuntu4 sysvinit-utils_3.01-1ubuntu1 tar_1.34+dfsg-1build3 tzdata_2022a-0ubuntu1 ubuntu-keyring_2021.03.26 usrmerge_25ubuntu2 util-linux_2.38-4ubuntu1 util-linux-extra_2.38-4ubuntu1 xz-utils_5.2.5-2.1 zlib1g_1:1.2.11.dfsg-2ubuntu9 zlib1g-dev_1:1.2.11.dfsg-2ubuntu9 +------------------------------------------------------------------------------+ | Build | +------------------------------------------------------------------------------+ Unpack source ------------- -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 3.0 (quilt) Source: golang-github-hashicorp-raft Binary: golang-github-hashicorp-raft-dev Architecture: all Version: 1.1.2-1 Maintainer: Debian Go Packaging Team Uploaders: Tim Potter , Dmitry Smirnov , Homepage: https://github.com/hashicorp/raft Standards-Version: 4.4.1 Vcs-Browser: https://salsa.debian.org/go-team/packages/golang-github-hashicorp-raft Vcs-Git: https://salsa.debian.org/go-team/packages/golang-github-hashicorp-raft.git Testsuite: autopkgtest-pkg-go Build-Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~) Package-List: golang-github-hashicorp-raft-dev deb devel optional arch=all Checksums-Sha1: 6eb88916d5b8568cc5e49dfd778e53f425f02612 103160 golang-github-hashicorp-raft_1.1.2.orig.tar.xz 3efa08a30605d234327a7c1293b36ecde63ffbbe 8980 golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz Checksums-Sha256: 6c3383ee430aa6b421cc4cb4cbc485b0e612bda2e94e88d416d13d65d375dc89 103160 golang-github-hashicorp-raft_1.1.2.orig.tar.xz 3877bf052d5719266221189613502262d1be1af152c736d3dc8ed49734ad6dd8 8980 golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz Files: 65844d5b08afa2d62ea4dedaf43e6047 103160 golang-github-hashicorp-raft_1.1.2.orig.tar.xz 036528358012213daa0588e86be2a3d4 8980 golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz Go-Import-Path: github.com/hashicorp/raft -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEULx8+TnSDCcqawZWUra72VOWjRsFAl4kBCwACgkQUra72VOW jRsPQBAAxzZChwaTUFMUIQz9GgdwyR5o2SjXw48JJ//lfygslStaiz8ju52lp33x a1d56obRwTy2QmY2s6+oV3Mq4sQWs7U/1PGnywKeA71Us10A4AAv5d9wMRq1eqzp PcyVdhfM0zFcmJf8Wr93Mi8wbCr9PT+oTxc59+zWDruS7xCoe3aDtaekVfb8QZyj OZPsaTtneSxKjqk+V5Jyx8X6Y6P2cwv89GDd4nxTxapw8H38e2cBk7BsYmXUNXye 6N7dOChDN+x6gqgBU0O97oex+QOpd4LNoFBl2gEi7kukw+jrjrdyKQCANPh0CKx0 K4MPq2BeI3qKJV21l2z7S30BSjduvyktC7efckw5CX4zgT9/kJ5e7ZAf/og+M2vc CoCV+aoAH1OOWbkPu0dsuJiA5z1ZkKGa8ZB5MgvUentk7LVd/HaXW9QDdRTNZT8z Jdh6/wnpSn8ZZEhAixTtqNt64s0sNWmh4krF0WgGNZ1djzYd6gVI1LqlJTaSRKFo nxGsEe+jnC6kv2u8XpLG+xevTKcu4UIFHq2ebpanFvxzqxAv1dWTBDDI13NApMZb Po5huhcdLapUW4NGW/3pkdJkpcY5u26zH6YvTxDICU/tHXDo5/BqI9E1jEj+5m5J XrNqpKLFjW2I9gRcmT5vy3O4edLCPBFs6MGQBK7u1V+nEkh8pJc= =IObb -----END PGP SIGNATURE----- gpgv: Signature made Sun Jan 19 07:24:28 2020 UTC gpgv: using RSA key 50BC7CF939D20C272A6B065652B6BBD953968D1B gpgv: Can't check signature: No public key dpkg-source: warning: cannot verify signature ./golang-github-hashicorp-raft_1.1.2-1.dsc dpkg-source: info: extracting golang-github-hashicorp-raft in /<> dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2.orig.tar.xz dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying t-skip--TestNetworkTransport_AppendEntriesPipeline_CloseStreams.patch dpkg-source: info: applying t-skip--TestRaft_AutoSnapshot.patch dpkg-source: info: applying t-skip--TestRaft_LeadershipTransferLeaderRejectsClientRequests.patch dpkg-source: info: applying t-skip--TestRaft_RecoverCluster.patch dpkg-source: info: applying t-skip--TestRaft_SnapshotRestore_PeerChange.patch Check disk space ---------------- Sufficient free space for build User Environment ---------------- APT_CONFIG=/var/lib/sbuild/apt.conf DEB_BUILD_OPTIONS=noautodbgsym parallel=4 HOME=/sbuild-nonexistent LANG=C.UTF-8 LC_ALL=C.UTF-8 LOGNAME=buildd PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games SCHROOT_ALIAS_NAME=build-PACKAGEBUILD-24000232 SCHROOT_CHROOT_NAME=build-PACKAGEBUILD-24000232 SCHROOT_COMMAND=env SCHROOT_GID=2501 SCHROOT_GROUP=buildd SCHROOT_SESSION_ID=build-PACKAGEBUILD-24000232 SCHROOT_UID=2001 SCHROOT_USER=buildd SHELL=/bin/sh TERM=unknown USER=buildd V=1 dpkg-buildpackage ----------------- Command: dpkg-buildpackage -us -uc -mLaunchpad Build Daemon -b -rfakeroot dpkg-buildpackage: info: source package golang-github-hashicorp-raft dpkg-buildpackage: info: source version 1.1.2-1 dpkg-buildpackage: info: source distribution unstable dpkg-source --before-build . dpkg-buildpackage: info: host architecture amd64 fakeroot debian/rules clean dh clean --buildsystem=golang --with=golang dh_auto_clean -O--buildsystem=golang dh_autoreconf_clean -O--buildsystem=golang dh_clean -O--buildsystem=golang debian/rules build dh build --buildsystem=golang --with=golang dh_update_autotools_config -O--buildsystem=golang dh_autoreconf -O--buildsystem=golang dh_auto_configure -O--buildsystem=golang dh_auto_build -O--buildsystem=golang cd obj-x86_64-linux-gnu && go install -trimpath -v -p 4 github.com/hashicorp/raft github.com/hashicorp/raft/bench internal/goarch internal/unsafeheader internal/goexperiment internal/cpu internal/goos internal/abi runtime/internal/atomic runtime/internal/math runtime/internal/sys runtime/internal/syscall internal/race sync/atomic internal/bytealg unicode unicode/utf8 container/list internal/itoa crypto/internal/subtle crypto/subtle math/bits encoding runtime math unicode/utf16 internal/nettrace vendor/golang.org/x/crypto/cryptobyte/asn1 vendor/golang.org/x/crypto/internal/subtle golang.org/x/sys/internal/unsafeheader internal/reflectlite sync internal/singleflight math/rand internal/testlog internal/sysinfo runtime/cgo errors sort io internal/oserror syscall strconv bytes strings reflect bufio time internal/syscall/unix path internal/syscall/execenv github.com/hashicorp/golang-lru/simplelru github.com/hashicorp/go-immutable-radix vendor/golang.org/x/net/dns/dnsmessage hash context io/fs internal/poll hash/crc32 encoding/binary internal/fmtsort crypto crypto/elliptic/internal/fiat embed crypto/cipher os encoding/base64 crypto/internal/randutil crypto/sha512 crypto/aes crypto/des crypto/ed25519/internal/edwards25519/field fmt internal/godebug internal/intern crypto/ed25519/internal/edwards25519 net/netip crypto/hmac crypto/md5 crypto/elliptic/internal/nistec net math/big encoding/json log compress/flate crypto/rc4 crypto/sha1 compress/gzip crypto/sha256 encoding/hex crypto/rand encoding/asn1 crypto/dsa encoding/pem crypto/elliptic crypto/ed25519 crypto/rsa vendor/golang.org/x/crypto/cryptobyte crypto/x509/pkix net/url path/filepath crypto/ecdsa vendor/golang.org/x/crypto/chacha20 vendor/golang.org/x/crypto/internal/poly1305 io/ioutil vendor/golang.org/x/sys/cpu vendor/golang.org/x/crypto/curve25519/internal/field vendor/golang.org/x/crypto/hkdf vendor/golang.org/x/text/transform vendor/golang.org/x/crypto/chacha20poly1305 vendor/golang.org/x/crypto/curve25519 vendor/golang.org/x/text/unicode/bidi vendor/golang.org/x/text/unicode/norm vendor/golang.org/x/net/http2/hpack mime vendor/golang.org/x/text/secure/bidirule mime/quotedprintable net/http/internal net/http/internal/ascii os/signal golang.org/x/sys/unix regexp/syntax vendor/golang.org/x/net/idna regexp encoding/gob crypto/x509 net/textproto vendor/golang.org/x/net/http/httpguts vendor/golang.org/x/net/http/httpproxy mime/multipart github.com/mattn/go-isatty go/token crypto/tls github.com/mattn/go-colorable html github.com/fatih/color text/template/parse hash/crc64 github.com/hashicorp/go-hclog flag runtime/debug runtime/trace testing text/template html/template net/http/httptrace net/http github.com/armon/go-metrics net/rpc github.com/hashicorp/go-msgpack/codec github.com/hashicorp/raft github.com/hashicorp/raft/bench dh_auto_test -O--buildsystem=golang cd obj-x86_64-linux-gnu && go test -vet=off -v -p 4 github.com/hashicorp/raft github.com/hashicorp/raft/bench === RUN TestCommitment_setVoters --- PASS: TestCommitment_setVoters (0.00s) === RUN TestCommitment_match_max --- PASS: TestCommitment_match_max (0.00s) === RUN TestCommitment_match_nonVoting --- PASS: TestCommitment_match_nonVoting (0.00s) === RUN TestCommitment_recalculate --- PASS: TestCommitment_recalculate (0.00s) === RUN TestCommitment_recalculate_startIndex --- PASS: TestCommitment_recalculate_startIndex (0.00s) === RUN TestCommitment_noVoterSanity --- PASS: TestCommitment_noVoterSanity (0.00s) === RUN TestCommitment_singleVoter --- PASS: TestCommitment_singleVoter (0.00s) === RUN TestConfiguration_Configuration_Clone --- PASS: TestConfiguration_Configuration_Clone (0.00s) === RUN TestConfiguration_configurations_Clone --- PASS: TestConfiguration_configurations_Clone (0.00s) === RUN TestConfiguration_hasVote --- PASS: TestConfiguration_hasVote (0.00s) === RUN TestConfiguration_checkConfiguration --- PASS: TestConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_nextConfiguration_table --- PASS: TestConfiguration_nextConfiguration_table (0.00s) === RUN TestConfiguration_nextConfiguration_prevIndex --- PASS: TestConfiguration_nextConfiguration_prevIndex (0.00s) === RUN TestConfiguration_nextConfiguration_checkConfiguration --- PASS: TestConfiguration_nextConfiguration_checkConfiguration (0.00s) === RUN TestConfiguration_encodeDecodePeers --- PASS: TestConfiguration_encodeDecodePeers (0.00s) === RUN TestConfiguration_encodeDecodeConfiguration --- PASS: TestConfiguration_encodeDecodeConfiguration (0.00s) === RUN TestDiscardSnapshotStoreImpl --- PASS: TestDiscardSnapshotStoreImpl (0.00s) === RUN TestDiscardSnapshotSinkImpl --- PASS: TestDiscardSnapshotSinkImpl (0.00s) === RUN TestFileSnapshotStoreImpl --- PASS: TestFileSnapshotStoreImpl (0.00s) === RUN TestFileSnapshotSinkImpl --- PASS: TestFileSnapshotSinkImpl (0.00s) === RUN TestFileSS_CreateSnapshotMissingParentDir testing.go:153: 2022-06-22T14:53:07.190Z [INFO] creating new snapshot: path=/tmp/raft1937488111/raft3631163025/snapshots/3-10-1655909587190.tmp --- PASS: TestFileSS_CreateSnapshotMissingParentDir (0.00s) === RUN TestFileSS_CreateSnapshot testing.go:153: 2022-06-22T14:53:07.193Z [INFO] creating new snapshot: path=/tmp/raft2506191216/snapshots/3-10-1655909587193.tmp testing.go:153: 2022-06-22T14:53:07.194Z [WARN] found temporary snapshot: name=3-10-1655909587193.tmp --- PASS: TestFileSS_CreateSnapshot (0.00s) === RUN TestFileSS_CancelSnapshot testing.go:153: 2022-06-22T14:53:07.198Z [INFO] creating new snapshot: path=/tmp/raft1206912702/snapshots/3-10-1655909587198.tmp --- PASS: TestFileSS_CancelSnapshot (0.00s) === RUN TestFileSS_Retention testing.go:153: 2022-06-22T14:53:07.200Z [INFO] creating new snapshot: path=/tmp/raft1347821793/snapshots/3-10-1655909587200.tmp testing.go:153: 2022-06-22T14:53:07.207Z [INFO] creating new snapshot: path=/tmp/raft1347821793/snapshots/3-11-1655909587207.tmp testing.go:153: 2022-06-22T14:53:07.210Z [INFO] creating new snapshot: path=/tmp/raft1347821793/snapshots/3-12-1655909587210.tmp testing.go:153: 2022-06-22T14:53:07.212Z [INFO] reaping snapshot: path=/tmp/raft1347821793/snapshots/3-10-1655909587200 testing.go:153: 2022-06-22T14:53:07.212Z [INFO] creating new snapshot: path=/tmp/raft1347821793/snapshots/3-13-1655909587212.tmp testing.go:153: 2022-06-22T14:53:07.216Z [INFO] reaping snapshot: path=/tmp/raft1347821793/snapshots/3-11-1655909587207 testing.go:153: 2022-06-22T14:53:07.216Z [INFO] creating new snapshot: path=/tmp/raft1347821793/snapshots/3-14-1655909587216.tmp testing.go:153: 2022-06-22T14:53:07.220Z [INFO] reaping snapshot: path=/tmp/raft1347821793/snapshots/3-12-1655909587210 --- PASS: TestFileSS_Retention (0.02s) === RUN TestFileSS_BadPerm --- PASS: TestFileSS_BadPerm (0.00s) === RUN TestFileSS_MissingParentDir --- PASS: TestFileSS_MissingParentDir (0.00s) === RUN TestFileSS_Ordering testing.go:153: 2022-06-22T14:53:07.221Z [INFO] creating new snapshot: path=/tmp/raft2734941656/snapshots/5-130350-1655909587221.tmp testing.go:153: 2022-06-22T14:53:07.225Z [INFO] creating new snapshot: path=/tmp/raft2734941656/snapshots/36-204917-1655909587225.tmp --- PASS: TestFileSS_Ordering (0.01s) === RUN TestDeferFutureSuccess --- PASS: TestDeferFutureSuccess (0.00s) === RUN TestDeferFutureError --- PASS: TestDeferFutureError (0.00s) === RUN TestDeferFutureConcurrent --- PASS: TestDeferFutureConcurrent (0.00s) === RUN TestInmemSnapshotStoreImpl --- PASS: TestInmemSnapshotStoreImpl (0.00s) === RUN TestInmemSnapshotSinkImpl --- PASS: TestInmemSnapshotSinkImpl (0.00s) === RUN TestInmemSS_CreateSnapshot --- PASS: TestInmemSS_CreateSnapshot (0.00s) === RUN TestInmemSS_OpenSnapshotTwice --- PASS: TestInmemSS_OpenSnapshotTwice (0.00s) === RUN TestInmemTransportImpl --- PASS: TestInmemTransportImpl (0.00s) === RUN TestInmemTransportWriteTimeout --- PASS: TestInmemTransportWriteTimeout (0.01s) === RUN TestRaft_Integ --- SKIP: TestRaft_Integ (0.00s) === RUN TestLogCache --- PASS: TestLogCache (0.00s) === RUN TestNetworkTransport_CloseStreams --- PASS: TestNetworkTransport_CloseStreams (0.00s) === RUN TestNetworkTransport_StartStop --- PASS: TestNetworkTransport_StartStop (0.00s) === RUN TestNetworkTransport_Heartbeat_FastPath --- PASS: TestNetworkTransport_Heartbeat_FastPath (0.00s) === RUN TestNetworkTransport_AppendEntries --- PASS: TestNetworkTransport_AppendEntries (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline --- PASS: TestNetworkTransport_AppendEntriesPipeline (0.00s) === RUN TestNetworkTransport_AppendEntriesPipeline_CloseStreams net_transport_test.go:339: Disabled unreliable test --- SKIP: TestNetworkTransport_AppendEntriesPipeline_CloseStreams (0.00s) === RUN TestNetworkTransport_RequestVote --- PASS: TestNetworkTransport_RequestVote (0.00s) === RUN TestNetworkTransport_InstallSnapshot --- PASS: TestNetworkTransport_InstallSnapshot (0.00s) === RUN TestNetworkTransport_EncodeDecode --- PASS: TestNetworkTransport_EncodeDecode (0.00s) === RUN TestNetworkTransport_EncodeDecode_AddressProvider --- PASS: TestNetworkTransport_EncodeDecode_AddressProvider (0.00s) === RUN TestNetworkTransport_PooledConn --- PASS: TestNetworkTransport_PooledConn (0.00s) === RUN TestNetworkTransport_ListenBackoff net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.249Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.255Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.265Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.285Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.326Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.407Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.567Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:07.888Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:08.529Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:09.530Z [ERROR] test: failed to accept connection: error="intentional error in test" net_transport_test.go:718: countingWriter: 2022-06-22T14:53:10.530Z [ERROR] test: failed to accept connection: error="intentional error in test" --- PASS: TestNetworkTransport_ListenBackoff (4.00s) === RUN TestPeersJSON_BadConfiguration --- PASS: TestPeersJSON_BadConfiguration (0.00s) === RUN TestPeersJSON_ReadPeersJSON --- PASS: TestPeersJSON_ReadPeersJSON (0.00s) === RUN TestPeersJSON_ReadConfigJSON --- PASS: TestPeersJSON_ReadConfigJSON (0.00s) === RUN TestRaft_StartStop testLoggerAdapter verbose: server-d69e8909-03e7-d186-a6a8-3deebc48291e: 2022-06-22T14:53:11.253Z [INFO] server-d69e8909-03e7-d186-a6a8-3deebc48291e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d69e8909-03e7-d186-a6a8-3deebc48291e Address:d69e8909-03e7-d186-a6a8-3deebc48291e}]" testing.go:149: server-d69e8909-03e7-d186-a6a8-3deebc48291e: 2022-06-22T14:53:11.253Z [INFO] server-d69e8909-03e7-d186-a6a8-3deebc48291e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d69e8909-03e7-d186-a6a8-3deebc48291e Address:d69e8909-03e7-d186-a6a8-3deebc48291e}]" --- PASS: TestRaft_StartStop (0.00s) === RUN TestRaft_AfterShutdown testLoggerAdapter verbose: server-d49cb06d-bbf1-6b6a-0593-310a99fac922: 2022-06-22T14:53:11.253Z [INFO] server-d49cb06d-bbf1-6b6a-0593-310a99fac922: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d49cb06d-bbf1-6b6a-0593-310a99fac922 Address:d49cb06d-bbf1-6b6a-0593-310a99fac922}]" testing.go:149: server-d49cb06d-bbf1-6b6a-0593-310a99fac922: 2022-06-22T14:53:11.253Z [INFO] server-d49cb06d-bbf1-6b6a-0593-310a99fac922: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d49cb06d-bbf1-6b6a-0593-310a99fac922 Address:d49cb06d-bbf1-6b6a-0593-310a99fac922}]" --- PASS: TestRaft_AfterShutdown (0.00s) === RUN TestRaft_LiveBootstrap testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.254Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: initial configuration: index=0 servers=[] testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.254Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.254Z [INFO] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: initial configuration: index=0 servers=[] testing.go:149: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.254Z [INFO] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.254Z [INFO] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: initial configuration: index=0 servers=[] testing.go:149: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.254Z [INFO] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.254Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering follower state: follower="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Follower]" leader= testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.254Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering follower state: follower="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Follower]" leader= testLoggerAdapter verbose: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.254Z [INFO] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: entering follower state: follower="Node at 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb [Follower]" leader= testLoggerAdapter verbose: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.255Z [INFO] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: entering follower state: follower="Node at 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 [Follower]" leader= testing.go:149: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.254Z [INFO] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: entering follower state: follower="Node at 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb [Follower]" leader= testing.go:149: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.255Z [INFO] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: entering follower state: follower="Node at 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.255Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:11.255Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.334Z [WARN] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: no known peers, aborting election testing.go:149: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.334Z [WARN] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: no known peers, aborting election testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering candidate state: node="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Candidate]" term=2 testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering candidate state: node="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Candidate]" term=2 testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: election won: tally=2 testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: election won: tally=2 testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering leader state: leader="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Leader]" testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: entering leader state: leader="Node at b566670c-e3bd-49eb-833d-60f2584fc6df [Leader]" testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: added peer, starting replication: peer=server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: added peer, starting replication: peer=server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: added peer, starting replication: peer=server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: added peer, starting replication: peer=server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 testLoggerAdapter verbose: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.343Z [WARN] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: failed to get previous log: previous-index=1 last-index=0 error="log not found" testing.go:149: server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: 2022-06-22T14:53:11.343Z [WARN] server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.343Z [WARN] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: failed to get previous log: previous-index=1 last-index=0 error="log not found" testing.go:149: server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: 2022-06-22T14:53:11.343Z [WARN] server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb: failed to get previous log: previous-index=1 last-index=0 error="log not found" testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: appendEntries rejected, sending older logs: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" next=1 testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: appendEntries rejected, sending older logs: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" next=1 testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: appendEntries rejected, sending older logs: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" next=1 testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [WARN] server-b566670c-e3bd-49eb-833d-60f2584fc6df: appendEntries rejected, sending older logs: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" next=1 testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: pipelining replication: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: pipelining replication: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: pipelining replication: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.343Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: pipelining replication: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.449Z [INFO] stable state for Follower reached at 2022-06-22 14:53:11.343391073 +0000 UTC m=+4.158530629 (2 nodes), 88.047529ms from start of poll, 88.784893ms from cluster start. Timeout at 2022-06-22 14:53:11.449357462 +0000 UTC m=+4.264497018, 105.966389ms after stability testing.go:149: cluster: 2022-06-22T14:53:11.449Z [INFO] stable state for Follower reached at 2022-06-22 14:53:11.343391073 +0000 UTC m=+4.158530629 (2 nodes), 88.047529ms from start of poll, 88.784893ms from cluster start. Timeout at 2022-06-22 14:53:11.449357462 +0000 UTC m=+4.264497018, 105.966389ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.449Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:11.449Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.555Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.449649856 +0000 UTC m=+4.264789412 (1 nodes), 1.703µs from start of poll, 195.043676ms from cluster start. Timeout at 2022-06-22 14:53:11.555567804 +0000 UTC m=+4.370707351, 105.917939ms after stability testing.go:149: cluster: 2022-06-22T14:53:11.555Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.449649856 +0000 UTC m=+4.264789412 (1 nodes), 1.703µs from start of poll, 195.043676ms from cluster start. Timeout at 2022-06-22 14:53:11.555567804 +0000 UTC m=+4.370707351, 105.917939ms after stability testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.567Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: aborting pipeline replication: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.567Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: aborting pipeline replication: peer="{Voter server-73b4b6b8-97a3-d577-a1a6-5f44f7726ba4 73b4b6b8-97a3-d577-a1a6-5f44f7726ba4}" testLoggerAdapter verbose: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.567Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: aborting pipeline replication: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" testing.go:149: server-b566670c-e3bd-49eb-833d-60f2584fc6df: 2022-06-22T14:53:11.567Z [INFO] server-b566670c-e3bd-49eb-833d-60f2584fc6df: aborting pipeline replication: peer="{Voter server-0c2e52d2-1c2a-f5e4-2258-b043fa5441bb 0c2e52d2-1c2a-f5e4-2258-b043fa5441bb}" --- PASS: TestRaft_LiveBootstrap (0.31s) === RUN TestRaft_RecoverCluster_NoState testLoggerAdapter verbose: server-e392e82c-c965-4b8b-5dd9-7e7c5432d6b8: 2022-06-22T14:53:11.568Z [INFO] server-e392e82c-c965-4b8b-5dd9-7e7c5432d6b8: initial configuration: index=0 servers=[] testing.go:149: server-e392e82c-c965-4b8b-5dd9-7e7c5432d6b8: 2022-06-22T14:53:11.568Z [INFO] server-e392e82c-c965-4b8b-5dd9-7e7c5432d6b8: initial configuration: index=0 servers=[] --- PASS: TestRaft_RecoverCluster_NoState (0.00s) === RUN TestRaft_RecoverCluster raft_test.go:119: Disabled unreliable test --- SKIP: TestRaft_RecoverCluster (0.00s) === RUN TestRaft_HasExistingState testLoggerAdapter verbose: server-44944bce-9e7e-92a1-864e-08389e6e925a: 2022-06-22T14:53:11.569Z [INFO] server-44944bce-9e7e-92a1-864e-08389e6e925a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217}]" testing.go:149: server-44944bce-9e7e-92a1-864e-08389e6e925a: 2022-06-22T14:53:11.569Z [INFO] server-44944bce-9e7e-92a1-864e-08389e6e925a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217}]" testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.569Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217}]" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.569Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217}]" testLoggerAdapter verbose: server-44944bce-9e7e-92a1-864e-08389e6e925a: 2022-06-22T14:53:11.569Z [INFO] server-44944bce-9e7e-92a1-864e-08389e6e925a: entering follower state: follower="Node at 44944bce-9e7e-92a1-864e-08389e6e925a [Follower]" leader= testing.go:149: server-44944bce-9e7e-92a1-864e-08389e6e925a: 2022-06-22T14:53:11.569Z [INFO] server-44944bce-9e7e-92a1-864e-08389e6e925a: entering follower state: follower="Node at 44944bce-9e7e-92a1-864e-08389e6e925a [Follower]" leader= testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.569Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering follower state: follower="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Follower]" leader= testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.569Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering follower state: follower="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Follower]" leader= testLoggerAdapter verbose: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.569Z [INFO] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: initial configuration: index=0 servers=[] testing.go:149: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.569Z [INFO] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.570Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:11.570Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.570Z [INFO] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: entering follower state: follower="Node at d716ffe2-6962-0c90-5b9d-1abaaf42636d [Follower]" leader= testing.go:149: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.570Z [INFO] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: entering follower state: follower="Node at d716ffe2-6962-0c90-5b9d-1abaaf42636d [Follower]" leader= testLoggerAdapter verbose: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.623Z [WARN] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: no known peers, aborting election testing.go:149: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.623Z [WARN] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: no known peers, aborting election testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [WARN] server-9405be71-2113-5cc0-9af4-a91835cdf217: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [WARN] server-9405be71-2113-5cc0-9af4-a91835cdf217: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering candidate state: node="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Candidate]" term=2 testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering candidate state: node="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Candidate]" term=2 testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: election won: tally=2 testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: election won: tally=2 testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering leader state: leader="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Leader]" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: entering leader state: leader="Node at 9405be71-2113-5cc0-9af4-a91835cdf217 [Leader]" testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: added peer, starting replication: peer=server-44944bce-9e7e-92a1-864e-08389e6e925a testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.646Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: added peer, starting replication: peer=server-44944bce-9e7e-92a1-864e-08389e6e925a testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.647Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: pipelining replication: peer="{Voter server-44944bce-9e7e-92a1-864e-08389e6e925a 44944bce-9e7e-92a1-864e-08389e6e925a}" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.647Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: pipelining replication: peer="{Voter server-44944bce-9e7e-92a1-864e-08389e6e925a 44944bce-9e7e-92a1-864e-08389e6e925a}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.752Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.647084778 +0000 UTC m=+4.462224795 (1 nodes), 76.940536ms from start of poll, 77.667231ms from cluster start. Timeout at 2022-06-22 14:53:11.752820141 +0000 UTC m=+4.567959687, 105.734892ms after stability testing.go:149: cluster: 2022-06-22T14:53:11.752Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.647084778 +0000 UTC m=+4.462224795 (1 nodes), 76.940536ms from start of poll, 77.667231ms from cluster start. Timeout at 2022-06-22 14:53:11.752820141 +0000 UTC m=+4.567959687, 105.734892ms after stability testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.752Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: updating configuration: command=AddStaging server-id=server-d716ffe2-6962-0c90-5b9d-1abaaf42636d server-addr=d716ffe2-6962-0c90-5b9d-1abaaf42636d servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217} {Suffrage:Voter ID:server-d716ffe2-6962-0c90-5b9d-1abaaf42636d Address:d716ffe2-6962-0c90-5b9d-1abaaf42636d}]" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.752Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: updating configuration: command=AddStaging server-id=server-d716ffe2-6962-0c90-5b9d-1abaaf42636d server-addr=d716ffe2-6962-0c90-5b9d-1abaaf42636d servers="[{Suffrage:Voter ID:server-44944bce-9e7e-92a1-864e-08389e6e925a Address:44944bce-9e7e-92a1-864e-08389e6e925a} {Suffrage:Voter ID:server-9405be71-2113-5cc0-9af4-a91835cdf217 Address:9405be71-2113-5cc0-9af4-a91835cdf217} {Suffrage:Voter ID:server-d716ffe2-6962-0c90-5b9d-1abaaf42636d Address:d716ffe2-6962-0c90-5b9d-1abaaf42636d}]" testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: added peer, starting replication: peer=server-d716ffe2-6962-0c90-5b9d-1abaaf42636d testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: added peer, starting replication: peer=server-d716ffe2-6962-0c90-5b9d-1abaaf42636d testLoggerAdapter verbose: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.753Z [WARN] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: 2022-06-22T14:53:11.753Z [WARN] server-d716ffe2-6962-0c90-5b9d-1abaaf42636d: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [WARN] server-9405be71-2113-5cc0-9af4-a91835cdf217: appendEntries rejected, sending older logs: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" next=1 testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [WARN] server-9405be71-2113-5cc0-9af4-a91835cdf217: appendEntries rejected, sending older logs: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" next=1 testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: pipelining replication: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.753Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: pipelining replication: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.753Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:11.753Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.859Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.753398345 +0000 UTC m=+4.568537891 (1 nodes), 621ns from start of poll, 183.980327ms from cluster start. Timeout at 2022-06-22 14:53:11.859451038 +0000 UTC m=+4.674590604, 106.052713ms after stability testing.go:149: cluster: 2022-06-22T14:53:11.859Z [INFO] stable state for Leader reached at 2022-06-22 14:53:11.753398345 +0000 UTC m=+4.568537891 (1 nodes), 621ns from start of poll, 183.980327ms from cluster start. Timeout at 2022-06-22 14:53:11.859451038 +0000 UTC m=+4.674590604, 106.052713ms after stability testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.859Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: aborting pipeline replication: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.859Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: aborting pipeline replication: peer="{Voter server-d716ffe2-6962-0c90-5b9d-1abaaf42636d d716ffe2-6962-0c90-5b9d-1abaaf42636d}" testLoggerAdapter verbose: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.859Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: aborting pipeline replication: peer="{Voter server-44944bce-9e7e-92a1-864e-08389e6e925a 44944bce-9e7e-92a1-864e-08389e6e925a}" testing.go:149: server-9405be71-2113-5cc0-9af4-a91835cdf217: 2022-06-22T14:53:11.859Z [INFO] server-9405be71-2113-5cc0-9af4-a91835cdf217: aborting pipeline replication: peer="{Voter server-44944bce-9e7e-92a1-864e-08389e6e925a 44944bce-9e7e-92a1-864e-08389e6e925a}" --- PASS: TestRaft_HasExistingState (0.29s) === RUN TestRaft_SingleNode testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.860Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d54df8c9-065c-d78f-a950-ab0d4570661f Address:d54df8c9-065c-d78f-a950-ab0d4570661f}]" testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.860Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d54df8c9-065c-d78f-a950-ab0d4570661f Address:d54df8c9-065c-d78f-a950-ab0d4570661f}]" testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.860Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering follower state: follower="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Follower]" leader= testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.860Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering follower state: follower="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Follower]" leader= testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [WARN] server-d54df8c9-065c-d78f-a950-ab0d4570661f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [WARN] server-d54df8c9-065c-d78f-a950-ab0d4570661f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering candidate state: node="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Candidate]" term=2 testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering candidate state: node="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Candidate]" term=2 testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: election won: tally=1 testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: election won: tally=1 testLoggerAdapter verbose: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering leader state: leader="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Leader]" testing.go:149: server-d54df8c9-065c-d78f-a950-ab0d4570661f: 2022-06-22T14:53:11.958Z [INFO] server-d54df8c9-065c-d78f-a950-ab0d4570661f: entering leader state: leader="Node at d54df8c9-065c-d78f-a950-ab0d4570661f [Leader]" --- PASS: TestRaft_SingleNode (0.10s) === RUN TestRaft_TripleNode testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:11.960Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:11.960Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testLoggerAdapter verbose: server-d9732d01-6378-e7d1-559e-1bbcfc31b029: 2022-06-22T14:53:11.960Z [INFO] server-d9732d01-6378-e7d1-559e-1bbcfc31b029: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testing.go:149: server-d9732d01-6378-e7d1-559e-1bbcfc31b029: 2022-06-22T14:53:11.960Z [INFO] server-d9732d01-6378-e7d1-559e-1bbcfc31b029: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testLoggerAdapter verbose: server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: 2022-06-22T14:53:11.960Z [INFO] server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testing.go:149: server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: 2022-06-22T14:53:11.960Z [INFO] server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6503aa6c-38d6-311c-eb81-0fdd3258b36e Address:6503aa6c-38d6-311c-eb81-0fdd3258b36e} {Suffrage:Voter ID:server-d9732d01-6378-e7d1-559e-1bbcfc31b029 Address:d9732d01-6378-e7d1-559e-1bbcfc31b029} {Suffrage:Voter ID:server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 Address:aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}]" testLoggerAdapter verbose: server-d9732d01-6378-e7d1-559e-1bbcfc31b029: 2022-06-22T14:53:11.960Z [INFO] server-d9732d01-6378-e7d1-559e-1bbcfc31b029: entering follower state: follower="Node at d9732d01-6378-e7d1-559e-1bbcfc31b029 [Follower]" leader= testLoggerAdapter verbose: server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: 2022-06-22T14:53:11.960Z [INFO] server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: entering follower state: follower="Node at aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:11.960Z [INFO] starting stability test: raft-state=Follower testing.go:149: server-d9732d01-6378-e7d1-559e-1bbcfc31b029: 2022-06-22T14:53:11.960Z [INFO] server-d9732d01-6378-e7d1-559e-1bbcfc31b029: entering follower state: follower="Node at d9732d01-6378-e7d1-559e-1bbcfc31b029 [Follower]" leader= testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:11.960Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering follower state: follower="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Follower]" leader= testing.go:149: server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: 2022-06-22T14:53:11.960Z [INFO] server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62: entering follower state: follower="Node at aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:11.960Z [INFO] starting stability test: raft-state=Follower testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:11.960Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering follower state: follower="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Follower]" leader= testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [WARN] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [WARN] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering candidate state: node="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Candidate]" term=2 testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering candidate state: node="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Candidate]" term=2 testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: election won: tally=2 testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: election won: tally=2 testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering leader state: leader="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Leader]" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: entering leader state: leader="Node at 6503aa6c-38d6-311c-eb81-0fdd3258b36e [Leader]" testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: added peer, starting replication: peer=server-d9732d01-6378-e7d1-559e-1bbcfc31b029 testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: added peer, starting replication: peer=server-d9732d01-6378-e7d1-559e-1bbcfc31b029 testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: added peer, starting replication: peer=server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.013Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: added peer, starting replication: peer=server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.014Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: pipelining replication: peer="{Voter server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.014Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: pipelining replication: peer="{Voter server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}" testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.014Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: pipelining replication: peer="{Voter server-d9732d01-6378-e7d1-559e-1bbcfc31b029 d9732d01-6378-e7d1-559e-1bbcfc31b029}" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.014Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: pipelining replication: peer="{Voter server-d9732d01-6378-e7d1-559e-1bbcfc31b029 d9732d01-6378-e7d1-559e-1bbcfc31b029}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.119Z [INFO] stable state for Follower reached at 2022-06-22 14:53:12.013976629 +0000 UTC m=+4.829116175 (2 nodes), 52.927165ms from start of poll, 53.963326ms from cluster start. Timeout at 2022-06-22 14:53:12.119344591 +0000 UTC m=+4.934484157, 105.367982ms after stability testing.go:149: cluster: 2022-06-22T14:53:12.119Z [INFO] stable state for Follower reached at 2022-06-22 14:53:12.013976629 +0000 UTC m=+4.829116175 (2 nodes), 52.927165ms from start of poll, 53.963326ms from cluster start. Timeout at 2022-06-22 14:53:12.119344591 +0000 UTC m=+4.934484157, 105.367982ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.119Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:12.119Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.225Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.119541273 +0000 UTC m=+4.934680819 (1 nodes), 1.252µs from start of poll, 159.52797ms from cluster start. Timeout at 2022-06-22 14:53:12.225115917 +0000 UTC m=+5.040255473, 105.574654ms after stability testing.go:149: cluster: 2022-06-22T14:53:12.225Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.119541273 +0000 UTC m=+4.934680819 (1 nodes), 1.252µs from start of poll, 159.52797ms from cluster start. Timeout at 2022-06-22 14:53:12.225115917 +0000 UTC m=+5.040255473, 105.574654ms after stability testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.236Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: aborting pipeline replication: peer="{Voter server-d9732d01-6378-e7d1-559e-1bbcfc31b029 d9732d01-6378-e7d1-559e-1bbcfc31b029}" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.236Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: aborting pipeline replication: peer="{Voter server-d9732d01-6378-e7d1-559e-1bbcfc31b029 d9732d01-6378-e7d1-559e-1bbcfc31b029}" testLoggerAdapter verbose: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.236Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: aborting pipeline replication: peer="{Voter server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}" testing.go:149: server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: 2022-06-22T14:53:12.236Z [INFO] server-6503aa6c-38d6-311c-eb81-0fdd3258b36e: aborting pipeline replication: peer="{Voter server-aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62 aa36f18a-b0bf-3cf8-85f5-fa2fcb8b2a62}" --- PASS: TestRaft_TripleNode (0.28s) === RUN TestRaft_LeaderFail testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.239Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.239Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.239Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.239Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.239Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.239Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 Address:d5d5cd28-56db-8ef5-29ff-637bb2552d32} {Suffrage:Voter ID:server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 Address:6da7aeed-71c1-8970-1f2f-dde8a3d885e0} {Suffrage:Voter ID:server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 Address:5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.239Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.239Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering follower state: follower="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Follower]" leader= testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.239Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:12.239Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.239Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering follower state: follower="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Follower]" leader= testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.239Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering follower state: follower="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Follower]" leader= testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.239Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering follower state: follower="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Follower]" leader= testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.239Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader= testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.297Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.297Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=2 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=2 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: election won: tally=2 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: election won: tally=2 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering leader state: leader="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Leader]" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering leader state: leader="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Leader]" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: added peer, starting replication: peer=server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: added peer, starting replication: peer=server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: pipelining replication: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: pipelining replication: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.298Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.403Z [INFO] stable state for Follower reached at 2022-06-22 14:53:12.298449695 +0000 UTC m=+5.113589241 (2 nodes), 58.35485ms from start of poll, 59.381061ms from cluster start. Timeout at 2022-06-22 14:53:12.40391472 +0000 UTC m=+5.219054266, 105.465025ms after stability testing.go:149: cluster: 2022-06-22T14:53:12.403Z [INFO] stable state for Follower reached at 2022-06-22 14:53:12.298449695 +0000 UTC m=+5.113589241 (2 nodes), 58.35485ms from start of poll, 59.381061ms from cluster start. Timeout at 2022-06-22 14:53:12.40391472 +0000 UTC m=+5.219054266, 105.465025ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.404Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:12.404Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.509Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.404144896 +0000 UTC m=+5.219284442 (1 nodes), 1.462µs from start of poll, 165.076262ms from cluster start. Timeout at 2022-06-22 14:53:12.509446663 +0000 UTC m=+5.324586219, 105.301777ms after stability testing.go:149: cluster: 2022-06-22T14:53:12.509Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.404144896 +0000 UTC m=+5.219284442 (1 nodes), 1.462µs from start of poll, 165.076262ms from cluster start. Timeout at 2022-06-22 14:53:12.509446663 +0000 UTC m=+5.324586219, 105.301777ms after stability raft_test.go:336: [INFO] Disconnecting Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Leader] testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.522Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to pipeline appendEntries: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="append pipeline closed" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.522Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to pipeline appendEntries: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="append pipeline closed" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.523Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.523Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.526Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:12.526Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.526Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.526Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to pipeline appendEntries: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="append pipeline closed" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to pipeline appendEntries: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="append pipeline closed" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: aborting pipeline replication: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.527Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: aborting pipeline replication: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.532Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.532Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.534Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.534Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.543Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.543Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.546Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.546Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.548Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.548Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.551Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.551Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.559Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.559Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.565Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.565Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.567Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 time=50.995627ms testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.567Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 time=50.995627ms testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.568Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.568Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.573Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.573Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 time=61.955422ms testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 time=61.955422ms testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 time=59.052921ms testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact: server-id=server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 time=59.052921ms testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact quorum of nodes, stepping down testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader= testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.578Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader= testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: heartbeat timeout reached, starting election: last-leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: heartbeat timeout reached, starting election: last-leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=3 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=3 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.584Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.584Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: rejecting vote request since we have a leader: from=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.584Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: rejecting vote request since we have a leader: from=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.591Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.591Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to heartbeat to: peer=d5d5cd28-56db-8ef5-29ff-637bb2552d32 error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.596Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.596Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to appendEntries to: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.606Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: heartbeat timeout reached, starting election: last-leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.606Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: heartbeat timeout reached, starting election: last-leader=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.606Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering candidate state: node="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Candidate]" term=3 testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.606Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering candidate state: node="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Candidate]" term=3 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.606Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: duplicate requestVote for same term: term=3 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.606Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.607Z [ERROR] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.607Z [ERROR] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.667Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.667Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=3 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=3 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.668Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: Election timeout reached, restarting election testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: Election timeout reached, restarting election testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=4 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=4 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to make requestVote RPC: target="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.683Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering follower state: follower="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Follower]" leader= testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: election won: tally=2 testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.683Z [INFO] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: entering follower state: follower="Node at d5d5cd28-56db-8ef5-29ff-637bb2552d32 [Follower]" leader= testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.683Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: election won: tally=2 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering leader state: leader="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Leader]" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering leader state: leader="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Leader]" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.684Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.693Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.693Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.705Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.705Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.711Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.711Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.725Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.725Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: Election timeout reached, restarting election testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: Election timeout reached, restarting election testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=4 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=4 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2}" error="failed to connect to peer: 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.726Z [ERROR] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to make requestVote RPC: target="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" error="failed to connect to peer: d5d5cd28-56db-8ef5-29ff-637bb2552d32" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.732Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.732Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.735Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to contact: server-id=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 time=50.968015ms testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.735Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to contact: server-id=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 time=50.968015ms testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.751Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.751Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to appendEntries to: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.763Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.763Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="failed to connect to peer: 6da7aeed-71c1-8970-1f2f-dde8a3d885e0" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.784Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to contact: server-id=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 time=100.094075ms testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.784Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to contact: server-id=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 time=100.094075ms testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.789Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.684113149 +0000 UTC m=+5.499252715 (1 nodes), 157.776252ms from start of poll, 445.044535ms from cluster start. Timeout at 2022-06-22 14:53:12.78974997 +0000 UTC m=+5.604889516, 105.636801ms after stability testing.go:149: cluster: 2022-06-22T14:53:12.789Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.684113149 +0000 UTC m=+5.499252715 (1 nodes), 157.776252ms from start of poll, 445.044535ms from cluster start. Timeout at 2022-06-22 14:53:12.78974997 +0000 UTC m=+5.604889516, 105.636801ms after stability raft_test.go:371: [INFO] Reconnecting Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate] testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.792Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: Election timeout reached, restarting election testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.792Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: Election timeout reached, restarting election testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.793Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=5 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.793Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering candidate state: node="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Candidate]" term=5 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.793Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: rejecting vote request since we have a leader: from=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testLoggerAdapter verbose: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.793Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: rejecting vote request since we have a leader: from=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.793Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: rejecting vote request since we have a leader: from=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testing.go:149: server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: 2022-06-22T14:53:12.793Z [WARN] server-d5d5cd28-56db-8ef5-29ff-637bb2552d32: rejecting vote request since we have a leader: from=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.800Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: peer has newer term, stopping replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.800Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: peer has newer term, stopping replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.800Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering follower state: follower="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Follower]" leader= testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.800Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering follower state: follower="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Follower]" leader= testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.801Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.801Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=5 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering candidate state: node="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Candidate]" term=5 testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.861Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: duplicate requestVote for same term: term=5 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: election won: tally=2 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.861Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: duplicate requestVote for same term: term=5 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.861Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: election won: tally=2 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering leader state: leader="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Leader]" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: entering leader state: leader="Node at 5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 [Leader]" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: added peer, starting replication: peer=server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.862Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to get previous log: previous-index=5 last-index=3 error="log not found" testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.862Z [WARN] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: failed to get previous log: previous-index=5 last-index=3 error="log not found" testLoggerAdapter verbose: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.862Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testing.go:149: server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: 2022-06-22T14:53:12.862Z [INFO] server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0: entering follower state: follower="Node at 6da7aeed-71c1-8970-1f2f-dde8a3d885e0 [Follower]" leader=5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: appendEntries rejected, sending older logs: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" next=4 testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [WARN] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: appendEntries rejected, sending older logs: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" next=4 testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.862Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: pipelining replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.868Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.868Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-6da7aeed-71c1-8970-1f2f-dde8a3d885e0 6da7aeed-71c1-8970-1f2f-dde8a3d885e0}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.868Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.868Z [INFO] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: aborting pipeline replication: peer="{Voter server-d5d5cd28-56db-8ef5-29ff-637bb2552d32 d5d5cd28-56db-8ef5-29ff-637bb2552d32}" testLoggerAdapter verbose: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.918Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="command timed out" testing.go:149: server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: 2022-06-22T14:53:12.918Z [ERROR] server-5dd7c8ee-4651-58e2-ded1-2c2f1c2aafb2: failed to heartbeat to: peer=6da7aeed-71c1-8970-1f2f-dde8a3d885e0 error="command timed out" --- PASS: TestRaft_LeaderFail (0.68s) === RUN TestRaft_BehindFollower testLoggerAdapter verbose: server-a120ca87-a269-cc67-327c-7997f669a59a: 2022-06-22T14:53:12.920Z [INFO] server-a120ca87-a269-cc67-327c-7997f669a59a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testing.go:149: server-a120ca87-a269-cc67-327c-7997f669a59a: 2022-06-22T14:53:12.920Z [INFO] server-a120ca87-a269-cc67-327c-7997f669a59a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testLoggerAdapter verbose: server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: 2022-06-22T14:53:12.920Z [INFO] server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testing.go:149: server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: 2022-06-22T14:53:12.920Z [INFO] server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testLoggerAdapter verbose: server-a120ca87-a269-cc67-327c-7997f669a59a: 2022-06-22T14:53:12.920Z [INFO] server-a120ca87-a269-cc67-327c-7997f669a59a: entering follower state: follower="Node at a120ca87-a269-cc67-327c-7997f669a59a [Follower]" leader= testing.go:149: server-a120ca87-a269-cc67-327c-7997f669a59a: 2022-06-22T14:53:12.920Z [INFO] server-a120ca87-a269-cc67-327c-7997f669a59a: entering follower state: follower="Node at a120ca87-a269-cc67-327c-7997f669a59a [Follower]" leader= testLoggerAdapter verbose: server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: 2022-06-22T14:53:12.920Z [INFO] server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: entering follower state: follower="Node at 2414cd9c-e13e-02c7-93cf-ba15e771c6ce [Follower]" leader= testing.go:149: server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: 2022-06-22T14:53:12.920Z [INFO] server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce: entering follower state: follower="Node at 2414cd9c-e13e-02c7-93cf-ba15e771c6ce [Follower]" leader= testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.920Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.920Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a120ca87-a269-cc67-327c-7997f669a59a Address:a120ca87-a269-cc67-327c-7997f669a59a} {Suffrage:Voter ID:server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce Address:2414cd9c-e13e-02c7-93cf-ba15e771c6ce} {Suffrage:Voter ID:server-c35e0622-8b38-958a-f334-bd46939d3433 Address:c35e0622-8b38-958a-f334-bd46939d3433}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:12.920Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:12.920Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.920Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering follower state: follower="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Follower]" leader= testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.920Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering follower state: follower="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Follower]" leader= testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [WARN] server-c35e0622-8b38-958a-f334-bd46939d3433: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [WARN] server-c35e0622-8b38-958a-f334-bd46939d3433: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering candidate state: node="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Candidate]" term=2 testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering candidate state: node="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Candidate]" term=2 testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: election won: tally=2 testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: election won: tally=2 testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering leader state: leader="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Leader]" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: entering leader state: leader="Node at c35e0622-8b38-958a-f334-bd46939d3433 [Leader]" testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: added peer, starting replication: peer=server-a120ca87-a269-cc67-327c-7997f669a59a testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: added peer, starting replication: peer=server-a120ca87-a269-cc67-327c-7997f669a59a testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: added peer, starting replication: peer=server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: added peer, starting replication: peer=server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce 2414cd9c-e13e-02c7-93cf-ba15e771c6ce}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce 2414cd9c-e13e-02c7-93cf-ba15e771c6ce}" testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:12.980Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.085Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.980346894 +0000 UTC m=+5.795486440 (1 nodes), 59.629601ms from start of poll, 60.367318ms from cluster start. Timeout at 2022-06-22 14:53:13.085891457 +0000 UTC m=+5.901031013, 105.544573ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.085Z [INFO] stable state for Leader reached at 2022-06-22 14:53:12.980346894 +0000 UTC m=+5.795486440 (1 nodes), 59.629601ms from start of poll, 60.367318ms from cluster start. Timeout at 2022-06-22 14:53:13.085891457 +0000 UTC m=+5.901031013, 105.544573ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.085Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:13.085Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.192Z [INFO] stable state for Follower reached at 2022-06-22 14:53:13.086061719 +0000 UTC m=+5.901201275 (2 nodes), 49.764µs from start of poll, 166.082153ms from cluster start. Timeout at 2022-06-22 14:53:13.192111004 +0000 UTC m=+6.007250560, 106.049285ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.192Z [INFO] stable state for Follower reached at 2022-06-22 14:53:13.086061719 +0000 UTC m=+5.901201275 (2 nodes), 49.764µs from start of poll, 166.082153ms from cluster start. Timeout at 2022-06-22 14:53:13.192111004 +0000 UTC m=+6.007250560, 106.049285ms after stability testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [ERROR] server-c35e0622-8b38-958a-f334-bd46939d3433: failed to pipeline appendEntries: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" error="append pipeline closed" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [ERROR] server-c35e0622-8b38-958a-f334-bd46939d3433: failed to pipeline appendEntries: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" error="append pipeline closed" testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [ERROR] server-c35e0622-8b38-958a-f334-bd46939d3433: failed to appendEntries to: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" error="failed to connect to peer: a120ca87-a269-cc67-327c-7997f669a59a" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.192Z [ERROR] server-c35e0622-8b38-958a-f334-bd46939d3433: failed to appendEntries to: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" error="failed to connect to peer: a120ca87-a269-cc67-327c-7997f669a59a" raft_test.go:420: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.203Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.203Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: pipelining replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.211Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:13.211Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.317Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.211736754 +0000 UTC m=+6.026876300 (1 nodes), 1.473µs from start of poll, 291.757178ms from cluster start. Timeout at 2022-06-22 14:53:13.317139857 +0000 UTC m=+6.132279413, 105.403113ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.317Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.211736754 +0000 UTC m=+6.026876300 (1 nodes), 1.473µs from start of poll, 291.757178ms from cluster start. Timeout at 2022-06-22 14:53:13.317139857 +0000 UTC m=+6.132279413, 105.403113ms after stability testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.317Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce 2414cd9c-e13e-02c7-93cf-ba15e771c6ce}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.317Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-2414cd9c-e13e-02c7-93cf-ba15e771c6ce 2414cd9c-e13e-02c7-93cf-ba15e771c6ce}" testLoggerAdapter verbose: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.317Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" testing.go:149: server-c35e0622-8b38-958a-f334-bd46939d3433: 2022-06-22T14:53:13.317Z [INFO] server-c35e0622-8b38-958a-f334-bd46939d3433: aborting pipeline replication: peer="{Voter server-a120ca87-a269-cc67-327c-7997f669a59a a120ca87-a269-cc67-327c-7997f669a59a}" --- PASS: TestRaft_BehindFollower (0.40s) === RUN TestRaft_ApplyNonLeader testLoggerAdapter verbose: server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: 2022-06-22T14:53:13.319Z [INFO] server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testing.go:149: server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: 2022-06-22T14:53:13.319Z [INFO] server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testLoggerAdapter verbose: server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: 2022-06-22T14:53:13.319Z [INFO] server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: entering follower state: follower="Node at 18de4d7b-088c-9dc1-747c-ec8bb17e1865 [Follower]" leader= testing.go:149: server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: 2022-06-22T14:53:13.319Z [INFO] server-18de4d7b-088c-9dc1-747c-ec8bb17e1865: entering follower state: follower="Node at 18de4d7b-088c-9dc1-747c-ec8bb17e1865 [Follower]" leader= testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.319Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.319Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.319Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering follower state: follower="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Follower]" leader= testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.319Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering follower state: follower="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Follower]" leader= testLoggerAdapter verbose: server-18366c24-e708-e07e-e808-d895fa5be503: 2022-06-22T14:53:13.319Z [INFO] server-18366c24-e708-e07e-e808-d895fa5be503: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testing.go:149: server-18366c24-e708-e07e-e808-d895fa5be503: 2022-06-22T14:53:13.319Z [INFO] server-18366c24-e708-e07e-e808-d895fa5be503: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 Address:18de4d7b-088c-9dc1-747c-ec8bb17e1865} {Suffrage:Voter ID:server-2596704c-b4a1-77fa-1992-81c5b66966a5 Address:2596704c-b4a1-77fa-1992-81c5b66966a5} {Suffrage:Voter ID:server-18366c24-e708-e07e-e808-d895fa5be503 Address:18366c24-e708-e07e-e808-d895fa5be503}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.319Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:13.319Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-18366c24-e708-e07e-e808-d895fa5be503: 2022-06-22T14:53:13.319Z [INFO] server-18366c24-e708-e07e-e808-d895fa5be503: entering follower state: follower="Node at 18366c24-e708-e07e-e808-d895fa5be503 [Follower]" leader= testing.go:149: server-18366c24-e708-e07e-e808-d895fa5be503: 2022-06-22T14:53:13.319Z [INFO] server-18366c24-e708-e07e-e808-d895fa5be503: entering follower state: follower="Node at 18366c24-e708-e07e-e808-d895fa5be503 [Follower]" leader= testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [WARN] server-2596704c-b4a1-77fa-1992-81c5b66966a5: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [WARN] server-2596704c-b4a1-77fa-1992-81c5b66966a5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering candidate state: node="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Candidate]" term=2 testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering candidate state: node="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Candidate]" term=2 testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: election won: tally=2 testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: election won: tally=2 testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering leader state: leader="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Leader]" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: entering leader state: leader="Node at 2596704c-b4a1-77fa-1992-81c5b66966a5 [Leader]" testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: added peer, starting replication: peer=server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: added peer, starting replication: peer=server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: added peer, starting replication: peer=server-18366c24-e708-e07e-e808-d895fa5be503 testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: added peer, starting replication: peer=server-18366c24-e708-e07e-e808-d895fa5be503 testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: pipelining replication: peer="{Voter server-18366c24-e708-e07e-e808-d895fa5be503 18366c24-e708-e07e-e808-d895fa5be503}" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: pipelining replication: peer="{Voter server-18366c24-e708-e07e-e808-d895fa5be503 18366c24-e708-e07e-e808-d895fa5be503}" testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: pipelining replication: peer="{Voter server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 18de4d7b-088c-9dc1-747c-ec8bb17e1865}" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.375Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: pipelining replication: peer="{Voter server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 18de4d7b-088c-9dc1-747c-ec8bb17e1865}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.481Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.375592519 +0000 UTC m=+6.190732065 (1 nodes), 55.774635ms from start of poll, 56.57497ms from cluster start. Timeout at 2022-06-22 14:53:13.481301491 +0000 UTC m=+6.296441057, 105.708992ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.481Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.375592519 +0000 UTC m=+6.190732065 (1 nodes), 55.774635ms from start of poll, 56.57497ms from cluster start. Timeout at 2022-06-22 14:53:13.481301491 +0000 UTC m=+6.296441057, 105.708992ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.481Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:13.481Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.587Z [INFO] stable state for Follower reached at 2022-06-22 14:53:13.481531176 +0000 UTC m=+6.296670722 (2 nodes), 1.903µs from start of poll, 162.513627ms from cluster start. Timeout at 2022-06-22 14:53:13.587612011 +0000 UTC m=+6.402751568, 106.080846ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.587Z [INFO] stable state for Follower reached at 2022-06-22 14:53:13.481531176 +0000 UTC m=+6.296670722 (2 nodes), 1.903µs from start of poll, 162.513627ms from cluster start. Timeout at 2022-06-22 14:53:13.587612011 +0000 UTC m=+6.402751568, 106.080846ms after stability testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.587Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: aborting pipeline replication: peer="{Voter server-18366c24-e708-e07e-e808-d895fa5be503 18366c24-e708-e07e-e808-d895fa5be503}" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.587Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: aborting pipeline replication: peer="{Voter server-18366c24-e708-e07e-e808-d895fa5be503 18366c24-e708-e07e-e808-d895fa5be503}" testLoggerAdapter verbose: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.588Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: aborting pipeline replication: peer="{Voter server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 18de4d7b-088c-9dc1-747c-ec8bb17e1865}" testing.go:149: server-2596704c-b4a1-77fa-1992-81c5b66966a5: 2022-06-22T14:53:13.588Z [INFO] server-2596704c-b4a1-77fa-1992-81c5b66966a5: aborting pipeline replication: peer="{Voter server-18de4d7b-088c-9dc1-747c-ec8bb17e1865 18de4d7b-088c-9dc1-747c-ec8bb17e1865}" --- PASS: TestRaft_ApplyNonLeader (0.27s) === RUN TestRaft_ApplyConcurrent testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.589Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.589Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testLoggerAdapter verbose: server-a03f8a13-211b-ec1c-e346-1b770b868fcb: 2022-06-22T14:53:13.589Z [INFO] server-a03f8a13-211b-ec1c-e346-1b770b868fcb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testing.go:149: server-a03f8a13-211b-ec1c-e346-1b770b868fcb: 2022-06-22T14:53:13.589Z [INFO] server-a03f8a13-211b-ec1c-e346-1b770b868fcb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.589Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering follower state: follower="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Follower]" leader= testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.589Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering follower state: follower="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Follower]" leader= testLoggerAdapter verbose: server-a03f8a13-211b-ec1c-e346-1b770b868fcb: 2022-06-22T14:53:13.589Z [INFO] server-a03f8a13-211b-ec1c-e346-1b770b868fcb: entering follower state: follower="Node at a03f8a13-211b-ec1c-e346-1b770b868fcb [Follower]" leader= testing.go:149: server-a03f8a13-211b-ec1c-e346-1b770b868fcb: 2022-06-22T14:53:13.589Z [INFO] server-a03f8a13-211b-ec1c-e346-1b770b868fcb: entering follower state: follower="Node at a03f8a13-211b-ec1c-e346-1b770b868fcb [Follower]" leader= testLoggerAdapter verbose: server-4703c091-b304-c2d5-d977-4d102b11f982: 2022-06-22T14:53:13.589Z [INFO] server-4703c091-b304-c2d5-d977-4d102b11f982: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testing.go:149: server-4703c091-b304-c2d5-d977-4d102b11f982: 2022-06-22T14:53:13.589Z [INFO] server-4703c091-b304-c2d5-d977-4d102b11f982: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6fef7afa-8db3-dd16-5761-c32fee94cd88 Address:6fef7afa-8db3-dd16-5761-c32fee94cd88} {Suffrage:Voter ID:server-a03f8a13-211b-ec1c-e346-1b770b868fcb Address:a03f8a13-211b-ec1c-e346-1b770b868fcb} {Suffrage:Voter ID:server-4703c091-b304-c2d5-d977-4d102b11f982 Address:4703c091-b304-c2d5-d977-4d102b11f982}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.589Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:13.589Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-4703c091-b304-c2d5-d977-4d102b11f982: 2022-06-22T14:53:13.589Z [INFO] server-4703c091-b304-c2d5-d977-4d102b11f982: entering follower state: follower="Node at 4703c091-b304-c2d5-d977-4d102b11f982 [Follower]" leader= testing.go:149: server-4703c091-b304-c2d5-d977-4d102b11f982: 2022-06-22T14:53:13.589Z [INFO] server-4703c091-b304-c2d5-d977-4d102b11f982: entering follower state: follower="Node at 4703c091-b304-c2d5-d977-4d102b11f982 [Follower]" leader= testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.734Z [WARN] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.734Z [WARN] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering candidate state: node="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Candidate]" term=2 testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering candidate state: node="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Candidate]" term=2 testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: election won: tally=2 testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: election won: tally=2 testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering leader state: leader="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Leader]" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: entering leader state: leader="Node at 6fef7afa-8db3-dd16-5761-c32fee94cd88 [Leader]" testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: added peer, starting replication: peer=server-a03f8a13-211b-ec1c-e346-1b770b868fcb testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: added peer, starting replication: peer=server-a03f8a13-211b-ec1c-e346-1b770b868fcb testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: added peer, starting replication: peer=server-4703c091-b304-c2d5-d977-4d102b11f982 testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: added peer, starting replication: peer=server-4703c091-b304-c2d5-d977-4d102b11f982 testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: pipelining replication: peer="{Voter server-4703c091-b304-c2d5-d977-4d102b11f982 4703c091-b304-c2d5-d977-4d102b11f982}" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: pipelining replication: peer="{Voter server-4703c091-b304-c2d5-d977-4d102b11f982 4703c091-b304-c2d5-d977-4d102b11f982}" testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: pipelining replication: peer="{Voter server-a03f8a13-211b-ec1c-e346-1b770b868fcb a03f8a13-211b-ec1c-e346-1b770b868fcb}" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.735Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: pipelining replication: peer="{Voter server-a03f8a13-211b-ec1c-e346-1b770b868fcb a03f8a13-211b-ec1c-e346-1b770b868fcb}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.941Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.735545693 +0000 UTC m=+6.550685249 (1 nodes), 146.11995ms from start of poll, 146.582104ms from cluster start. Timeout at 2022-06-22 14:53:13.940974088 +0000 UTC m=+6.756113644, 205.428395ms after stability testing.go:149: cluster: 2022-06-22T14:53:13.941Z [INFO] stable state for Leader reached at 2022-06-22 14:53:13.735545693 +0000 UTC m=+6.550685249 (1 nodes), 146.11995ms from start of poll, 146.582104ms from cluster start. Timeout at 2022-06-22 14:53:13.940974088 +0000 UTC m=+6.756113644, 205.428395ms after stability testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.960Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: aborting pipeline replication: peer="{Voter server-4703c091-b304-c2d5-d977-4d102b11f982 4703c091-b304-c2d5-d977-4d102b11f982}" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.960Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: aborting pipeline replication: peer="{Voter server-4703c091-b304-c2d5-d977-4d102b11f982 4703c091-b304-c2d5-d977-4d102b11f982}" testLoggerAdapter verbose: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.960Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: aborting pipeline replication: peer="{Voter server-a03f8a13-211b-ec1c-e346-1b770b868fcb a03f8a13-211b-ec1c-e346-1b770b868fcb}" testing.go:149: server-6fef7afa-8db3-dd16-5761-c32fee94cd88: 2022-06-22T14:53:13.960Z [INFO] server-6fef7afa-8db3-dd16-5761-c32fee94cd88: aborting pipeline replication: peer="{Voter server-a03f8a13-211b-ec1c-e346-1b770b868fcb a03f8a13-211b-ec1c-e346-1b770b868fcb}" --- PASS: TestRaft_ApplyConcurrent (0.37s) === RUN TestRaft_ApplyConcurrent_Timeout testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:13.961Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a883538-76e7-4532-f966-a85cbb9a52a4 Address:1a883538-76e7-4532-f966-a85cbb9a52a4}]" testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:13.961Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a883538-76e7-4532-f966-a85cbb9a52a4 Address:1a883538-76e7-4532-f966-a85cbb9a52a4}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:13.961Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:13.961Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:13.961Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering follower state: follower="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Follower]" leader= testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:13.961Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering follower state: follower="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Follower]" leader= testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [WARN] server-1a883538-76e7-4532-f966-a85cbb9a52a4: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [WARN] server-1a883538-76e7-4532-f966-a85cbb9a52a4: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering candidate state: node="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Candidate]" term=2 testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering candidate state: node="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Candidate]" term=2 testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: election won: tally=1 testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: election won: tally=1 testLoggerAdapter verbose: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering leader state: leader="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Leader]" testing.go:149: server-1a883538-76e7-4532-f966-a85cbb9a52a4: 2022-06-22T14:53:14.145Z [INFO] server-1a883538-76e7-4532-f966-a85cbb9a52a4: entering leader state: leader="Node at 1a883538-76e7-4532-f966-a85cbb9a52a4 [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.346Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.145520886 +0000 UTC m=+6.960660432 (1 nodes), 183.734004ms from start of poll, 183.951515ms from cluster start. Timeout at 2022-06-22 14:53:14.346808546 +0000 UTC m=+7.161948103, 201.287671ms after stability testing.go:149: cluster: 2022-06-22T14:53:14.346Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.145520886 +0000 UTC m=+6.960660432 (1 nodes), 183.734004ms from start of poll, 183.951515ms from cluster start. Timeout at 2022-06-22 14:53:14.346808546 +0000 UTC m=+7.161948103, 201.287671ms after stability --- PASS: TestRaft_ApplyConcurrent_Timeout (0.39s) === RUN TestRaft_JoinNode testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.348Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301}]" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.348Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301}]" testLoggerAdapter verbose: server-82dece42-c017-c590-36db-082cbffe2301: 2022-06-22T14:53:14.348Z [INFO] server-82dece42-c017-c590-36db-082cbffe2301: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301}]" testing.go:149: server-82dece42-c017-c590-36db-082cbffe2301: 2022-06-22T14:53:14.348Z [INFO] server-82dece42-c017-c590-36db-082cbffe2301: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301}]" testLoggerAdapter verbose: server-82dece42-c017-c590-36db-082cbffe2301: 2022-06-22T14:53:14.348Z [INFO] server-82dece42-c017-c590-36db-082cbffe2301: entering follower state: follower="Node at 82dece42-c017-c590-36db-082cbffe2301 [Follower]" leader= testing.go:149: server-82dece42-c017-c590-36db-082cbffe2301: 2022-06-22T14:53:14.348Z [INFO] server-82dece42-c017-c590-36db-082cbffe2301: entering follower state: follower="Node at 82dece42-c017-c590-36db-082cbffe2301 [Follower]" leader= testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.348Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering follower state: follower="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Follower]" leader= testLoggerAdapter verbose: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.348Z [INFO] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: initial configuration: index=0 servers=[] testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.348Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering follower state: follower="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Follower]" leader= testing.go:149: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.348Z [INFO] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.348Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:14.348Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.349Z [INFO] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: entering follower state: follower="Node at 30d5972d-3ad6-ffd4-92d2-cc49f61fa895 [Follower]" leader= testing.go:149: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.349Z [INFO] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: entering follower state: follower="Node at 30d5972d-3ad6-ffd4-92d2-cc49f61fa895 [Follower]" leader= testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [WARN] server-c102e103-2636-b824-d33e-8238efa0aa0a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [WARN] server-c102e103-2636-b824-d33e-8238efa0aa0a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering candidate state: node="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Candidate]" term=2 testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering candidate state: node="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Candidate]" term=2 testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: election won: tally=2 testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: election won: tally=2 testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering leader state: leader="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Leader]" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: entering leader state: leader="Node at c102e103-2636-b824-d33e-8238efa0aa0a [Leader]" testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: added peer, starting replication: peer=server-82dece42-c017-c590-36db-082cbffe2301 testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: added peer, starting replication: peer=server-82dece42-c017-c590-36db-082cbffe2301 testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: pipelining replication: peer="{Voter server-82dece42-c017-c590-36db-082cbffe2301 82dece42-c017-c590-36db-082cbffe2301}" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.402Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: pipelining replication: peer="{Voter server-82dece42-c017-c590-36db-082cbffe2301 82dece42-c017-c590-36db-082cbffe2301}" testLoggerAdapter verbose: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.415Z [WARN] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: no known peers, aborting election testing.go:149: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.415Z [WARN] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: no known peers, aborting election testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.507Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.402396536 +0000 UTC m=+7.217536092 (1 nodes), 53.414959ms from start of poll, 54.193892ms from cluster start. Timeout at 2022-06-22 14:53:14.507852363 +0000 UTC m=+7.322991909, 105.455817ms after stability testing.go:149: cluster: 2022-06-22T14:53:14.507Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.402396536 +0000 UTC m=+7.217536092 (1 nodes), 53.414959ms from start of poll, 54.193892ms from cluster start. Timeout at 2022-06-22 14:53:14.507852363 +0000 UTC m=+7.322991909, 105.455817ms after stability testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: updating configuration: command=AddStaging server-id=server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 server-addr=30d5972d-3ad6-ffd4-92d2-cc49f61fa895 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301} {Suffrage:Voter ID:server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 Address:30d5972d-3ad6-ffd4-92d2-cc49f61fa895}]" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: updating configuration: command=AddStaging server-id=server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 server-addr=30d5972d-3ad6-ffd4-92d2-cc49f61fa895 servers="[{Suffrage:Voter ID:server-c102e103-2636-b824-d33e-8238efa0aa0a Address:c102e103-2636-b824-d33e-8238efa0aa0a} {Suffrage:Voter ID:server-82dece42-c017-c590-36db-082cbffe2301 Address:82dece42-c017-c590-36db-082cbffe2301} {Suffrage:Voter ID:server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 Address:30d5972d-3ad6-ffd4-92d2-cc49f61fa895}]" testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: added peer, starting replication: peer=server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: added peer, starting replication: peer=server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 testLoggerAdapter verbose: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.508Z [WARN] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: 2022-06-22T14:53:14.508Z [WARN] server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [WARN] server-c102e103-2636-b824-d33e-8238efa0aa0a: appendEntries rejected, sending older logs: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" next=1 testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [WARN] server-c102e103-2636-b824-d33e-8238efa0aa0a: appendEntries rejected, sending older logs: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" next=1 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.508Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:14.508Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: pipelining replication: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.508Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: pipelining replication: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.613Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.508530857 +0000 UTC m=+7.323670403 (1 nodes), 2.435µs from start of poll, 160.328203ms from cluster start. Timeout at 2022-06-22 14:53:14.613725591 +0000 UTC m=+7.428865137, 105.194734ms after stability testing.go:149: cluster: 2022-06-22T14:53:14.613Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.508530857 +0000 UTC m=+7.323670403 (1 nodes), 2.435µs from start of poll, 160.328203ms from cluster start. Timeout at 2022-06-22 14:53:14.613725591 +0000 UTC m=+7.428865137, 105.194734ms after stability testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.614Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: aborting pipeline replication: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.614Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: aborting pipeline replication: peer="{Voter server-30d5972d-3ad6-ffd4-92d2-cc49f61fa895 30d5972d-3ad6-ffd4-92d2-cc49f61fa895}" testLoggerAdapter verbose: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.614Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: aborting pipeline replication: peer="{Voter server-82dece42-c017-c590-36db-082cbffe2301 82dece42-c017-c590-36db-082cbffe2301}" testing.go:149: server-c102e103-2636-b824-d33e-8238efa0aa0a: 2022-06-22T14:53:14.614Z [INFO] server-c102e103-2636-b824-d33e-8238efa0aa0a: aborting pipeline replication: peer="{Voter server-82dece42-c017-c590-36db-082cbffe2301 82dece42-c017-c590-36db-082cbffe2301}" --- PASS: TestRaft_JoinNode (0.27s) === RUN TestRaft_JoinNode_ConfigStore testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.615Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141}]" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.615Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141}]" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.615Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering follower state: follower="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Follower]" leader= testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.615Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering follower state: follower="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Follower]" leader= testLoggerAdapter verbose: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.615Z [INFO] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: initial configuration: index=0 servers=[] testing.go:149: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.615Z [INFO] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: initial configuration: index=0 servers=[] testLoggerAdapter verbose: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.615Z [INFO] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: entering follower state: follower="Node at f21e86e2-f2a5-6e37-3d1a-92cef1601fde [Follower]" leader= testing.go:149: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.615Z [INFO] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: entering follower state: follower="Node at f21e86e2-f2a5-6e37-3d1a-92cef1601fde [Follower]" leader= testLoggerAdapter verbose: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.615Z [INFO] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: initial configuration: index=0 servers=[] testing.go:149: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.615Z [INFO] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.616Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:14.616Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.616Z [INFO] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: entering follower state: follower="Node at 7dba8eaa-c169-edf6-adbd-bc42bc777116 [Follower]" leader= testing.go:149: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.616Z [INFO] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: entering follower state: follower="Node at 7dba8eaa-c169-edf6-adbd-bc42bc777116 [Follower]" leader= testLoggerAdapter verbose: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.686Z [WARN] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: no known peers, aborting election testing.go:149: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.686Z [WARN] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: no known peers, aborting election testLoggerAdapter verbose: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.686Z [WARN] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: no known peers, aborting election testing.go:149: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.686Z [WARN] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: no known peers, aborting election testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering candidate state: node="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Candidate]" term=2 testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering candidate state: node="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Candidate]" term=2 testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: election won: tally=1 testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: election won: tally=1 testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering leader state: leader="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Leader]" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.716Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: entering leader state: leader="Node at a3851f34-ff82-2661-6ebe-ab62080c4141 [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.821Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.716431717 +0000 UTC m=+7.531571263 (1 nodes), 100.341673ms from start of poll, 101.224953ms from cluster start. Timeout at 2022-06-22 14:53:14.821795762 +0000 UTC m=+7.636935328, 105.364065ms after stability testing.go:149: cluster: 2022-06-22T14:53:14.821Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.716431717 +0000 UTC m=+7.531571263 (1 nodes), 100.341673ms from start of poll, 101.224953ms from cluster start. Timeout at 2022-06-22 14:53:14.821795762 +0000 UTC m=+7.636935328, 105.364065ms after stability testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.821Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: updating configuration: command=AddStaging server-id=server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde server-addr=f21e86e2-f2a5-6e37-3d1a-92cef1601fde servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141} {Suffrage:Voter ID:server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde Address:f21e86e2-f2a5-6e37-3d1a-92cef1601fde}]" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.821Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: updating configuration: command=AddStaging server-id=server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde server-addr=f21e86e2-f2a5-6e37-3d1a-92cef1601fde servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141} {Suffrage:Voter ID:server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde Address:f21e86e2-f2a5-6e37-3d1a-92cef1601fde}]" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: added peer, starting replication: peer=server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: added peer, starting replication: peer=server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.822Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:14.822Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.822Z [WARN] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: 2022-06-22T14:53:14.822Z [WARN] server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: appendEntries rejected, sending older logs: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" next=1 testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: appendEntries rejected, sending older logs: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" next=1 testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: pipelining replication: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.822Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: pipelining replication: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.927Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.822211808 +0000 UTC m=+7.637351354 (1 nodes), 1.282µs from start of poll, 207.005044ms from cluster start. Timeout at 2022-06-22 14:53:14.927749982 +0000 UTC m=+7.742889528, 105.538174ms after stability testing.go:149: cluster: 2022-06-22T14:53:14.927Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.822211808 +0000 UTC m=+7.637351354 (1 nodes), 1.282µs from start of poll, 207.005044ms from cluster start. Timeout at 2022-06-22 14:53:14.927749982 +0000 UTC m=+7.742889528, 105.538174ms after stability testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.927Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: updating configuration: command=AddStaging server-id=server-7dba8eaa-c169-edf6-adbd-bc42bc777116 server-addr=7dba8eaa-c169-edf6-adbd-bc42bc777116 servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141} {Suffrage:Voter ID:server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde Address:f21e86e2-f2a5-6e37-3d1a-92cef1601fde} {Suffrage:Voter ID:server-7dba8eaa-c169-edf6-adbd-bc42bc777116 Address:7dba8eaa-c169-edf6-adbd-bc42bc777116}]" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.927Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: updating configuration: command=AddStaging server-id=server-7dba8eaa-c169-edf6-adbd-bc42bc777116 server-addr=7dba8eaa-c169-edf6-adbd-bc42bc777116 servers="[{Suffrage:Voter ID:server-a3851f34-ff82-2661-6ebe-ab62080c4141 Address:a3851f34-ff82-2661-6ebe-ab62080c4141} {Suffrage:Voter ID:server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde Address:f21e86e2-f2a5-6e37-3d1a-92cef1601fde} {Suffrage:Voter ID:server-7dba8eaa-c169-edf6-adbd-bc42bc777116 Address:7dba8eaa-c169-edf6-adbd-bc42bc777116}]" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: added peer, starting replication: peer=server-7dba8eaa-c169-edf6-adbd-bc42bc777116 testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: added peer, starting replication: peer=server-7dba8eaa-c169-edf6-adbd-bc42bc777116 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:14.928Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:14.928Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.928Z [WARN] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: failed to get previous log: previous-index=4 last-index=0 error="log not found" testing.go:149: server-7dba8eaa-c169-edf6-adbd-bc42bc777116: 2022-06-22T14:53:14.928Z [WARN] server-7dba8eaa-c169-edf6-adbd-bc42bc777116: failed to get previous log: previous-index=4 last-index=0 error="log not found" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: appendEntries rejected, sending older logs: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" next=1 testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [WARN] server-a3851f34-ff82-2661-6ebe-ab62080c4141: appendEntries rejected, sending older logs: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" next=1 testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: pipelining replication: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:14.928Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: pipelining replication: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.033Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.928211666 +0000 UTC m=+7.743351212 (1 nodes), 1.182µs from start of poll, 313.004902ms from cluster start. Timeout at 2022-06-22 14:53:15.033675558 +0000 UTC m=+7.848815104, 105.463892ms after stability testing.go:149: cluster: 2022-06-22T14:53:15.033Z [INFO] stable state for Leader reached at 2022-06-22 14:53:14.928211666 +0000 UTC m=+7.743351212 (1 nodes), 1.182µs from start of poll, 313.004902ms from cluster start. Timeout at 2022-06-22 14:53:15.033675558 +0000 UTC m=+7.848815104, 105.463892ms after stability testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:15.034Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: aborting pipeline replication: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:15.034Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: aborting pipeline replication: peer="{Voter server-7dba8eaa-c169-edf6-adbd-bc42bc777116 7dba8eaa-c169-edf6-adbd-bc42bc777116}" testLoggerAdapter verbose: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:15.034Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: aborting pipeline replication: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" testing.go:149: server-a3851f34-ff82-2661-6ebe-ab62080c4141: 2022-06-22T14:53:15.034Z [INFO] server-a3851f34-ff82-2661-6ebe-ab62080c4141: aborting pipeline replication: peer="{Voter server-f21e86e2-f2a5-6e37-3d1a-92cef1601fde f21e86e2-f2a5-6e37-3d1a-92cef1601fde}" --- PASS: TestRaft_JoinNode_ConfigStore (0.42s) === RUN TestRaft_RemoveFollower testLoggerAdapter verbose: server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: 2022-06-22T14:53:15.035Z [INFO] server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testing.go:149: server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: 2022-06-22T14:53:15.035Z [INFO] server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testLoggerAdapter verbose: server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: 2022-06-22T14:53:15.035Z [INFO] server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testing.go:149: server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: 2022-06-22T14:53:15.035Z [INFO] server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testLoggerAdapter verbose: server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: 2022-06-22T14:53:15.035Z [INFO] server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: entering follower state: follower="Node at fb5a2ee0-32f7-b335-f843-1ff32bbd8cca [Follower]" leader= testLoggerAdapter verbose: server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: 2022-06-22T14:53:15.035Z [INFO] server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: entering follower state: follower="Node at c8419d02-14a7-4cdd-aefb-97f51a41a15e [Follower]" leader= testing.go:149: server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: 2022-06-22T14:53:15.035Z [INFO] server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca: entering follower state: follower="Node at fb5a2ee0-32f7-b335-f843-1ff32bbd8cca [Follower]" leader= testing.go:149: server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: 2022-06-22T14:53:15.035Z [INFO] server-c8419d02-14a7-4cdd-aefb-97f51a41a15e: entering follower state: follower="Node at c8419d02-14a7-4cdd-aefb-97f51a41a15e [Follower]" leader= testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.035Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.035Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca Address:fb5a2ee0-32f7-b335-f843-1ff32bbd8cca} {Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.036Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:15.036Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.036Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering follower state: follower="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Follower]" leader= testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.036Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering follower state: follower="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Follower]" leader= testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [WARN] server-99e42361-b8e9-8075-0081-3d048dee2e98: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [WARN] server-99e42361-b8e9-8075-0081-3d048dee2e98: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering candidate state: node="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Candidate]" term=2 testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering candidate state: node="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Candidate]" term=2 testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: election won: tally=2 testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: election won: tally=2 testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering leader state: leader="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Leader]" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.089Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: entering leader state: leader="Node at 99e42361-b8e9-8075-0081-3d048dee2e98 [Leader]" testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: added peer, starting replication: peer=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: added peer, starting replication: peer=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: added peer, starting replication: peer=server-c8419d02-14a7-4cdd-aefb-97f51a41a15e testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: added peer, starting replication: peer=server-c8419d02-14a7-4cdd-aefb-97f51a41a15e testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: pipelining replication: peer="{Voter server-c8419d02-14a7-4cdd-aefb-97f51a41a15e c8419d02-14a7-4cdd-aefb-97f51a41a15e}" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: pipelining replication: peer="{Voter server-c8419d02-14a7-4cdd-aefb-97f51a41a15e c8419d02-14a7-4cdd-aefb-97f51a41a15e}" testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: pipelining replication: peer="{Voter server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca fb5a2ee0-32f7-b335-f843-1ff32bbd8cca}" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.090Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: pipelining replication: peer="{Voter server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca fb5a2ee0-32f7-b335-f843-1ff32bbd8cca}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.196Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.090051214 +0000 UTC m=+7.905190770 (1 nodes), 53.944127ms from start of poll, 54.582204ms from cluster start. Timeout at 2022-06-22 14:53:15.196080582 +0000 UTC m=+8.011220138, 106.029368ms after stability testing.go:149: cluster: 2022-06-22T14:53:15.196Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.090051214 +0000 UTC m=+7.905190770 (1 nodes), 53.944127ms from start of poll, 54.582204ms from cluster start. Timeout at 2022-06-22 14:53:15.196080582 +0000 UTC m=+8.011220138, 106.029368ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.201Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:15.201Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.307Z [INFO] stable state for Follower reached at 2022-06-22 14:53:15.202004287 +0000 UTC m=+8.017143833 (2 nodes), 1.282µs from start of poll, 166.535267ms from cluster start. Timeout at 2022-06-22 14:53:15.307259653 +0000 UTC m=+8.122399219, 105.255386ms after stability testing.go:149: cluster: 2022-06-22T14:53:15.307Z [INFO] stable state for Follower reached at 2022-06-22 14:53:15.202004287 +0000 UTC m=+8.017143833 (2 nodes), 1.282µs from start of poll, 166.535267ms from cluster start. Timeout at 2022-06-22 14:53:15.307259653 +0000 UTC m=+8.122399219, 105.255386ms after stability testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: updating configuration: command=RemoveServer server-id=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca server-addr= servers="[{Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: updating configuration: command=RemoveServer server-id=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca server-addr= servers="[{Suffrage:Voter ID:server-c8419d02-14a7-4cdd-aefb-97f51a41a15e Address:c8419d02-14a7-4cdd-aefb-97f51a41a15e} {Suffrage:Voter ID:server-99e42361-b8e9-8075-0081-3d048dee2e98 Address:99e42361-b8e9-8075-0081-3d048dee2e98}]" testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: removed peer, stopping replication: peer=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca last-index=3 testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: removed peer, stopping replication: peer=server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca last-index=3 testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: aborting pipeline replication: peer="{Voter server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca fb5a2ee0-32f7-b335-f843-1ff32bbd8cca}" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.307Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: aborting pipeline replication: peer="{Voter server-fb5a2ee0-32f7-b335-f843-1ff32bbd8cca fb5a2ee0-32f7-b335-f843-1ff32bbd8cca}" testLoggerAdapter verbose: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.414Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: aborting pipeline replication: peer="{Voter server-c8419d02-14a7-4cdd-aefb-97f51a41a15e c8419d02-14a7-4cdd-aefb-97f51a41a15e}" testing.go:149: server-99e42361-b8e9-8075-0081-3d048dee2e98: 2022-06-22T14:53:15.414Z [INFO] server-99e42361-b8e9-8075-0081-3d048dee2e98: aborting pipeline replication: peer="{Voter server-c8419d02-14a7-4cdd-aefb-97f51a41a15e c8419d02-14a7-4cdd-aefb-97f51a41a15e}" --- PASS: TestRaft_RemoveFollower (0.38s) === RUN TestRaft_RemoveLeader testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.415Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.415Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.415Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.415Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.415Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering follower state: follower="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Follower]" leader= testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.415Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.415Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering follower state: follower="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Follower]" leader= testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.415Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering follower state: follower="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Follower]" leader= testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.415Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4} {Suffrage:Voter ID:server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a Address:5d41d8f0-7a1f-7562-9b58-dbd5477fc11a}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.416Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:15.416Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.415Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering follower state: follower="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Follower]" leader= testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.416Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering follower state: follower="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Follower]" leader= testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.416Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering follower state: follower="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Follower]" leader= testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.475Z [WARN] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.475Z [WARN] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering candidate state: node="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Candidate]" term=2 testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering candidate state: node="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Candidate]" term=2 testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: election won: tally=2 testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: election won: tally=2 testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering leader state: leader="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Leader]" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: entering leader state: leader="Node at 5d41d8f0-7a1f-7562-9b58-dbd5477fc11a [Leader]" testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: added peer, starting replication: peer=server-b6b529f8-cf95-cade-2da1-7f440814a676 testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: added peer, starting replication: peer=server-b6b529f8-cf95-cade-2da1-7f440814a676 testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: added peer, starting replication: peer=server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: added peer, starting replication: peer=server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: pipelining replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: pipelining replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: pipelining replication: peer="{Voter server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 280eef8f-dec7-7dfc-b34d-a6ea15c114d4}" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.476Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: pipelining replication: peer="{Voter server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 280eef8f-dec7-7dfc-b34d-a6ea15c114d4}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.582Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.476461114 +0000 UTC m=+8.291600661 (1 nodes), 60.332813ms from start of poll, 61.020785ms from cluster start. Timeout at 2022-06-22 14:53:15.582323095 +0000 UTC m=+8.397462681, 105.86202ms after stability testing.go:149: cluster: 2022-06-22T14:53:15.582Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.476461114 +0000 UTC m=+8.291600661 (1 nodes), 60.332813ms from start of poll, 61.020785ms from cluster start. Timeout at 2022-06-22 14:53:15.582323095 +0000 UTC m=+8.397462681, 105.86202ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.588Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:15.588Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.694Z [INFO] stable state for Follower reached at 2022-06-22 14:53:15.588381105 +0000 UTC m=+8.403520651 (2 nodes), 1.573µs from start of poll, 172.940775ms from cluster start. Timeout at 2022-06-22 14:53:15.694267472 +0000 UTC m=+8.509407239, 105.886588ms after stability testing.go:149: cluster: 2022-06-22T14:53:15.694Z [INFO] stable state for Follower reached at 2022-06-22 14:53:15.588381105 +0000 UTC m=+8.403520651 (2 nodes), 1.573µs from start of poll, 172.940775ms from cluster start. Timeout at 2022-06-22 14:53:15.694267472 +0000 UTC m=+8.509407239, 105.886588ms after stability testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.694Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: updating configuration: command=RemoveServer server-id=server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a server-addr= servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4}]" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.694Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: updating configuration: command=RemoveServer server-id=server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a server-addr= servers="[{Suffrage:Voter ID:server-b6b529f8-cf95-cade-2da1-7f440814a676 Address:b6b529f8-cf95-cade-2da1-7f440814a676} {Suffrage:Voter ID:server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 Address:280eef8f-dec7-7dfc-b34d-a6ea15c114d4}]" testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.694Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: removed ourself, shutting down testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.694Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: removed ourself, shutting down testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.695Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: aborting pipeline replication: peer="{Voter server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 280eef8f-dec7-7dfc-b34d-a6ea15c114d4}" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.695Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: aborting pipeline replication: peer="{Voter server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4 280eef8f-dec7-7dfc-b34d-a6ea15c114d4}" testLoggerAdapter verbose: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.695Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: aborting pipeline replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testing.go:149: server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: 2022-06-22T14:53:15.695Z [INFO] server-5d41d8f0-7a1f-7562-9b58-dbd5477fc11a: aborting pipeline replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.754Z [WARN] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: heartbeat timeout reached, starting election: last-leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.754Z [WARN] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: heartbeat timeout reached, starting election: last-leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.754Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering candidate state: node="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Candidate]" term=3 testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.754Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering candidate state: node="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Candidate]" term=3 testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.755Z [WARN] server-b6b529f8-cf95-cade-2da1-7f440814a676: rejecting vote request since we have a leader: from=280eef8f-dec7-7dfc-b34d-a6ea15c114d4 leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.755Z [WARN] server-b6b529f8-cf95-cade-2da1-7f440814a676: rejecting vote request since we have a leader: from=280eef8f-dec7-7dfc-b34d-a6ea15c114d4 leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.758Z [WARN] server-b6b529f8-cf95-cade-2da1-7f440814a676: heartbeat timeout reached, starting election: last-leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.758Z [WARN] server-b6b529f8-cf95-cade-2da1-7f440814a676: heartbeat timeout reached, starting election: last-leader=5d41d8f0-7a1f-7562-9b58-dbd5477fc11a testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.758Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering candidate state: node="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Candidate]" term=3 testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.758Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering candidate state: node="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Candidate]" term=3 testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.758Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: duplicate requestVote for same term: term=3 testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.758Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.850Z [WARN] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: Election timeout reached, restarting election testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.850Z [WARN] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: Election timeout reached, restarting election testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering candidate state: node="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Candidate]" term=4 testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering candidate state: node="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Candidate]" term=4 testLoggerAdapter verbose: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.851Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering follower state: follower="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Follower]" leader= testing.go:149: server-b6b529f8-cf95-cade-2da1-7f440814a676: 2022-06-22T14:53:15.851Z [INFO] server-b6b529f8-cf95-cade-2da1-7f440814a676: entering follower state: follower="Node at b6b529f8-cf95-cade-2da1-7f440814a676 [Follower]" leader= testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: election won: tally=2 testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: election won: tally=2 testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering leader state: leader="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Leader]" testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: entering leader state: leader="Node at 280eef8f-dec7-7dfc-b34d-a6ea15c114d4 [Leader]" testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: added peer, starting replication: peer=server-b6b529f8-cf95-cade-2da1-7f440814a676 testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: added peer, starting replication: peer=server-b6b529f8-cf95-cade-2da1-7f440814a676 testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: pipelining replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:15.851Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: pipelining replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:15.905Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:15.905Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.012Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.906059574 +0000 UTC m=+8.721199120 (1 nodes), 35.095µs from start of poll, 490.619244ms from cluster start. Timeout at 2022-06-22 14:53:16.012051781 +0000 UTC m=+8.827191337, 105.992217ms after stability testing.go:149: cluster: 2022-06-22T14:53:16.012Z [INFO] stable state for Leader reached at 2022-06-22 14:53:15.906059574 +0000 UTC m=+8.721199120 (1 nodes), 35.095µs from start of poll, 490.619244ms from cluster start. Timeout at 2022-06-22 14:53:16.012051781 +0000 UTC m=+8.827191337, 105.992217ms after stability testLoggerAdapter verbose: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:16.012Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: aborting pipeline replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" testing.go:149: server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: 2022-06-22T14:53:16.012Z [INFO] server-280eef8f-dec7-7dfc-b34d-a6ea15c114d4: aborting pipeline replication: peer="{Voter server-b6b529f8-cf95-cade-2da1-7f440814a676 b6b529f8-cf95-cade-2da1-7f440814a676}" --- PASS: TestRaft_RemoveLeader (0.60s) === RUN TestRaft_RemoveLeader_NoShutdown testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.013Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.013Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.013Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.013Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.013Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering follower state: follower="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Follower]" leader= testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.013Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering follower state: follower="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Follower]" leader= testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.013Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.013Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8} {Suffrage:Voter ID:server-5e9175db-8394-7bf7-1344-2f54ec84d712 Address:5e9175db-8394-7bf7-1344-2f54ec84d712}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.013Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:16.013Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.013Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering follower state: follower="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Follower]" leader= testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.013Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering follower state: follower="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Follower]" leader= testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.013Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering follower state: follower="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Follower]" leader= testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.013Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering follower state: follower="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Follower]" leader= testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [WARN] server-5e9175db-8394-7bf7-1344-2f54ec84d712: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [WARN] server-5e9175db-8394-7bf7-1344-2f54ec84d712: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering candidate state: node="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Candidate]" term=2 testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering candidate state: node="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Candidate]" term=2 testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: election won: tally=2 testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: election won: tally=2 testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering leader state: leader="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Leader]" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering leader state: leader="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Leader]" testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: added peer, starting replication: peer=server-091beef7-feec-c9d7-2e88-962a3ffa22ba testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: added peer, starting replication: peer=server-091beef7-feec-c9d7-2e88-962a3ffa22ba testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: added peer, starting replication: peer=server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: added peer, starting replication: peer=server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: pipelining replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.074Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: pipelining replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.075Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: pipelining replication: peer="{Voter server-091beef7-feec-c9d7-2e88-962a3ffa22ba 091beef7-feec-c9d7-2e88-962a3ffa22ba}" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.075Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: pipelining replication: peer="{Voter server-091beef7-feec-c9d7-2e88-962a3ffa22ba 091beef7-feec-c9d7-2e88-962a3ffa22ba}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.181Z [INFO] stable state for Follower reached at 2022-06-22 14:53:16.075078356 +0000 UTC m=+8.890217902 (2 nodes), 61.150506ms from start of poll, 61.702711ms from cluster start. Timeout at 2022-06-22 14:53:16.181101267 +0000 UTC m=+8.996240833, 106.022931ms after stability testing.go:149: cluster: 2022-06-22T14:53:16.181Z [INFO] stable state for Follower reached at 2022-06-22 14:53:16.075078356 +0000 UTC m=+8.890217902 (2 nodes), 61.150506ms from start of poll, 61.702711ms from cluster start. Timeout at 2022-06-22 14:53:16.181101267 +0000 UTC m=+8.996240833, 106.022931ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.181Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:16.181Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.286Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.181497036 +0000 UTC m=+8.996636582 (1 nodes), 3.397µs from start of poll, 168.121391ms from cluster start. Timeout at 2022-06-22 14:53:16.286726145 +0000 UTC m=+9.101865691, 105.229109ms after stability testing.go:149: cluster: 2022-06-22T14:53:16.286Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.181497036 +0000 UTC m=+8.996636582 (1 nodes), 3.397µs from start of poll, 168.121391ms from cluster start. Timeout at 2022-06-22 14:53:16.286726145 +0000 UTC m=+9.101865691, 105.229109ms after stability testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.289Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: updating configuration: command=RemoveServer server-id=server-5e9175db-8394-7bf7-1344-2f54ec84d712 server-addr= servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8}]" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.289Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: updating configuration: command=RemoveServer server-id=server-5e9175db-8394-7bf7-1344-2f54ec84d712 server-addr= servers="[{Suffrage:Voter ID:server-091beef7-feec-c9d7-2e88-962a3ffa22ba Address:091beef7-feec-c9d7-2e88-962a3ffa22ba} {Suffrage:Voter ID:server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 Address:bfbf37cf-6aa8-9557-07a4-3801250ccea8}]" testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.289Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: removed ourself, transitioning to follower testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.289Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: removed ourself, transitioning to follower testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering follower state: follower="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Follower]" leader= testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: entering follower state: follower="Node at 5e9175db-8394-7bf7-1344-2f54ec84d712 [Follower]" leader= testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: aborting pipeline replication: peer="{Voter server-091beef7-feec-c9d7-2e88-962a3ffa22ba 091beef7-feec-c9d7-2e88-962a3ffa22ba}" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: aborting pipeline replication: peer="{Voter server-091beef7-feec-c9d7-2e88-962a3ffa22ba 091beef7-feec-c9d7-2e88-962a3ffa22ba}" testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: aborting pipeline replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.290Z [INFO] server-5e9175db-8394-7bf7-1344-2f54ec84d712: aborting pipeline replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testLoggerAdapter verbose: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.367Z [WARN] server-5e9175db-8394-7bf7-1344-2f54ec84d712: not part of stable configuration, aborting election testing.go:149: server-5e9175db-8394-7bf7-1344-2f54ec84d712: 2022-06-22T14:53:16.367Z [WARN] server-5e9175db-8394-7bf7-1344-2f54ec84d712: not part of stable configuration, aborting election testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.367Z [WARN] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: heartbeat timeout reached, starting election: last-leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.367Z [WARN] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: heartbeat timeout reached, starting election: last-leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.367Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering candidate state: node="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Candidate]" term=3 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.367Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering candidate state: node="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Candidate]" term=3 testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.367Z [WARN] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: rejecting vote request since we have a leader: from=091beef7-feec-c9d7-2e88-962a3ffa22ba leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.367Z [WARN] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: rejecting vote request since we have a leader: from=091beef7-feec-c9d7-2e88-962a3ffa22ba leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.395Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:16.395Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.395Z [WARN] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: heartbeat timeout reached, starting election: last-leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.395Z [WARN] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: heartbeat timeout reached, starting election: last-leader=5e9175db-8394-7bf7-1344-2f54ec84d712 testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.395Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering candidate state: node="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Candidate]" term=3 testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.395Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering candidate state: node="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Candidate]" term=3 testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.395Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: duplicate requestVote for same term: term=3 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.395Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: duplicate requestVote for same term: term=3 testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [WARN] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: Election timeout reached, restarting election testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [WARN] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: Election timeout reached, restarting election testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering candidate state: node="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Candidate]" term=4 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering candidate state: node="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Candidate]" term=4 testLoggerAdapter verbose: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.448Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering follower state: follower="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Follower]" leader= testing.go:149: server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: 2022-06-22T14:53:16.448Z [INFO] server-bfbf37cf-6aa8-9557-07a4-3801250ccea8: entering follower state: follower="Node at bfbf37cf-6aa8-9557-07a4-3801250ccea8 [Follower]" leader= testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: election won: tally=2 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: election won: tally=2 testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering leader state: leader="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Leader]" testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: entering leader state: leader="Node at 091beef7-feec-c9d7-2e88-962a3ffa22ba [Leader]" testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: added peer, starting replication: peer=server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.448Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: added peer, starting replication: peer=server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.449Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: pipelining replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.449Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: pipelining replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.554Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.448937585 +0000 UTC m=+9.264077131 (1 nodes), 53.242668ms from start of poll, 435.56194ms from cluster start. Timeout at 2022-06-22 14:53:16.554216067 +0000 UTC m=+9.369355653, 105.278522ms after stability testing.go:149: cluster: 2022-06-22T14:53:16.554Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.448937585 +0000 UTC m=+9.264077131 (1 nodes), 53.242668ms from start of poll, 435.56194ms from cluster start. Timeout at 2022-06-22 14:53:16.554216067 +0000 UTC m=+9.369355653, 105.278522ms after stability testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.659Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: aborting pipeline replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.659Z [INFO] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: aborting pipeline replication: peer="{Voter server-bfbf37cf-6aa8-9557-07a4-3801250ccea8 bfbf37cf-6aa8-9557-07a4-3801250ccea8}" testLoggerAdapter verbose: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.710Z [ERROR] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: failed to heartbeat to: peer=bfbf37cf-6aa8-9557-07a4-3801250ccea8 error="command timed out" testing.go:149: server-091beef7-feec-c9d7-2e88-962a3ffa22ba: 2022-06-22T14:53:16.710Z [ERROR] server-091beef7-feec-c9d7-2e88-962a3ffa22ba: failed to heartbeat to: peer=bfbf37cf-6aa8-9557-07a4-3801250ccea8 error="command timed out" --- PASS: TestRaft_RemoveLeader_NoShutdown (0.70s) === RUN TestRaft_RemoveFollower_SplitCluster testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.711Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.711Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testLoggerAdapter verbose: server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: 2022-06-22T14:53:16.711Z [INFO] server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testing.go:149: server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: 2022-06-22T14:53:16.711Z [INFO] server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testLoggerAdapter verbose: server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: 2022-06-22T14:53:16.711Z [INFO] server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testLoggerAdapter verbose: server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: 2022-06-22T14:53:16.711Z [INFO] server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: entering follower state: follower="Node at b3b42bb8-dfbd-86a2-b2f1-2713932639a3 [Follower]" leader= testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.711Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering follower state: follower="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Follower]" leader= testing.go:149: server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: 2022-06-22T14:53:16.711Z [INFO] server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3: entering follower state: follower="Node at b3b42bb8-dfbd-86a2-b2f1-2713932639a3 [Follower]" leader= testing.go:149: server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: 2022-06-22T14:53:16.711Z [INFO] server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.711Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering follower state: follower="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Follower]" leader= testLoggerAdapter verbose: server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: 2022-06-22T14:53:16.712Z [INFO] server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testing.go:149: server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: 2022-06-22T14:53:16.712Z [INFO] server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018} {Suffrage:Voter ID:server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 Address:37bd4ed9-4c7c-97ce-4952-cb49f4299869}]" testLoggerAdapter verbose: server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: 2022-06-22T14:53:16.712Z [INFO] server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: entering follower state: follower="Node at 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 [Follower]" leader= testLoggerAdapter verbose: server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: 2022-06-22T14:53:16.712Z [INFO] server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: entering follower state: follower="Node at 37bd4ed9-4c7c-97ce-4952-cb49f4299869 [Follower]" leader= testing.go:149: server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: 2022-06-22T14:53:16.712Z [INFO] server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018: entering follower state: follower="Node at 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 [Follower]" leader= testing.go:149: server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: 2022-06-22T14:53:16.712Z [INFO] server-37bd4ed9-4c7c-97ce-4952-cb49f4299869: entering follower state: follower="Node at 37bd4ed9-4c7c-97ce-4952-cb49f4299869 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.712Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:16.712Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering candidate state: node="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Candidate]" term=2 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering candidate state: node="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Candidate]" term=2 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: election won: tally=3 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.770Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: election won: tally=3 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering leader state: leader="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Leader]" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering leader state: leader="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Leader]" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: added peer, starting replication: peer=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 37bd4ed9-4c7c-97ce-4952-cb49f4299869}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:16.771Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: pipelining replication: peer="{Voter server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 37bd4ed9-4c7c-97ce-4952-cb49f4299869}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.877Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.771234085 +0000 UTC m=+9.586373631 (1 nodes), 58.794561ms from start of poll, 59.755669ms from cluster start. Timeout at 2022-06-22 14:53:16.877073869 +0000 UTC m=+9.692213415, 105.839784ms after stability testing.go:149: cluster: 2022-06-22T14:53:16.877Z [INFO] stable state for Leader reached at 2022-06-22 14:53:16.771234085 +0000 UTC m=+9.586373631 (1 nodes), 58.794561ms from start of poll, 59.755669ms from cluster start. Timeout at 2022-06-22 14:53:16.877073869 +0000 UTC m=+9.692213415, 105.839784ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:16.982Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:16.982Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.087Z [INFO] stable state for Follower reached at 2022-06-22 14:53:16.982708435 +0000 UTC m=+9.797847981 (3 nodes), 1.423µs from start of poll, 271.230019ms from cluster start. Timeout at 2022-06-22 14:53:17.087947328 +0000 UTC m=+9.903086874, 105.238893ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.087Z [INFO] stable state for Follower reached at 2022-06-22 14:53:16.982708435 +0000 UTC m=+9.797847981 (3 nodes), 1.423µs from start of poll, 271.230019ms from cluster start. Timeout at 2022-06-22 14:53:17.087947328 +0000 UTC m=+9.903086874, 105.238893ms after stability testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: updating configuration: command=RemoveServer server-id=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 server-addr= servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}]" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: updating configuration: command=RemoveServer server-id=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 server-addr= servers="[{Suffrage:Voter ID:server-742825aa-4093-7501-2cb7-515c79742e38 Address:742825aa-4093-7501-2cb7-515c79742e38} {Suffrage:Voter ID:server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 Address:b3b42bb8-dfbd-86a2-b2f1-2713932639a3} {Suffrage:Voter ID:server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 Address:2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}]" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: removed peer, stopping replication: peer=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 last-index=3 testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: removed peer, stopping replication: peer=server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 last-index=3 testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to pipeline appendEntries: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="append pipeline closed" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to pipeline appendEntries: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="append pipeline closed" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to pipeline appendEntries: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="append pipeline closed" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to pipeline appendEntries: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="append pipeline closed" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 37bd4ed9-4c7c-97ce-4952-cb49f4299869}" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.088Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: aborting pipeline replication: peer="{Voter server-37bd4ed9-4c7c-97ce-4952-cb49f4299869 37bd4ed9-4c7c-97ce-4952-cb49f4299869}" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.091Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.091Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.091Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.091Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.097Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.097Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.097Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.097Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.108Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.108Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.110Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.110Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.115Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.115Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.117Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.117Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.125Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.125Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.131Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.131Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to heartbeat to: peer=b3b42bb8-dfbd-86a2-b2f1-2713932639a3 error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.135Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.135Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 b3b42bb8-dfbd-86a2-b2f1-2713932639a3}" error="failed to connect to peer: b3b42bb8-dfbd-86a2-b2f1-2713932639a3" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [ERROR] server-742825aa-4093-7501-2cb7-515c79742e38: failed to appendEntries to: peer="{Voter server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018}" error="failed to connect to peer: 2cba07f3-0d31-1edb-52d5-1fc8d0dc1018" testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact: server-id=server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 time=51.104872ms testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact: server-id=server-b3b42bb8-dfbd-86a2-b2f1-2713932639a3 time=51.104872ms testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact: server-id=server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 time=51.104341ms testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact: server-id=server-2cba07f3-0d31-1edb-52d5-1fc8d0dc1018 time=51.104341ms testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact quorum of nodes, stepping down testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [WARN] server-742825aa-4093-7501-2cb7-515c79742e38: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering follower state: follower="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Follower]" leader= testing.go:149: server-742825aa-4093-7501-2cb7-515c79742e38: 2022-06-22T14:53:17.136Z [INFO] server-742825aa-4093-7501-2cb7-515c79742e38: entering follower state: follower="Node at 742825aa-4093-7501-2cb7-515c79742e38 [Follower]" leader= --- PASS: TestRaft_RemoveFollower_SplitCluster (0.43s) === RUN TestRaft_AddKnownPeer testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.138Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.138Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.138Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering follower state: follower="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Follower]" leader= testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.138Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering follower state: follower="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Follower]" leader= testLoggerAdapter verbose: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.139Z [INFO] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testing.go:149: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.139Z [INFO] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testLoggerAdapter verbose: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.139Z [INFO] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: entering follower state: follower="Node at e6e969f5-43ce-81c1-0a6a-e5b819d4d265 [Follower]" leader= testing.go:149: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.139Z [INFO] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: entering follower state: follower="Node at e6e969f5-43ce-81c1-0a6a-e5b819d4d265 [Follower]" leader= testLoggerAdapter verbose: server-039909ce-ca5b-158d-7552-019567ef279b: 2022-06-22T14:53:17.139Z [INFO] server-039909ce-ca5b-158d-7552-019567ef279b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testing.go:149: server-039909ce-ca5b-158d-7552-019567ef279b: 2022-06-22T14:53:17.139Z [INFO] server-039909ce-ca5b-158d-7552-019567ef279b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.139Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:17.139Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-039909ce-ca5b-158d-7552-019567ef279b: 2022-06-22T14:53:17.139Z [INFO] server-039909ce-ca5b-158d-7552-019567ef279b: entering follower state: follower="Node at 039909ce-ca5b-158d-7552-019567ef279b [Follower]" leader= testing.go:149: server-039909ce-ca5b-158d-7552-019567ef279b: 2022-06-22T14:53:17.139Z [INFO] server-039909ce-ca5b-158d-7552-019567ef279b: entering follower state: follower="Node at 039909ce-ca5b-158d-7552-019567ef279b [Follower]" leader= testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [WARN] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [WARN] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering candidate state: node="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Candidate]" term=2 testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering candidate state: node="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Candidate]" term=2 testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: election won: tally=2 testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: election won: tally=2 testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering leader state: leader="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Leader]" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: entering leader state: leader="Node at f7e0c904-50c1-c3f2-3799-ce949a026f9f [Leader]" testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: added peer, starting replication: peer=server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: added peer, starting replication: peer=server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: added peer, starting replication: peer=server-039909ce-ca5b-158d-7552-019567ef279b testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: added peer, starting replication: peer=server-039909ce-ca5b-158d-7552-019567ef279b testLoggerAdapter verbose: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.195Z [WARN] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: rejecting vote request since our last term is greater: candidate=f7e0c904-50c1-c3f2-3799-ce949a026f9f last-term=2 last-candidate-term=1 testing.go:149: server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: 2022-06-22T14:53:17.195Z [WARN] server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265: rejecting vote request since our last term is greater: candidate=f7e0c904-50c1-c3f2-3799-ce949a026f9f last-term=2 last-candidate-term=1 testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: pipelining replication: peer="{Voter server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 e6e969f5-43ce-81c1-0a6a-e5b819d4d265}" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: pipelining replication: peer="{Voter server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 e6e969f5-43ce-81c1-0a6a-e5b819d4d265}" testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: pipelining replication: peer="{Voter server-039909ce-ca5b-158d-7552-019567ef279b 039909ce-ca5b-158d-7552-019567ef279b}" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.195Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: pipelining replication: peer="{Voter server-039909ce-ca5b-158d-7552-019567ef279b 039909ce-ca5b-158d-7552-019567ef279b}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.301Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.195785999 +0000 UTC m=+10.010925545 (1 nodes), 55.892458ms from start of poll, 57.187788ms from cluster start. Timeout at 2022-06-22 14:53:17.301311233 +0000 UTC m=+10.116451009, 105.525464ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.301Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.195785999 +0000 UTC m=+10.010925545 (1 nodes), 55.892458ms from start of poll, 57.187788ms from cluster start. Timeout at 2022-06-22 14:53:17.301311233 +0000 UTC m=+10.116451009, 105.525464ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.301Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:17.301Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.407Z [INFO] stable state for Follower reached at 2022-06-22 14:53:17.301561856 +0000 UTC m=+10.116701402 (2 nodes), 2.024µs from start of poll, 162.963645ms from cluster start. Timeout at 2022-06-22 14:53:17.407322927 +0000 UTC m=+10.222462474, 105.761072ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.407Z [INFO] stable state for Follower reached at 2022-06-22 14:53:17.301561856 +0000 UTC m=+10.116701402 (2 nodes), 2.024µs from start of poll, 162.963645ms from cluster start. Timeout at 2022-06-22 14:53:17.407322927 +0000 UTC m=+10.222462474, 105.761072ms after stability testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: updating configuration: command=AddStaging server-id=server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 server-addr=e6e969f5-43ce-81c1-0a6a-e5b819d4d265 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: updating configuration: command=AddStaging server-id=server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 server-addr=e6e969f5-43ce-81c1-0a6a-e5b819d4d265 servers="[{Suffrage:Voter ID:server-f7e0c904-50c1-c3f2-3799-ce949a026f9f Address:f7e0c904-50c1-c3f2-3799-ce949a026f9f} {Suffrage:Voter ID:server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 Address:e6e969f5-43ce-81c1-0a6a-e5b819d4d265} {Suffrage:Voter ID:server-039909ce-ca5b-158d-7552-019567ef279b Address:039909ce-ca5b-158d-7552-019567ef279b}]" testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: aborting pipeline replication: peer="{Voter server-039909ce-ca5b-158d-7552-019567ef279b 039909ce-ca5b-158d-7552-019567ef279b}" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: aborting pipeline replication: peer="{Voter server-039909ce-ca5b-158d-7552-019567ef279b 039909ce-ca5b-158d-7552-019567ef279b}" testLoggerAdapter verbose: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: aborting pipeline replication: peer="{Voter server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 e6e969f5-43ce-81c1-0a6a-e5b819d4d265}" testing.go:149: server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: 2022-06-22T14:53:17.407Z [INFO] server-f7e0c904-50c1-c3f2-3799-ce949a026f9f: aborting pipeline replication: peer="{Voter server-e6e969f5-43ce-81c1-0a6a-e5b819d4d265 e6e969f5-43ce-81c1-0a6a-e5b819d4d265}" --- PASS: TestRaft_AddKnownPeer (0.27s) === RUN TestRaft_RemoveUnknownPeer testLoggerAdapter verbose: server-fb30b376-83ba-6ead-1730-a9072861e074: 2022-06-22T14:53:17.410Z [INFO] server-fb30b376-83ba-6ead-1730-a9072861e074: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testing.go:149: server-fb30b376-83ba-6ead-1730-a9072861e074: 2022-06-22T14:53:17.410Z [INFO] server-fb30b376-83ba-6ead-1730-a9072861e074: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testLoggerAdapter verbose: server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: 2022-06-22T14:53:17.410Z [INFO] server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testing.go:149: server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: 2022-06-22T14:53:17.410Z [INFO] server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testLoggerAdapter verbose: server-fb30b376-83ba-6ead-1730-a9072861e074: 2022-06-22T14:53:17.410Z [INFO] server-fb30b376-83ba-6ead-1730-a9072861e074: entering follower state: follower="Node at fb30b376-83ba-6ead-1730-a9072861e074 [Follower]" leader= testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.410Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testing.go:149: server-fb30b376-83ba-6ead-1730-a9072861e074: 2022-06-22T14:53:17.410Z [INFO] server-fb30b376-83ba-6ead-1730-a9072861e074: entering follower state: follower="Node at fb30b376-83ba-6ead-1730-a9072861e074 [Follower]" leader= testLoggerAdapter verbose: server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: 2022-06-22T14:53:17.410Z [INFO] server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: entering follower state: follower="Node at 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 [Follower]" leader= testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.410Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.411Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering follower state: follower="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Follower]" leader= testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.411Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering follower state: follower="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Follower]" leader= testing.go:149: server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: 2022-06-22T14:53:17.410Z [INFO] server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8: entering follower state: follower="Node at 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.411Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:17.411Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.473Z [WARN] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.473Z [WARN] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.473Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering candidate state: node="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Candidate]" term=2 testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.473Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering candidate state: node="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Candidate]" term=2 testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: election won: tally=2 testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: election won: tally=2 testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering leader state: leader="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Leader]" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: entering leader state: leader="Node at 36d00258-c2cc-e18a-b5c1-9364c19b77e3 [Leader]" testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: added peer, starting replication: peer=server-fb30b376-83ba-6ead-1730-a9072861e074 testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: added peer, starting replication: peer=server-fb30b376-83ba-6ead-1730-a9072861e074 testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: added peer, starting replication: peer=server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: added peer, starting replication: peer=server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: pipelining replication: peer="{Voter server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8}" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: pipelining replication: peer="{Voter server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8}" testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: pipelining replication: peer="{Voter server-fb30b376-83ba-6ead-1730-a9072861e074 fb30b376-83ba-6ead-1730-a9072861e074}" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.474Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: pipelining replication: peer="{Voter server-fb30b376-83ba-6ead-1730-a9072861e074 fb30b376-83ba-6ead-1730-a9072861e074}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.580Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.474230742 +0000 UTC m=+10.289370288 (1 nodes), 62.684436ms from start of poll, 64.236501ms from cluster start. Timeout at 2022-06-22 14:53:17.580230795 +0000 UTC m=+10.395370341, 106.000053ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.580Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.474230742 +0000 UTC m=+10.289370288 (1 nodes), 62.684436ms from start of poll, 64.236501ms from cluster start. Timeout at 2022-06-22 14:53:17.580230795 +0000 UTC m=+10.395370341, 106.000053ms after stability testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: updating configuration: command=RemoveServer server-id=0b5e3c8c-6fc3-f6b9-7205-b88dee2df1b1 server-addr= servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: updating configuration: command=RemoveServer server-id=0b5e3c8c-6fc3-f6b9-7205-b88dee2df1b1 server-addr= servers="[{Suffrage:Voter ID:server-fb30b376-83ba-6ead-1730-a9072861e074 Address:fb30b376-83ba-6ead-1730-a9072861e074} {Suffrage:Voter ID:server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 Address:8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8} {Suffrage:Voter ID:server-36d00258-c2cc-e18a-b5c1-9364c19b77e3 Address:36d00258-c2cc-e18a-b5c1-9364c19b77e3}]" testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: aborting pipeline replication: peer="{Voter server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8}" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: aborting pipeline replication: peer="{Voter server-8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8 8e1d9fa5-9cfc-104a-4d1f-35fc9c6b89f8}" testLoggerAdapter verbose: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: aborting pipeline replication: peer="{Voter server-fb30b376-83ba-6ead-1730-a9072861e074 fb30b376-83ba-6ead-1730-a9072861e074}" testing.go:149: server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: 2022-06-22T14:53:17.580Z [INFO] server-36d00258-c2cc-e18a-b5c1-9364c19b77e3: aborting pipeline replication: peer="{Voter server-fb30b376-83ba-6ead-1730-a9072861e074 fb30b376-83ba-6ead-1730-a9072861e074}" --- PASS: TestRaft_RemoveUnknownPeer (0.17s) === RUN TestRaft_SnapshotRestore testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.581Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6e8f4c77-bac3-e13d-459c-251a1df71c69 Address:6e8f4c77-bac3-e13d-459c-251a1df71c69}]" testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.581Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6e8f4c77-bac3-e13d-459c-251a1df71c69 Address:6e8f4c77-bac3-e13d-459c-251a1df71c69}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.581Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:17.581Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.581Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering follower state: follower="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Follower]" leader= testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.581Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering follower state: follower="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Follower]" leader= testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [WARN] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [WARN] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering candidate state: node="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Candidate]" term=2 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering candidate state: node="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Candidate]" term=2 testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: election won: tally=1 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: election won: tally=1 testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering leader state: leader="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Leader]" testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.649Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: entering leader state: leader="Node at 6e8f4c77-bac3-e13d-459c-251a1df71c69 [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.755Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.650060268 +0000 UTC m=+10.465199824 (1 nodes), 68.460733ms from start of poll, 68.632348ms from cluster start. Timeout at 2022-06-22 14:53:17.755735433 +0000 UTC m=+10.570874999, 105.675175ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.755Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.650060268 +0000 UTC m=+10.465199824 (1 nodes), 68.460733ms from start of poll, 68.632348ms from cluster start. Timeout at 2022-06-22 14:53:17.755735433 +0000 UTC m=+10.570874999, 105.675175ms after stability testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.756Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: starting snapshot up to: index=102 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.756Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: starting snapshot up to: index=102 testing.go:153: 2022-06-22T14:53:17.756Z [INFO] creating new snapshot: path=/tmp/raft1342571864/snapshots/2-102-1655909597756.tmp testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.760Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: compacting logs: from=1 to=92 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.760Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.760Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: snapshot complete up to: index=102 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.760Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: snapshot complete up to: index=102 testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.761Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: restored from snapshot: id=2-102-1655909597756 testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.761Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: restored from snapshot: id=2-102-1655909597756 testLoggerAdapter verbose: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.761Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6e8f4c77-bac3-e13d-459c-251a1df71c69 Address:6e8f4c77-bac3-e13d-459c-251a1df71c69}]" testing.go:149: server-6e8f4c77-bac3-e13d-459c-251a1df71c69: 2022-06-22T14:53:17.761Z [INFO] server-6e8f4c77-bac3-e13d-459c-251a1df71c69: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6e8f4c77-bac3-e13d-459c-251a1df71c69 Address:6e8f4c77-bac3-e13d-459c-251a1df71c69}]" --- PASS: TestRaft_SnapshotRestore (0.18s) === RUN TestRaft_NoRestoreOnStart testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.762Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-21d57a48-b80b-63df-15b4-44a5ca0dfb58 Address:21d57a48-b80b-63df-15b4-44a5ca0dfb58}]" testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.762Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-21d57a48-b80b-63df-15b4-44a5ca0dfb58 Address:21d57a48-b80b-63df-15b4-44a5ca0dfb58}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.762Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:17.762Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.762Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering follower state: follower="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Follower]" leader= testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.762Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering follower state: follower="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Follower]" leader= testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [WARN] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [WARN] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering candidate state: node="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Candidate]" term=2 testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering candidate state: node="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Candidate]" term=2 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: election won: tally=1 testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: election won: tally=1 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering leader state: leader="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Leader]" testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.855Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering leader state: leader="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.961Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.855725129 +0000 UTC m=+10.670864685 (1 nodes), 93.393879ms from start of poll, 93.635526ms from cluster start. Timeout at 2022-06-22 14:53:17.961088817 +0000 UTC m=+10.776228373, 105.363688ms after stability testing.go:149: cluster: 2022-06-22T14:53:17.961Z [INFO] stable state for Leader reached at 2022-06-22 14:53:17.855725129 +0000 UTC m=+10.670864685 (1 nodes), 93.393879ms from start of poll, 93.635526ms from cluster start. Timeout at 2022-06-22 14:53:17.961088817 +0000 UTC m=+10.776228373, 105.363688ms after stability testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.962Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: starting snapshot up to: index=102 testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.962Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: starting snapshot up to: index=102 testing.go:153: 2022-06-22T14:53:17.962Z [INFO] creating new snapshot: path=/tmp/raft3519026089/snapshots/2-102-1655909597962.tmp testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: compacting logs: from=1 to=92 testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: snapshot complete up to: index=102 testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: snapshot complete up to: index=102 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-21d57a48-b80b-63df-15b4-44a5ca0dfb58 Address:21d57a48-b80b-63df-15b4-44a5ca0dfb58}]" testing.go:149: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.965Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-21d57a48-b80b-63df-15b4-44a5ca0dfb58 Address:21d57a48-b80b-63df-15b4-44a5ca0dfb58}]" --- PASS: TestRaft_NoRestoreOnStart (0.20s) === RUN TestRaft_SnapshotRestore_PeerChange raft_test.go:1026: Disabled unreliable test testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:17.966Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering follower state: follower="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Follower]" leader= --- SKIP: TestRaft_SnapshotRestore_PeerChange (0.00s) === RUN TestRaft_AutoSnapshot raft_test.go:1132: Disabled unreliable test --- SKIP: TestRaft_AutoSnapshot (0.00s) === RUN TestRaft_UserSnapshot testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:17.966Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75c9e145-8fe2-5141-b3f8-bb122430c2f6 Address:75c9e145-8fe2-5141-b3f8-bb122430c2f6}]" testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:17.966Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75c9e145-8fe2-5141-b3f8-bb122430c2f6 Address:75c9e145-8fe2-5141-b3f8-bb122430c2f6}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:17.966Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:17.966Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:17.967Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering follower state: follower="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Follower]" leader= testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:17.967Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering follower state: follower="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Follower]" leader= testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [WARN] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [WARN] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering candidate state: node="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Candidate]" term=2 testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering candidate state: node="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Candidate]" term=2 testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: election won: tally=1 testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: election won: tally=1 testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering leader state: leader="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Leader]" testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.023Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: entering leader state: leader="Node at 75c9e145-8fe2-5141-b3f8-bb122430c2f6 [Leader]" testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:18.044Z [WARN] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:18.044Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering candidate state: node="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Candidate]" term=3 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:18.044Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: election won: tally=1 testLoggerAdapter verbose: server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: 2022-06-22T14:53:18.044Z [INFO] server-21d57a48-b80b-63df-15b4-44a5ca0dfb58: entering leader state: leader="Node at 21d57a48-b80b-63df-15b4-44a5ca0dfb58 [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.129Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.024046429 +0000 UTC m=+10.839185975 (1 nodes), 57.038495ms from start of poll, 57.317954ms from cluster start. Timeout at 2022-06-22 14:53:18.129926039 +0000 UTC m=+10.945065585, 105.87961ms after stability testing.go:149: cluster: 2022-06-22T14:53:18.129Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.024046429 +0000 UTC m=+10.839185975 (1 nodes), 57.038495ms from start of poll, 57.317954ms from cluster start. Timeout at 2022-06-22 14:53:18.129926039 +0000 UTC m=+10.945065585, 105.87961ms after stability testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.130Z [ERROR] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.130Z [ERROR] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.130Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: starting snapshot up to: index=12 testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.130Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:18.130Z [INFO] creating new snapshot: path=/tmp/raft1225976558/snapshots/2-12-1655909598130.tmp testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.134Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: compacting logs: from=1 to=2 testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.134Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: compacting logs: from=1 to=2 testLoggerAdapter verbose: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.134Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: snapshot complete up to: index=12 testing.go:149: server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: 2022-06-22T14:53:18.134Z [INFO] server-75c9e145-8fe2-5141-b3f8-bb122430c2f6: snapshot complete up to: index=12 --- PASS: TestRaft_UserSnapshot (0.17s) === RUN TestRaft_UserRestore testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.136Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.136Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testLoggerAdapter verbose: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.136Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testing.go:149: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.136Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.136Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering follower state: follower="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Follower]" leader= testLoggerAdapter verbose: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.136Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: entering follower state: follower="Node at 915826de-3e5d-5453-848d-066187ec3f62 [Follower]" leader= testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.136Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering follower state: follower="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Follower]" leader= testing.go:149: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.136Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: entering follower state: follower="Node at 915826de-3e5d-5453-848d-066187ec3f62 [Follower]" leader= testLoggerAdapter verbose: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.137Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testing.go:149: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.137Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-4c82bb8d-6404-e596-964a-63bf893e6f41 Address:4c82bb8d-6404-e596-964a-63bf893e6f41} {Suffrage:Voter ID:server-915826de-3e5d-5453-848d-066187ec3f62 Address:915826de-3e5d-5453-848d-066187ec3f62} {Suffrage:Voter ID:server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 Address:af84098b-5c6f-e357-f46f-6b3c1c34efc1}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.137Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:18.137Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.137Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: entering follower state: follower="Node at af84098b-5c6f-e357-f46f-6b3c1c34efc1 [Follower]" leader= testing.go:149: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.137Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: entering follower state: follower="Node at af84098b-5c6f-e357-f46f-6b3c1c34efc1 [Follower]" leader= testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [WARN] server-4c82bb8d-6404-e596-964a-63bf893e6f41: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [WARN] server-4c82bb8d-6404-e596-964a-63bf893e6f41: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering candidate state: node="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Candidate]" term=2 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering candidate state: node="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Candidate]" term=2 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: election won: tally=2 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: election won: tally=2 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering leader state: leader="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Leader]" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: entering leader state: leader="Node at 4c82bb8d-6404-e596-964a-63bf893e6f41 [Leader]" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: added peer, starting replication: peer=server-915826de-3e5d-5453-848d-066187ec3f62 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: added peer, starting replication: peer=server-915826de-3e5d-5453-848d-066187ec3f62 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: added peer, starting replication: peer=server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: added peer, starting replication: peer=server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.191Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.297Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.191325923 +0000 UTC m=+11.006465469 (1 nodes), 53.979436ms from start of poll, 54.930857ms from cluster start. Timeout at 2022-06-22 14:53:18.297218698 +0000 UTC m=+11.112358244, 105.892775ms after stability testing.go:149: cluster: 2022-06-22T14:53:18.297Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.191325923 +0000 UTC m=+11.006465469 (1 nodes), 53.979436ms from start of poll, 54.930857ms from cluster start. Timeout at 2022-06-22 14:53:18.297218698 +0000 UTC m=+11.112358244, 105.892775ms after stability testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.297Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: starting snapshot up to: index=12 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.297Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:18.297Z [INFO] creating new snapshot: path=/tmp/raft1449577982/snapshots/2-12-1655909598297.tmp testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.301Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: snapshot complete up to: index=12 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.301Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:18.302Z [INFO] creating new snapshot: path=/tmp/raft1449577982/snapshots/2-23-1655909598302.tmp testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.305Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: copied to local snapshot: bytes=71 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.305Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.305Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: restored user snapshot: index=1 testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.305Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: restored user snapshot: index=1 testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.306Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.312Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.312Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.313Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.313Z [ERROR] server-4c82bb8d-6404-e596-964a-63bf893e6f41: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.313Z [INFO] creating new snapshot: path=/tmp/raft1145174082/snapshots/2-23-1655909598313.tmp testing.go:153: 2022-06-22T14:53:18.313Z [INFO] creating new snapshot: path=/tmp/raft1234152303/snapshots/2-23-1655909598313.tmp testLoggerAdapter verbose: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.316Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: copied to local snapshot: bytes=71 testing.go:149: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.316Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.316Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: copied to local snapshot: bytes=71 testing.go:149: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.316Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.317Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: Installed remote snapshot testing.go:149: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.317Z [INFO] server-915826de-3e5d-5453-848d-066187ec3f62: Installed remote snapshot testLoggerAdapter verbose: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.317Z [WARN] server-915826de-3e5d-5453-848d-066187ec3f62: skipping application of old log: index=22 testing.go:149: server-915826de-3e5d-5453-848d-066187ec3f62: 2022-06-22T14:53:18.317Z [WARN] server-915826de-3e5d-5453-848d-066187ec3f62: skipping application of old log: index=22 testLoggerAdapter verbose: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.317Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: Installed remote snapshot testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.317Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testing.go:149: server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: 2022-06-22T14:53:18.317Z [INFO] server-af84098b-5c6f-e357-f46f-6b3c1c34efc1: Installed remote snapshot testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.317Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.317Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.317Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: pipelining replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.329Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.329Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-af84098b-5c6f-e357-f46f-6b3c1c34efc1 af84098b-5c6f-e357-f46f-6b3c1c34efc1}" testLoggerAdapter verbose: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.329Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testing.go:149: server-4c82bb8d-6404-e596-964a-63bf893e6f41: 2022-06-22T14:53:18.329Z [INFO] server-4c82bb8d-6404-e596-964a-63bf893e6f41: aborting pipeline replication: peer="{Voter server-915826de-3e5d-5453-848d-066187ec3f62 915826de-3e5d-5453-848d-066187ec3f62}" testLoggerAdapter verbose: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.330Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testing.go:149: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.330Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testLoggerAdapter verbose: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.330Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: entering follower state: follower="Node at 78037867-6519-2c82-1504-1c3b62e3ac99 [Follower]" leader= testing.go:149: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.330Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: entering follower state: follower="Node at 78037867-6519-2c82-1504-1c3b62e3ac99 [Follower]" leader= testLoggerAdapter verbose: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.330Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testing.go:149: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.330Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testLoggerAdapter verbose: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.330Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: entering follower state: follower="Node at c57affeb-7fb1-f61a-4d83-4c6a20a21c23 [Follower]" leader= testing.go:149: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.330Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: entering follower state: follower="Node at c57affeb-7fb1-f61a-4d83-4c6a20a21c23 [Follower]" leader= testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.330Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.330Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-78037867-6519-2c82-1504-1c3b62e3ac99 Address:78037867-6519-2c82-1504-1c3b62e3ac99} {Suffrage:Voter ID:server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 Address:c57affeb-7fb1-f61a-4d83-4c6a20a21c23} {Suffrage:Voter ID:server-8c6ec67d-b669-2d97-7c8a-4137af8efa03 Address:8c6ec67d-b669-2d97-7c8a-4137af8efa03}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.330Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:18.330Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.331Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering follower state: follower="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Follower]" leader= testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.331Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering follower state: follower="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Follower]" leader= testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [WARN] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [WARN] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering candidate state: node="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Candidate]" term=2 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering candidate state: node="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Candidate]" term=2 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: election won: tally=2 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: election won: tally=2 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering leader state: leader="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Leader]" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: entering leader state: leader="Node at 8c6ec67d-b669-2d97-7c8a-4137af8efa03 [Leader]" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: added peer, starting replication: peer=server-78037867-6519-2c82-1504-1c3b62e3ac99 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: added peer, starting replication: peer=server-78037867-6519-2c82-1504-1c3b62e3ac99 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: added peer, starting replication: peer=server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.411Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: added peer, starting replication: peer=server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.412Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.412Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.412Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.412Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.517Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.411835335 +0000 UTC m=+11.226974891 (1 nodes), 80.774912ms from start of poll, 81.488052ms from cluster start. Timeout at 2022-06-22 14:53:18.517293358 +0000 UTC m=+11.332432914, 105.458023ms after stability testing.go:149: cluster: 2022-06-22T14:53:18.517Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.411835335 +0000 UTC m=+11.226974891 (1 nodes), 80.774912ms from start of poll, 81.488052ms from cluster start. Timeout at 2022-06-22 14:53:18.517293358 +0000 UTC m=+11.332432914, 105.458023ms after stability testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.517Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: starting snapshot up to: index=12 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.517Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:18.517Z [INFO] creating new snapshot: path=/tmp/raft1226891374/snapshots/2-12-1655909598517.tmp testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.523Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: snapshot complete up to: index=12 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.523Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:18.523Z [INFO] creating new snapshot: path=/tmp/raft1226891374/snapshots/2-23-1655909598523.tmp testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.527Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: copied to local snapshot: bytes=71 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.527Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: restored user snapshot: index=1 testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: restored user snapshot: index=1 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.528Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.535Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.535Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.536Z [INFO] creating new snapshot: path=/tmp/raft2225572981/snapshots/2-23-1655909598536.tmp testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.538Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.538Z [ERROR] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.540Z [INFO] creating new snapshot: path=/tmp/raft1402241384/snapshots/2-23-1655909598540.tmp testLoggerAdapter verbose: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: copied to local snapshot: bytes=71 testing.go:149: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: Installed remote snapshot testing.go:149: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [INFO] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: Installed remote snapshot testLoggerAdapter verbose: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [WARN] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: skipping application of old log: index=22 testing.go:149: server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: 2022-06-22T14:53:18.541Z [WARN] server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23: skipping application of old log: index=22 testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.541Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.541Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testLoggerAdapter verbose: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.544Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: copied to local snapshot: bytes=71 testing.go:149: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.544Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.545Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: Installed remote snapshot testing.go:149: server-78037867-6519-2c82-1504-1c3b62e3ac99: 2022-06-22T14:53:18.545Z [INFO] server-78037867-6519-2c82-1504-1c3b62e3ac99: Installed remote snapshot testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.545Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.545Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: pipelining replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.558Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.558Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-c57affeb-7fb1-f61a-4d83-4c6a20a21c23 c57affeb-7fb1-f61a-4d83-4c6a20a21c23}" testLoggerAdapter verbose: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.558Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testing.go:149: server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: 2022-06-22T14:53:18.558Z [INFO] server-8c6ec67d-b669-2d97-7c8a-4137af8efa03: aborting pipeline replication: peer="{Voter server-78037867-6519-2c82-1504-1c3b62e3ac99 78037867-6519-2c82-1504-1c3b62e3ac99}" testLoggerAdapter verbose: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.559Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testing.go:149: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.559Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.560Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.560Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testLoggerAdapter verbose: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.560Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testing.go:149: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.560Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa Address:8c9f8ed2-f12e-71f9-1cfa-9fce738771aa} {Suffrage:Voter ID:server-f03129e2-9f67-b974-9faa-215da8ab7939 Address:f03129e2-9f67-b974-9faa-215da8ab7939} {Suffrage:Voter ID:server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 Address:577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}]" testLoggerAdapter verbose: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.560Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: entering follower state: follower="Node at 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa [Follower]" leader= testLoggerAdapter verbose: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.560Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: entering follower state: follower="Node at 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 [Follower]" leader= testing.go:149: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.560Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: entering follower state: follower="Node at 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 [Follower]" leader= testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.560Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering follower state: follower="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Follower]" leader= testing.go:149: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.560Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: entering follower state: follower="Node at 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.560Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.560Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering follower state: follower="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:18.560Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [WARN] server-f03129e2-9f67-b974-9faa-215da8ab7939: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [WARN] server-f03129e2-9f67-b974-9faa-215da8ab7939: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering candidate state: node="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Candidate]" term=2 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering candidate state: node="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Candidate]" term=2 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: election won: tally=2 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: election won: tally=2 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering leader state: leader="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Leader]" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: entering leader state: leader="Node at f03129e2-9f67-b974-9faa-215da8ab7939 [Leader]" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: added peer, starting replication: peer=server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: added peer, starting replication: peer=server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: added peer, starting replication: peer=server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: added peer, starting replication: peer=server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.642Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.748Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.642723138 +0000 UTC m=+11.457862694 (1 nodes), 82.257807ms from start of poll, 82.958072ms from cluster start. Timeout at 2022-06-22 14:53:18.748126748 +0000 UTC m=+11.563266304, 105.40361ms after stability testing.go:149: cluster: 2022-06-22T14:53:18.748Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.642723138 +0000 UTC m=+11.457862694 (1 nodes), 82.257807ms from start of poll, 82.958072ms from cluster start. Timeout at 2022-06-22 14:53:18.748126748 +0000 UTC m=+11.563266304, 105.40361ms after stability testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.748Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: starting snapshot up to: index=12 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.748Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:18.748Z [INFO] creating new snapshot: path=/tmp/raft1083818451/snapshots/2-12-1655909598748.tmp testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.752Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: snapshot complete up to: index=12 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.752Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:18.753Z [INFO] creating new snapshot: path=/tmp/raft1083818451/snapshots/2-23-1655909598753.tmp testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.757Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: copied to local snapshot: bytes=71 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.757Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: restored user snapshot: index=1 testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: restored user snapshot: index=1 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.758Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.764Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.764Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.765Z [INFO] creating new snapshot: path=/tmp/raft137100234/snapshots/2-23-1655909598765.tmp testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.766Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.766Z [ERROR] server-f03129e2-9f67-b974-9faa-215da8ab7939: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.767Z [INFO] creating new snapshot: path=/tmp/raft2698232228/snapshots/2-23-1655909598767.tmp testLoggerAdapter verbose: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.768Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: copied to local snapshot: bytes=71 testing.go:149: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.768Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.769Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: Installed remote snapshot testing.go:149: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.769Z [INFO] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: Installed remote snapshot testLoggerAdapter verbose: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.769Z [WARN] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: skipping application of old log: index=22 testing.go:149: server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: 2022-06-22T14:53:18.769Z [WARN] server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa: skipping application of old log: index=22 testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.769Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.769Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testLoggerAdapter verbose: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.770Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: copied to local snapshot: bytes=71 testing.go:149: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.770Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.770Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: Installed remote snapshot testing.go:149: server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: 2022-06-22T14:53:18.770Z [INFO] server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4: Installed remote snapshot testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.771Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.771Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: pipelining replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.785Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.785Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-577b8e4b-c8fc-bb62-672a-b4877cc7c6d4 577b8e4b-c8fc-bb62-672a-b4877cc7c6d4}" testLoggerAdapter verbose: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.785Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testing.go:149: server-f03129e2-9f67-b974-9faa-215da8ab7939: 2022-06-22T14:53:18.785Z [INFO] server-f03129e2-9f67-b974-9faa-215da8ab7939: aborting pipeline replication: peer="{Voter server-8c9f8ed2-f12e-71f9-1cfa-9fce738771aa 8c9f8ed2-f12e-71f9-1cfa-9fce738771aa}" testLoggerAdapter verbose: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.786Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testing.go:149: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.786Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testLoggerAdapter verbose: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.787Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testing.go:149: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.787Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.787Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.787Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d9147728-b379-4451-6c36-28ff5ab4494c Address:d9147728-b379-4451-6c36-28ff5ab4494c} {Suffrage:Voter ID:server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc Address:505b11fe-74d4-3e6b-c095-2a1d39a28ffc} {Suffrage:Voter ID:server-a69b2368-e875-41e4-4a5f-e9d9513203b6 Address:a69b2368-e875-41e4-4a5f-e9d9513203b6}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.787Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:18.787Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.787Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: entering follower state: follower="Node at d9147728-b379-4451-6c36-28ff5ab4494c [Follower]" leader= testing.go:149: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.787Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: entering follower state: follower="Node at d9147728-b379-4451-6c36-28ff5ab4494c [Follower]" leader= testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.787Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering follower state: follower="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Follower]" leader= testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.787Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering follower state: follower="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Follower]" leader= testLoggerAdapter verbose: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.787Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: entering follower state: follower="Node at 505b11fe-74d4-3e6b-c095-2a1d39a28ffc [Follower]" leader= testing.go:149: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.787Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: entering follower state: follower="Node at 505b11fe-74d4-3e6b-c095-2a1d39a28ffc [Follower]" leader= testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [WARN] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [WARN] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering candidate state: node="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Candidate]" term=2 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering candidate state: node="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Candidate]" term=2 testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: election won: tally=2 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: election won: tally=2 testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering leader state: leader="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Leader]" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: entering leader state: leader="Node at a69b2368-e875-41e4-4a5f-e9d9513203b6 [Leader]" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: added peer, starting replication: peer=server-d9147728-b379-4451-6c36-28ff5ab4494c testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: added peer, starting replication: peer=server-d9147728-b379-4451-6c36-28ff5ab4494c testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: added peer, starting replication: peer=server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: added peer, starting replication: peer=server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.845Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.846Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.846Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.951Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.845957964 +0000 UTC m=+11.661097510 (1 nodes), 58.731315ms from start of poll, 59.0356ms from cluster start. Timeout at 2022-06-22 14:53:18.951547145 +0000 UTC m=+11.766686691, 105.589181ms after stability testing.go:149: cluster: 2022-06-22T14:53:18.951Z [INFO] stable state for Leader reached at 2022-06-22 14:53:18.845957964 +0000 UTC m=+11.661097510 (1 nodes), 58.731315ms from start of poll, 59.0356ms from cluster start. Timeout at 2022-06-22 14:53:18.951547145 +0000 UTC m=+11.766686691, 105.589181ms after stability testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.952Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: starting snapshot up to: index=12 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.952Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:18.952Z [INFO] creating new snapshot: path=/tmp/raft959985312/snapshots/2-12-1655909598952.tmp testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.956Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: snapshot complete up to: index=12 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.956Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:18.957Z [INFO] creating new snapshot: path=/tmp/raft959985312/snapshots/2-113-1655909598957.tmp testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: copied to local snapshot: bytes=71 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: restored user snapshot: index=1 testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: restored user snapshot: index=1 testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.961Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.970Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.970Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.970Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.970Z [ERROR] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:18.971Z [INFO] creating new snapshot: path=/tmp/raft1124869635/snapshots/2-113-1655909598971.tmp testing.go:153: 2022-06-22T14:53:18.971Z [INFO] creating new snapshot: path=/tmp/raft1005899996/snapshots/2-113-1655909598971.tmp testLoggerAdapter verbose: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: copied to local snapshot: bytes=71 testing.go:149: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: Installed remote snapshot testing.go:149: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [INFO] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: Installed remote snapshot testLoggerAdapter verbose: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [WARN] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: skipping application of old log: index=22 testing.go:149: server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: 2022-06-22T14:53:18.976Z [WARN] server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc: skipping application of old log: index=22 testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.977Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.977Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testLoggerAdapter verbose: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.977Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: copied to local snapshot: bytes=71 testing.go:149: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.977Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.977Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: Installed remote snapshot testing.go:149: server-d9147728-b379-4451-6c36-28ff5ab4494c: 2022-06-22T14:53:18.977Z [INFO] server-d9147728-b379-4451-6c36-28ff5ab4494c: Installed remote snapshot testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.977Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.977Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: pipelining replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.994Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.994Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-505b11fe-74d4-3e6b-c095-2a1d39a28ffc 505b11fe-74d4-3e6b-c095-2a1d39a28ffc}" testLoggerAdapter verbose: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.994Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testing.go:149: server-a69b2368-e875-41e4-4a5f-e9d9513203b6: 2022-06-22T14:53:18.994Z [INFO] server-a69b2368-e875-41e4-4a5f-e9d9513203b6: aborting pipeline replication: peer="{Voter server-d9147728-b379-4451-6c36-28ff5ab4494c d9147728-b379-4451-6c36-28ff5ab4494c}" testLoggerAdapter verbose: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:18.996Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testing.go:149: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:18.996Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testLoggerAdapter verbose: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:18.996Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: entering follower state: follower="Node at 923e170a-9fe6-12f2-cbd7-89c10f08a953 [Follower]" leader= testing.go:149: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:18.996Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: entering follower state: follower="Node at 923e170a-9fe6-12f2-cbd7-89c10f08a953 [Follower]" leader= testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:18.996Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:18.996Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:18.996Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering follower state: follower="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Follower]" leader= testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:18.996Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering follower state: follower="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Follower]" leader= testLoggerAdapter verbose: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:18.996Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testing.go:149: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:18.996Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-923e170a-9fe6-12f2-cbd7-89c10f08a953 Address:923e170a-9fe6-12f2-cbd7-89c10f08a953} {Suffrage:Voter ID:server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 Address:c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1} {Suffrage:Voter ID:server-30f32feb-b3d3-37da-ab1f-b57d70ca368c Address:30f32feb-b3d3-37da-ab1f-b57d70ca368c}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:18.996Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:18.996Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:18.996Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: entering follower state: follower="Node at 30f32feb-b3d3-37da-ab1f-b57d70ca368c [Follower]" leader= testing.go:149: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:18.996Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: entering follower state: follower="Node at 30f32feb-b3d3-37da-ab1f-b57d70ca368c [Follower]" leader= testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [WARN] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [WARN] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering candidate state: node="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Candidate]" term=2 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering candidate state: node="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Candidate]" term=2 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: election won: tally=2 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: election won: tally=2 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering leader state: leader="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Leader]" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.063Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: entering leader state: leader="Node at c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1 [Leader]" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: added peer, starting replication: peer=server-923e170a-9fe6-12f2-cbd7-89c10f08a953 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: added peer, starting replication: peer=server-923e170a-9fe6-12f2-cbd7-89c10f08a953 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: added peer, starting replication: peer=server-30f32feb-b3d3-37da-ab1f-b57d70ca368c testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: added peer, starting replication: peer=server-30f32feb-b3d3-37da-ab1f-b57d70ca368c testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.064Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.169Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.064155961 +0000 UTC m=+11.879295507 (1 nodes), 67.402295ms from start of poll, 68.115274ms from cluster start. Timeout at 2022-06-22 14:53:19.169850747 +0000 UTC m=+11.984990303, 105.694796ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.169Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.064155961 +0000 UTC m=+11.879295507 (1 nodes), 67.402295ms from start of poll, 68.115274ms from cluster start. Timeout at 2022-06-22 14:53:19.169850747 +0000 UTC m=+11.984990303, 105.694796ms after stability testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.172Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: starting snapshot up to: index=12 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.172Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:19.172Z [INFO] creating new snapshot: path=/tmp/raft707135263/snapshots/2-12-1655909599172.tmp testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.176Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: snapshot complete up to: index=12 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.176Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:19.177Z [INFO] creating new snapshot: path=/tmp/raft707135263/snapshots/2-1013-1655909599177.tmp testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: copied to local snapshot: bytes=71 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: restored user snapshot: index=1 testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: restored user snapshot: index=1 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.180Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.181Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.186Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.186Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:19.187Z [INFO] creating new snapshot: path=/tmp/raft3441675326/snapshots/2-1013-1655909599187.tmp testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.189Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.189Z [ERROR] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:19.189Z [INFO] creating new snapshot: path=/tmp/raft2165033697/snapshots/2-1013-1655909599189.tmp testLoggerAdapter verbose: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.190Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: copied to local snapshot: bytes=71 testing.go:149: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.190Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.191Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: Installed remote snapshot testing.go:149: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.191Z [INFO] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: Installed remote snapshot testLoggerAdapter verbose: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.191Z [WARN] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: skipping application of old log: index=22 testing.go:149: server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: 2022-06-22T14:53:19.191Z [WARN] server-30f32feb-b3d3-37da-ab1f-b57d70ca368c: skipping application of old log: index=22 testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.191Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.191Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testLoggerAdapter verbose: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:19.192Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: copied to local snapshot: bytes=71 testing.go:149: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:19.192Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:19.193Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: Installed remote snapshot testing.go:149: server-923e170a-9fe6-12f2-cbd7-89c10f08a953: 2022-06-22T14:53:19.193Z [INFO] server-923e170a-9fe6-12f2-cbd7-89c10f08a953: Installed remote snapshot testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.193Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.193Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: pipelining replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.208Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.208Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-30f32feb-b3d3-37da-ab1f-b57d70ca368c 30f32feb-b3d3-37da-ab1f-b57d70ca368c}" testLoggerAdapter verbose: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.208Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testing.go:149: server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: 2022-06-22T14:53:19.208Z [INFO] server-c783d2b2-7c62-dfe0-9f65-7c65f7ddbda1: aborting pipeline replication: peer="{Voter server-923e170a-9fe6-12f2-cbd7-89c10f08a953 923e170a-9fe6-12f2-cbd7-89c10f08a953}" testLoggerAdapter verbose: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.210Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testing.go:149: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.210Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.210Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.210Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testLoggerAdapter verbose: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.210Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: entering follower state: follower="Node at b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 [Follower]" leader= testing.go:149: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.210Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: entering follower state: follower="Node at b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 [Follower]" leader= testLoggerAdapter verbose: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.210Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testing.go:149: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.210Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 Address:b7613e8c-e5a6-7fbc-5707-722e7a4a86b0} {Suffrage:Voter ID:server-5fb2df84-71b2-2d16-4595-eb761b1e339e Address:5fb2df84-71b2-2d16-4595-eb761b1e339e} {Suffrage:Voter ID:server-c16843da-0957-d656-c2a0-89b7edb72db6 Address:c16843da-0957-d656-c2a0-89b7edb72db6}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.210Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:19.210Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.210Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering follower state: follower="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Follower]" leader= testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.210Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering follower state: follower="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Follower]" leader= testLoggerAdapter verbose: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.210Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: entering follower state: follower="Node at c16843da-0957-d656-c2a0-89b7edb72db6 [Follower]" leader= testing.go:149: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.210Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: entering follower state: follower="Node at c16843da-0957-d656-c2a0-89b7edb72db6 [Follower]" leader= testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [WARN] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [WARN] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering candidate state: node="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Candidate]" term=2 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering candidate state: node="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Candidate]" term=2 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: election won: tally=2 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: election won: tally=2 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering leader state: leader="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Leader]" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: entering leader state: leader="Node at 5fb2df84-71b2-2d16-4595-eb761b1e339e [Leader]" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: added peer, starting replication: peer=server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.262Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: added peer, starting replication: peer=server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: added peer, starting replication: peer=server-c16843da-0957-d656-c2a0-89b7edb72db6 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: added peer, starting replication: peer=server-c16843da-0957-d656-c2a0-89b7edb72db6 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.263Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.368Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.263065616 +0000 UTC m=+12.078205162 (1 nodes), 52.659948ms from start of poll, 53.146098ms from cluster start. Timeout at 2022-06-22 14:53:19.368453692 +0000 UTC m=+12.183593238, 105.388076ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.368Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.263065616 +0000 UTC m=+12.078205162 (1 nodes), 52.659948ms from start of poll, 53.146098ms from cluster start. Timeout at 2022-06-22 14:53:19.368453692 +0000 UTC m=+12.183593238, 105.388076ms after stability testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.368Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: starting snapshot up to: index=12 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.368Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: starting snapshot up to: index=12 testing.go:153: 2022-06-22T14:53:19.368Z [INFO] creating new snapshot: path=/tmp/raft2189477821/snapshots/2-12-1655909599368.tmp testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.373Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: snapshot complete up to: index=12 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.373Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: snapshot complete up to: index=12 testing.go:153: 2022-06-22T14:53:19.373Z [INFO] creating new snapshot: path=/tmp/raft2189477821/snapshots/2-10013-1655909599373.tmp testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: copied to local snapshot: bytes=71 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: restored user snapshot: index=1 testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: restored user snapshot: index=1 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.377Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.378Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.378Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.386Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.386Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:19.386Z [INFO] creating new snapshot: path=/tmp/raft134856612/snapshots/2-10013-1655909599386.tmp testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.388Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.388Z [ERROR] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: failed to get log: index=23 error="log not found" testing.go:153: 2022-06-22T14:53:19.389Z [INFO] creating new snapshot: path=/tmp/raft1528782611/snapshots/2-10013-1655909599389.tmp testLoggerAdapter verbose: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.391Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: copied to local snapshot: bytes=71 testing.go:149: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.391Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.392Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: Installed remote snapshot testing.go:149: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.392Z [INFO] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: Installed remote snapshot testLoggerAdapter verbose: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.392Z [WARN] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: skipping application of old log: index=22 testing.go:149: server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: 2022-06-22T14:53:19.392Z [WARN] server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0: skipping application of old log: index=22 testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.392Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.392Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testLoggerAdapter verbose: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.392Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: copied to local snapshot: bytes=71 testing.go:149: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.392Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: copied to local snapshot: bytes=71 testLoggerAdapter verbose: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.393Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: Installed remote snapshot testing.go:149: server-c16843da-0957-d656-c2a0-89b7edb72db6: 2022-06-22T14:53:19.393Z [INFO] server-c16843da-0957-d656-c2a0-89b7edb72db6: Installed remote snapshot testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.393Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.393Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: pipelining replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.409Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.409Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-c16843da-0957-d656-c2a0-89b7edb72db6 c16843da-0957-d656-c2a0-89b7edb72db6}" testLoggerAdapter verbose: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.409Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" testing.go:149: server-5fb2df84-71b2-2d16-4595-eb761b1e339e: 2022-06-22T14:53:19.409Z [INFO] server-5fb2df84-71b2-2d16-4595-eb761b1e339e: aborting pipeline replication: peer="{Voter server-b7613e8c-e5a6-7fbc-5707-722e7a4a86b0 b7613e8c-e5a6-7fbc-5707-722e7a4a86b0}" --- PASS: TestRaft_UserRestore (1.27s) === RUN TestRaft_SendSnapshotFollower testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.410Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.410Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.410Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.410Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.410Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: entering follower state: follower="Node at 0d6d839e-fd5a-3866-a634-cc8972cf8242 [Follower]" leader= testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.410Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: entering follower state: follower="Node at 0d6d839e-fd5a-3866-a634-cc8972cf8242 [Follower]" leader= testLoggerAdapter verbose: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.411Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testing.go:149: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.411Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0d6d839e-fd5a-3866-a634-cc8972cf8242 Address:0d6d839e-fd5a-3866-a634-cc8972cf8242} {Suffrage:Voter ID:server-419476db-0329-72dd-22ce-57a95b18ba8f Address:419476db-0329-72dd-22ce-57a95b18ba8f} {Suffrage:Voter ID:server-49bb6f79-006f-ab21-060b-db5fa98918cd Address:49bb6f79-006f-ab21-060b-db5fa98918cd}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.411Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:19.411Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.411Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering follower state: follower="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Follower]" leader= testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.411Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering follower state: follower="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Follower]" leader= testLoggerAdapter verbose: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.411Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: entering follower state: follower="Node at 49bb6f79-006f-ab21-060b-db5fa98918cd [Follower]" leader= testing.go:149: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.411Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: entering follower state: follower="Node at 49bb6f79-006f-ab21-060b-db5fa98918cd [Follower]" leader= testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [WARN] server-419476db-0329-72dd-22ce-57a95b18ba8f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [WARN] server-419476db-0329-72dd-22ce-57a95b18ba8f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering candidate state: node="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Candidate]" term=2 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering candidate state: node="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Candidate]" term=2 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: election won: tally=2 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: election won: tally=2 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering leader state: leader="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Leader]" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: entering leader state: leader="Node at 419476db-0329-72dd-22ce-57a95b18ba8f [Leader]" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: added peer, starting replication: peer=server-0d6d839e-fd5a-3866-a634-cc8972cf8242 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: added peer, starting replication: peer=server-0d6d839e-fd5a-3866-a634-cc8972cf8242 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: added peer, starting replication: peer=server-49bb6f79-006f-ab21-060b-db5fa98918cd testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: added peer, starting replication: peer=server-49bb6f79-006f-ab21-060b-db5fa98918cd testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.468Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.469Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-49bb6f79-006f-ab21-060b-db5fa98918cd 49bb6f79-006f-ab21-060b-db5fa98918cd}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.469Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-49bb6f79-006f-ab21-060b-db5fa98918cd 49bb6f79-006f-ab21-060b-db5fa98918cd}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.574Z [INFO] stable state for Follower reached at 2022-06-22 14:53:19.468893608 +0000 UTC m=+12.284033284 (2 nodes), 57.796434ms from start of poll, 58.111009ms from cluster start. Timeout at 2022-06-22 14:53:19.574832966 +0000 UTC m=+12.389972512, 105.939228ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.574Z [INFO] stable state for Follower reached at 2022-06-22 14:53:19.468893608 +0000 UTC m=+12.284033284 (2 nodes), 57.796434ms from start of poll, 58.111009ms from cluster start. Timeout at 2022-06-22 14:53:19.574832966 +0000 UTC m=+12.389972512, 105.939228ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.574Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:19.574Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.680Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.574959405 +0000 UTC m=+12.390098951 (1 nodes), 921ns from start of poll, 164.176676ms from cluster start. Timeout at 2022-06-22 14:53:19.680481225 +0000 UTC m=+12.495620781, 105.52183ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.680Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.574959405 +0000 UTC m=+12.390098951 (1 nodes), 921ns from start of poll, 164.176676ms from cluster start. Timeout at 2022-06-22 14:53:19.680481225 +0000 UTC m=+12.495620781, 105.52183ms after stability testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to pipeline appendEntries: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" error="append pipeline closed" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to pipeline appendEntries: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" error="append pipeline closed" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to appendEntries to: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" error="failed to connect to peer: 0d6d839e-fd5a-3866-a634-cc8972cf8242" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.680Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to appendEntries to: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" error="failed to connect to peer: 0d6d839e-fd5a-3866-a634-cc8972cf8242" raft_test.go:1320: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.683Z [ERROR] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.683Z [ERROR] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.683Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: starting snapshot up to: index=102 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.683Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: starting snapshot up to: index=102 testing.go:153: 2022-06-22T14:53:19.683Z [INFO] creating new snapshot: path=/tmp/raft2011638969/snapshots/2-102-1655909599683.tmp testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.685Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to heartbeat to: peer=0d6d839e-fd5a-3866-a634-cc8972cf8242 error="failed to connect to peer: 0d6d839e-fd5a-3866-a634-cc8972cf8242" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.685Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to heartbeat to: peer=0d6d839e-fd5a-3866-a634-cc8972cf8242 error="failed to connect to peer: 0d6d839e-fd5a-3866-a634-cc8972cf8242" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.687Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: compacting logs: from=1 to=92 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.687Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.687Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: snapshot complete up to: index=102 testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.687Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: snapshot complete up to: index=102 testLoggerAdapter verbose: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.687Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: starting snapshot up to: index=99 testing.go:149: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.687Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: starting snapshot up to: index=99 testing.go:153: 2022-06-22T14:53:19.687Z [INFO] creating new snapshot: path=/tmp/raft1515618213/snapshots/2-99-1655909599687.tmp testLoggerAdapter verbose: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.692Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: compacting logs: from=1 to=92 testing.go:149: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.692Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.692Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: snapshot complete up to: index=99 testing.go:149: server-49bb6f79-006f-ab21-060b-db5fa98918cd: 2022-06-22T14:53:19.692Z [INFO] server-49bb6f79-006f-ab21-060b-db5fa98918cd: snapshot complete up to: index=99 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.692Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to get log: index=2 error="log not found" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.692Z [ERROR] server-419476db-0329-72dd-22ce-57a95b18ba8f: failed to get log: index=2 error="log not found" testing.go:153: 2022-06-22T14:53:19.693Z [INFO] creating new snapshot: path=/tmp/raft3783512469/snapshots/2-102-1655909599693.tmp testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.696Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: copied to local snapshot: bytes=693 testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.696Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.697Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: Installed remote snapshot testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.697Z [INFO] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: Installed remote snapshot testLoggerAdapter verbose: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.697Z [WARN] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: skipping application of old log: index=102 testing.go:149: server-0d6d839e-fd5a-3866-a634-cc8972cf8242: 2022-06-22T14:53:19.697Z [WARN] server-0d6d839e-fd5a-3866-a634-cc8972cf8242: skipping application of old log: index=102 testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: pipelining replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-49bb6f79-006f-ab21-060b-db5fa98918cd 49bb6f79-006f-ab21-060b-db5fa98918cd}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-49bb6f79-006f-ab21-060b-db5fa98918cd 49bb6f79-006f-ab21-060b-db5fa98918cd}" testLoggerAdapter verbose: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" testing.go:149: server-419476db-0329-72dd-22ce-57a95b18ba8f: 2022-06-22T14:53:19.698Z [INFO] server-419476db-0329-72dd-22ce-57a95b18ba8f: aborting pipeline replication: peer="{Voter server-0d6d839e-fd5a-3866-a634-cc8972cf8242 0d6d839e-fd5a-3866-a634-cc8972cf8242}" --- PASS: TestRaft_SendSnapshotFollower (0.29s) === RUN TestRaft_SendSnapshotAndLogsFollower testLoggerAdapter verbose: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.700Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testing.go:149: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.700Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.700Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.700Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testLoggerAdapter verbose: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.700Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: entering follower state: follower="Node at 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b [Follower]" leader= testing.go:149: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.700Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: entering follower state: follower="Node at 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b [Follower]" leader= testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.700Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering follower state: follower="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Follower]" leader= testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.700Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering follower state: follower="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Follower]" leader= testLoggerAdapter verbose: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.700Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testing.go:149: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.700Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b Address:11bca171-7f3f-d0b1-6b83-af43aaa0ee7b} {Suffrage:Voter ID:server-5abd5d9c-865e-70ee-8a21-cd68954bef05 Address:5abd5d9c-865e-70ee-8a21-cd68954bef05} {Suffrage:Voter ID:server-f195aad0-ec4b-0a51-c2e3-b49806462490 Address:f195aad0-ec4b-0a51-c2e3-b49806462490}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.700Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:19.700Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.700Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: entering follower state: follower="Node at f195aad0-ec4b-0a51-c2e3-b49806462490 [Follower]" leader= testing.go:149: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.700Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: entering follower state: follower="Node at f195aad0-ec4b-0a51-c2e3-b49806462490 [Follower]" leader= testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.755Z [WARN] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.755Z [WARN] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering candidate state: node="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Candidate]" term=2 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering candidate state: node="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Candidate]" term=2 testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: election won: tally=2 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: election won: tally=2 testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering leader state: leader="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Leader]" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: entering leader state: leader="Node at 5abd5d9c-865e-70ee-8a21-cd68954bef05 [Leader]" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: added peer, starting replication: peer=server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: added peer, starting replication: peer=server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: added peer, starting replication: peer=server-f195aad0-ec4b-0a51-c2e3-b49806462490 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: added peer, starting replication: peer=server-f195aad0-ec4b-0a51-c2e3-b49806462490 testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-f195aad0-ec4b-0a51-c2e3-b49806462490 f195aad0-ec4b-0a51-c2e3-b49806462490}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.756Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-f195aad0-ec4b-0a51-c2e3-b49806462490 f195aad0-ec4b-0a51-c2e3-b49806462490}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.862Z [INFO] stable state for Follower reached at 2022-06-22 14:53:19.756486832 +0000 UTC m=+12.571626378 (2 nodes), 55.84328ms from start of poll, 56.370147ms from cluster start. Timeout at 2022-06-22 14:53:19.862214981 +0000 UTC m=+12.677354527, 105.728149ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.862Z [INFO] stable state for Follower reached at 2022-06-22 14:53:19.756486832 +0000 UTC m=+12.571626378 (2 nodes), 55.84328ms from start of poll, 56.370147ms from cluster start. Timeout at 2022-06-22 14:53:19.862214981 +0000 UTC m=+12.677354527, 105.728149ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.862Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:19.862Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.968Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.862569733 +0000 UTC m=+12.677709279 (1 nodes), 2.034µs from start of poll, 162.453048ms from cluster start. Timeout at 2022-06-22 14:53:19.968531494 +0000 UTC m=+12.783671090, 105.961811ms after stability testing.go:149: cluster: 2022-06-22T14:53:19.968Z [INFO] stable state for Leader reached at 2022-06-22 14:53:19.862569733 +0000 UTC m=+12.677709279 (1 nodes), 2.034µs from start of poll, 162.453048ms from cluster start. Timeout at 2022-06-22 14:53:19.968531494 +0000 UTC m=+12.783671090, 105.961811ms after stability testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to pipeline appendEntries: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" error="append pipeline closed" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to pipeline appendEntries: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" error="append pipeline closed" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to appendEntries to: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" error="failed to connect to peer: 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.968Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to appendEntries to: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" error="failed to connect to peer: 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b" raft_test.go:1362: [INFO] Finished apply without behind follower testLoggerAdapter verbose: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.970Z [ERROR] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: failed to take snapshot: error="nothing new to snapshot" testing.go:149: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.970Z [ERROR] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: failed to take snapshot: error="nothing new to snapshot" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.970Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: starting snapshot up to: index=102 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.970Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: starting snapshot up to: index=102 testing.go:153: 2022-06-22T14:53:19.970Z [INFO] creating new snapshot: path=/tmp/raft1605899138/snapshots/2-102-1655909599970.tmp testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: compacting logs: from=1 to=92 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: compacting logs: from=1 to=92 testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: snapshot complete up to: index=102 testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: snapshot complete up to: index=102 testLoggerAdapter verbose: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.974Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: starting snapshot up to: index=54 testing.go:149: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.974Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: starting snapshot up to: index=54 testing.go:153: 2022-06-22T14:53:19.974Z [INFO] creating new snapshot: path=/tmp/raft92940402/snapshots/2-54-1655909599974.tmp testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to heartbeat to: peer=11bca171-7f3f-d0b1-6b83-af43aaa0ee7b error="failed to connect to peer: 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.974Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to heartbeat to: peer=11bca171-7f3f-d0b1-6b83-af43aaa0ee7b error="failed to connect to peer: 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b" testLoggerAdapter verbose: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.977Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: compacting logs: from=1 to=54 testing.go:149: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.977Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: compacting logs: from=1 to=54 testLoggerAdapter verbose: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.977Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: snapshot complete up to: index=54 testing.go:149: server-f195aad0-ec4b-0a51-c2e3-b49806462490: 2022-06-22T14:53:19.977Z [INFO] server-f195aad0-ec4b-0a51-c2e3-b49806462490: snapshot complete up to: index=54 testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.978Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to get log: index=2 error="log not found" raft_test.go:1383: [INFO] Finished apply without behind follower testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.978Z [ERROR] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: failed to get log: index=2 error="log not found" testing.go:153: 2022-06-22T14:53:19.979Z [INFO] creating new snapshot: path=/tmp/raft1683401249/snapshots/2-102-1655909599979.tmp testLoggerAdapter verbose: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.983Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: copied to local snapshot: bytes=693 testing.go:149: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.983Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: copied to local snapshot: bytes=693 testLoggerAdapter verbose: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.983Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: Installed remote snapshot testing.go:149: server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: 2022-06-22T14:53:19.983Z [INFO] server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b: Installed remote snapshot testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.984Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.984Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: pipelining replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.990Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-f195aad0-ec4b-0a51-c2e3-b49806462490 f195aad0-ec4b-0a51-c2e3-b49806462490}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.990Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-f195aad0-ec4b-0a51-c2e3-b49806462490 f195aad0-ec4b-0a51-c2e3-b49806462490}" testLoggerAdapter verbose: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.990Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" testing.go:149: server-5abd5d9c-865e-70ee-8a21-cd68954bef05: 2022-06-22T14:53:19.990Z [INFO] server-5abd5d9c-865e-70ee-8a21-cd68954bef05: aborting pipeline replication: peer="{Voter server-11bca171-7f3f-d0b1-6b83-af43aaa0ee7b 11bca171-7f3f-d0b1-6b83-af43aaa0ee7b}" --- PASS: TestRaft_SendSnapshotAndLogsFollower (0.29s) === RUN TestRaft_ReJoinFollower testLoggerAdapter verbose: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:19.991Z [INFO] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testing.go:149: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:19.991Z [INFO] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testLoggerAdapter verbose: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:19.992Z [INFO] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: entering follower state: follower="Node at df7f67c9-93c2-4a77-7389-34deb23dbba7 [Follower]" leader= testing.go:149: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:19.992Z [INFO] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: entering follower state: follower="Node at df7f67c9-93c2-4a77-7389-34deb23dbba7 [Follower]" leader= testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:19.992Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:19.992Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testLoggerAdapter verbose: server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: 2022-06-22T14:53:19.992Z [INFO] server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:19.992Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering follower state: follower="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Follower]" leader= testing.go:149: server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: 2022-06-22T14:53:19.992Z [INFO] server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7} {Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:19.992Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:19.992Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:19.992Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering follower state: follower="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Follower]" leader= testLoggerAdapter verbose: server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: 2022-06-22T14:53:19.992Z [INFO] server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: entering follower state: follower="Node at 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e [Follower]" leader= testing.go:149: server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: 2022-06-22T14:53:19.992Z [INFO] server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e: entering follower state: follower="Node at 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e [Follower]" leader= testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [WARN] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [WARN] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering candidate state: node="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Candidate]" term=2 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering candidate state: node="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Candidate]" term=2 testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: election won: tally=2 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: election won: tally=2 testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering leader state: leader="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Leader]" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: entering leader state: leader="Node at 05fde1c2-9c50-a070-0f50-1c1d479a12aa [Leader]" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.055Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.161Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.055832023 +0000 UTC m=+12.870971579 (1 nodes), 63.316693ms from start of poll, 64.051343ms from cluster start. Timeout at 2022-06-22 14:53:20.161285558 +0000 UTC m=+12.976425115, 105.453536ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.161Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.055832023 +0000 UTC m=+12.870971579 (1 nodes), 63.316693ms from start of poll, 64.051343ms from cluster start. Timeout at 2022-06-22 14:53:20.161285558 +0000 UTC m=+12.976425115, 105.453536ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.166Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:20.166Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.272Z [INFO] stable state for Follower reached at 2022-06-22 14:53:20.166903976 +0000 UTC m=+12.982043522 (2 nodes), 1.382µs from start of poll, 175.123286ms from cluster start. Timeout at 2022-06-22 14:53:20.27197618 +0000 UTC m=+13.087115726, 105.072204ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.272Z [INFO] stable state for Follower reached at 2022-06-22 14:53:20.166903976 +0000 UTC m=+12.982043522 (2 nodes), 1.382µs from start of poll, 175.123286ms from cluster start. Timeout at 2022-06-22 14:53:20.27197618 +0000 UTC m=+13.087115726, 105.072204ms after stability testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: updating configuration: command=RemoveServer server-id=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 server-addr= servers="[{Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: updating configuration: command=RemoveServer server-id=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 server-addr= servers="[{Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}]" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: removed peer, stopping replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 last-index=3 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: removed peer, stopping replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 last-index=3 testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.272Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testLoggerAdapter verbose: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:20.354Z [WARN] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: not part of stable configuration, aborting election testing.go:149: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:20.354Z [WARN] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: not part of stable configuration, aborting election testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: updating configuration: command=AddStaging server-id=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 server-addr=df7f67c9-93c2-4a77-7389-34deb23dbba7 servers="[{Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e} {Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7}]" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: updating configuration: command=AddStaging server-id=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 server-addr=df7f67c9-93c2-4a77-7389-34deb23dbba7 servers="[{Suffrage:Voter ID:server-05fde1c2-9c50-a070-0f50-1c1d479a12aa Address:05fde1c2-9c50-a070-0f50-1c1d479a12aa} {Suffrage:Voter ID:server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e Address:64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e} {Suffrage:Voter ID:server-df7f67c9-93c2-4a77-7389-34deb23dbba7 Address:df7f67c9-93c2-4a77-7389-34deb23dbba7}]" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: added peer, starting replication: peer=server-df7f67c9-93c2-4a77-7389-34deb23dbba7 testLoggerAdapter verbose: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:20.384Z [WARN] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: failed to get previous log: previous-index=4 last-index=3 error="log not found" testing.go:149: server-df7f67c9-93c2-4a77-7389-34deb23dbba7: 2022-06-22T14:53:20.384Z [WARN] server-df7f67c9-93c2-4a77-7389-34deb23dbba7: failed to get previous log: previous-index=4 last-index=3 error="log not found" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [WARN] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: appendEntries rejected, sending older logs: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" next=4 testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [WARN] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: appendEntries rejected, sending older logs: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" next=4 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.384Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testing.go:149: cluster: 2022-06-22T14:53:20.384Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.384Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: pipelining replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.490Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.384665932 +0000 UTC m=+13.199805478 (1 nodes), 702ns from start of poll, 392.885242ms from cluster start. Timeout at 2022-06-22 14:53:20.490287093 +0000 UTC m=+13.305426649, 105.621171ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.490Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.384665932 +0000 UTC m=+13.199805478 (1 nodes), 702ns from start of poll, 392.885242ms from cluster start. Timeout at 2022-06-22 14:53:20.490287093 +0000 UTC m=+13.305426649, 105.621171ms after stability testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.490Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.490Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e 64cf0230-2c9f-8fd8-66b3-e3044c9a7f2e}" testLoggerAdapter verbose: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.490Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" testing.go:149: server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: 2022-06-22T14:53:20.490Z [INFO] server-05fde1c2-9c50-a070-0f50-1c1d479a12aa: aborting pipeline replication: peer="{Voter server-df7f67c9-93c2-4a77-7389-34deb23dbba7 df7f67c9-93c2-4a77-7389-34deb23dbba7}" --- PASS: TestRaft_ReJoinFollower (0.50s) === RUN TestRaft_LeaderLeaseExpire testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.491Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a57afc4d-94af-510e-2d2b-21b6e105754b Address:a57afc4d-94af-510e-2d2b-21b6e105754b} {Suffrage:Voter ID:server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 Address:d7be16ba-ccb0-61dd-f1a6-e4008c03b464}]" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.491Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a57afc4d-94af-510e-2d2b-21b6e105754b Address:a57afc4d-94af-510e-2d2b-21b6e105754b} {Suffrage:Voter ID:server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 Address:d7be16ba-ccb0-61dd-f1a6-e4008c03b464}]" testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.492Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a57afc4d-94af-510e-2d2b-21b6e105754b Address:a57afc4d-94af-510e-2d2b-21b6e105754b} {Suffrage:Voter ID:server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 Address:d7be16ba-ccb0-61dd-f1a6-e4008c03b464}]" testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.492Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a57afc4d-94af-510e-2d2b-21b6e105754b Address:a57afc4d-94af-510e-2d2b-21b6e105754b} {Suffrage:Voter ID:server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 Address:d7be16ba-ccb0-61dd-f1a6-e4008c03b464}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.492Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:20.492Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.492Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering follower state: follower="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Follower]" leader= testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.492Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering follower state: follower="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Follower]" leader= testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.492Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering follower state: follower="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Follower]" leader= testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.492Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering follower state: follower="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Follower]" leader= testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=2 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=2 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: election won: tally=2 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: election won: tally=2 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering leader state: leader="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Leader]" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering leader state: leader="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Leader]" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: added peer, starting replication: peer=server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: added peer, starting replication: peer=server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: pipelining replication: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.556Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: pipelining replication: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.662Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.556385919 +0000 UTC m=+13.371525475 (1 nodes), 64.203149ms from start of poll, 64.612714ms from cluster start. Timeout at 2022-06-22 14:53:20.662132398 +0000 UTC m=+13.477271944, 105.746469ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.662Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.556385919 +0000 UTC m=+13.371525475 (1 nodes), 64.203149ms from start of poll, 64.612714ms from cluster start. Timeout at 2022-06-22 14:53:20.662132398 +0000 UTC m=+13.477271944, 105.746469ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.667Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:20.667Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.773Z [INFO] stable state for Follower reached at 2022-06-22 14:53:20.667901601 +0000 UTC m=+13.483041147 (1 nodes), 1.303µs from start of poll, 176.128386ms from cluster start. Timeout at 2022-06-22 14:53:20.773329708 +0000 UTC m=+13.588469264, 105.428117ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.773Z [INFO] stable state for Follower reached at 2022-06-22 14:53:20.667901601 +0000 UTC m=+13.483041147 (1 nodes), 1.303µs from start of poll, 176.128386ms from cluster start. Timeout at 2022-06-22 14:53:20.773329708 +0000 UTC m=+13.588469264, 105.428117ms after stability raft_test.go:1492: [INFO] Disconnecting Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Follower] testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.777Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.777Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.779Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to pipeline appendEntries: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="append pipeline closed" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.779Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to pipeline appendEntries: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="append pipeline closed" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.779Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: aborting pipeline replication: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.779Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: aborting pipeline replication: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.786Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.786Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.795Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.795Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.806Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.806Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.814Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.814Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to heartbeat to: peer=d7be16ba-ccb0-61dd-f1a6-e4008c03b464 error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to contact: server-id=server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 time=50.796512ms testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to contact: server-id=server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 time=50.796512ms testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to contact quorum of nodes, stepping down testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to contact quorum of nodes, stepping down testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering follower state: follower="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Follower]" leader= testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.824Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering follower state: follower="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.824Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:20.824Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.825Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.825Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to appendEntries to: peer="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: heartbeat timeout reached, starting election: last-leader=a57afc4d-94af-510e-2d2b-21b6e105754b testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: heartbeat timeout reached, starting election: last-leader=a57afc4d-94af-510e-2d2b-21b6e105754b testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=3 testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=3 testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.869Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.885Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.885Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.886Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=3 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.886Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=3 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.886Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.886Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: Election timeout reached, restarting election testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: Election timeout reached, restarting election testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=4 testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=4 testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:20.970Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:20.974Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.869258826 +0000 UTC m=+13.684398382 (0 nodes), 44.751516ms from start of poll, 377.485621ms from cluster start. Timeout at 2022-06-22 14:53:20.974369664 +0000 UTC m=+13.789509391, 105.111009ms after stability testing.go:149: cluster: 2022-06-22T14:53:20.974Z [INFO] stable state for Leader reached at 2022-06-22 14:53:20.869258826 +0000 UTC m=+13.684398382 (0 nodes), 44.751516ms from start of poll, 377.485621ms from cluster start. Timeout at 2022-06-22 14:53:20.974369664 +0000 UTC m=+13.789509391, 105.111009ms after stability testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: Election timeout reached, restarting election testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: Election timeout reached, restarting election testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=4 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=4 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:20.975Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: Election timeout reached, restarting election testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [WARN] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: Election timeout reached, restarting election testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=5 testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [INFO] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: entering candidate state: node="Node at d7be16ba-ccb0-61dd-f1a6-e4008c03b464 [Candidate]" term=5 testLoggerAdapter verbose: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testing.go:149: server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: 2022-06-22T14:53:21.028Z [ERROR] server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464: failed to make requestVote RPC: target="{Voter server-a57afc4d-94af-510e-2d2b-21b6e105754b a57afc4d-94af-510e-2d2b-21b6e105754b}" error="failed to connect to peer: a57afc4d-94af-510e-2d2b-21b6e105754b" testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.036Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: Election timeout reached, restarting election testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.036Z [WARN] server-a57afc4d-94af-510e-2d2b-21b6e105754b: Election timeout reached, restarting election testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.037Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=5 testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.037Z [INFO] server-a57afc4d-94af-510e-2d2b-21b6e105754b: entering candidate state: node="Node at a57afc4d-94af-510e-2d2b-21b6e105754b [Candidate]" term=5 testLoggerAdapter verbose: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.037Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" testing.go:149: server-a57afc4d-94af-510e-2d2b-21b6e105754b: 2022-06-22T14:53:21.037Z [ERROR] server-a57afc4d-94af-510e-2d2b-21b6e105754b: failed to make requestVote RPC: target="{Voter server-d7be16ba-ccb0-61dd-f1a6-e4008c03b464 d7be16ba-ccb0-61dd-f1a6-e4008c03b464}" error="failed to connect to peer: d7be16ba-ccb0-61dd-f1a6-e4008c03b464" --- PASS: TestRaft_LeaderLeaseExpire (0.59s) === RUN TestRaft_Barrier testLoggerAdapter verbose: server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: 2022-06-22T14:53:21.081Z [INFO] server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testing.go:149: server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: 2022-06-22T14:53:21.081Z [INFO] server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.081Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.081Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testLoggerAdapter verbose: server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: 2022-06-22T14:53:21.081Z [INFO] server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: entering follower state: follower="Node at 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d [Follower]" leader= testLoggerAdapter verbose: server-c6b2cc84-d148-a152-acae-87fec832abdc: 2022-06-22T14:53:21.082Z [INFO] server-c6b2cc84-d148-a152-acae-87fec832abdc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testing.go:149: server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: 2022-06-22T14:53:21.081Z [INFO] server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d: entering follower state: follower="Node at 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d [Follower]" leader= testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.082Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering follower state: follower="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Follower]" leader= testing.go:149: server-c6b2cc84-d148-a152-acae-87fec832abdc: 2022-06-22T14:53:21.082Z [INFO] server-c6b2cc84-d148-a152-acae-87fec832abdc: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d Address:5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d} {Suffrage:Voter ID:server-46668d28-613d-e0d8-c86a-3d1b587cbbd9 Address:46668d28-613d-e0d8-c86a-3d1b587cbbd9} {Suffrage:Voter ID:server-c6b2cc84-d148-a152-acae-87fec832abdc Address:c6b2cc84-d148-a152-acae-87fec832abdc}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.082Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:21.082Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.082Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering follower state: follower="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Follower]" leader= testLoggerAdapter verbose: server-c6b2cc84-d148-a152-acae-87fec832abdc: 2022-06-22T14:53:21.082Z [INFO] server-c6b2cc84-d148-a152-acae-87fec832abdc: entering follower state: follower="Node at c6b2cc84-d148-a152-acae-87fec832abdc [Follower]" leader= testing.go:149: server-c6b2cc84-d148-a152-acae-87fec832abdc: 2022-06-22T14:53:21.082Z [INFO] server-c6b2cc84-d148-a152-acae-87fec832abdc: entering follower state: follower="Node at c6b2cc84-d148-a152-acae-87fec832abdc [Follower]" leader= testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [WARN] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [WARN] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering candidate state: node="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Candidate]" term=2 testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering candidate state: node="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Candidate]" term=2 testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: election won: tally=2 testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: election won: tally=2 testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering leader state: leader="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Leader]" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: entering leader state: leader="Node at 46668d28-613d-e0d8-c86a-3d1b587cbbd9 [Leader]" testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: added peer, starting replication: peer=server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: added peer, starting replication: peer=server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: added peer, starting replication: peer=server-c6b2cc84-d148-a152-acae-87fec832abdc testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.147Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: added peer, starting replication: peer=server-c6b2cc84-d148-a152-acae-87fec832abdc testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.148Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: pipelining replication: peer="{Voter server-c6b2cc84-d148-a152-acae-87fec832abdc c6b2cc84-d148-a152-acae-87fec832abdc}" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.148Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: pipelining replication: peer="{Voter server-c6b2cc84-d148-a152-acae-87fec832abdc c6b2cc84-d148-a152-acae-87fec832abdc}" testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.148Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: pipelining replication: peer="{Voter server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d}" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.148Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: pipelining replication: peer="{Voter server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.253Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.147788309 +0000 UTC m=+13.962927866 (1 nodes), 65.485ms from start of poll, 66.142434ms from cluster start. Timeout at 2022-06-22 14:53:21.253710407 +0000 UTC m=+14.068849963, 105.922097ms after stability testing.go:149: cluster: 2022-06-22T14:53:21.253Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.147788309 +0000 UTC m=+13.962927866 (1 nodes), 65.485ms from start of poll, 66.142434ms from cluster start. Timeout at 2022-06-22 14:53:21.253710407 +0000 UTC m=+14.068849963, 105.922097ms after stability testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.269Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: aborting pipeline replication: peer="{Voter server-c6b2cc84-d148-a152-acae-87fec832abdc c6b2cc84-d148-a152-acae-87fec832abdc}" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.269Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: aborting pipeline replication: peer="{Voter server-c6b2cc84-d148-a152-acae-87fec832abdc c6b2cc84-d148-a152-acae-87fec832abdc}" testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.269Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: aborting pipeline replication: peer="{Voter server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d}" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.269Z [INFO] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: aborting pipeline replication: peer="{Voter server-5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d 5724e82e-75fa-f3e6-6a9e-1d0c2c6ab23d}" testLoggerAdapter verbose: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.319Z [ERROR] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: failed to heartbeat to: peer=c6b2cc84-d148-a152-acae-87fec832abdc error="command timed out" testing.go:149: server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: 2022-06-22T14:53:21.319Z [ERROR] server-46668d28-613d-e0d8-c86a-3d1b587cbbd9: failed to heartbeat to: peer=c6b2cc84-d148-a152-acae-87fec832abdc error="command timed out" --- PASS: TestRaft_Barrier (0.24s) === RUN TestRaft_VerifyLeader testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.320Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.320Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testLoggerAdapter verbose: server-0a992990-3048-d092-d3ea-841b8093122f: 2022-06-22T14:53:21.321Z [INFO] server-0a992990-3048-d092-d3ea-841b8093122f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testing.go:149: server-0a992990-3048-d092-d3ea-841b8093122f: 2022-06-22T14:53:21.321Z [INFO] server-0a992990-3048-d092-d3ea-841b8093122f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testLoggerAdapter verbose: server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: 2022-06-22T14:53:21.321Z [INFO] server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testLoggerAdapter verbose: server-0a992990-3048-d092-d3ea-841b8093122f: 2022-06-22T14:53:21.321Z [INFO] server-0a992990-3048-d092-d3ea-841b8093122f: entering follower state: follower="Node at 0a992990-3048-d092-d3ea-841b8093122f [Follower]" leader= testing.go:149: server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: 2022-06-22T14:53:21.321Z [INFO] server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b2275830-c7ad-916e-110f-f0fc4fe4f927 Address:b2275830-c7ad-916e-110f-f0fc4fe4f927} {Suffrage:Voter ID:server-0a992990-3048-d092-d3ea-841b8093122f Address:0a992990-3048-d092-d3ea-841b8093122f} {Suffrage:Voter ID:server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d Address:90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.321Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-0a992990-3048-d092-d3ea-841b8093122f: 2022-06-22T14:53:21.321Z [INFO] server-0a992990-3048-d092-d3ea-841b8093122f: entering follower state: follower="Node at 0a992990-3048-d092-d3ea-841b8093122f [Follower]" leader= testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.321Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering follower state: follower="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:21.321Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: 2022-06-22T14:53:21.321Z [INFO] server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: entering follower state: follower="Node at 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d [Follower]" leader= testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.321Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering follower state: follower="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Follower]" leader= testing.go:149: server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: 2022-06-22T14:53:21.321Z [INFO] server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d: entering follower state: follower="Node at 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d [Follower]" leader= testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.406Z [WARN] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.406Z [WARN] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering candidate state: node="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Candidate]" term=2 testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering candidate state: node="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Candidate]" term=2 testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: election won: tally=2 testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: election won: tally=2 testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering leader state: leader="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Leader]" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: entering leader state: leader="Node at b2275830-c7ad-916e-110f-f0fc4fe4f927 [Leader]" testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: added peer, starting replication: peer=server-0a992990-3048-d092-d3ea-841b8093122f testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: added peer, starting replication: peer=server-0a992990-3048-d092-d3ea-841b8093122f testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: added peer, starting replication: peer=server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: added peer, starting replication: peer=server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: pipelining replication: peer="{Voter server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: pipelining replication: peer="{Voter server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}" testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: pipelining replication: peer="{Voter server-0a992990-3048-d092-d3ea-841b8093122f 0a992990-3048-d092-d3ea-841b8093122f}" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.407Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: pipelining replication: peer="{Voter server-0a992990-3048-d092-d3ea-841b8093122f 0a992990-3048-d092-d3ea-841b8093122f}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.512Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.407573413 +0000 UTC m=+14.222712969 (1 nodes), 86.192419ms from start of poll, 86.777165ms from cluster start. Timeout at 2022-06-22 14:53:21.51271221 +0000 UTC m=+14.327851766, 105.138797ms after stability testing.go:149: cluster: 2022-06-22T14:53:21.512Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.407573413 +0000 UTC m=+14.222712969 (1 nodes), 86.192419ms from start of poll, 86.777165ms from cluster start. Timeout at 2022-06-22 14:53:21.51271221 +0000 UTC m=+14.327851766, 105.138797ms after stability testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.512Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: aborting pipeline replication: peer="{Voter server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.512Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: aborting pipeline replication: peer="{Voter server-90ec5ccc-0521-a4df-8beb-94e1c3da1e8d 90ec5ccc-0521-a4df-8beb-94e1c3da1e8d}" testLoggerAdapter verbose: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.512Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: aborting pipeline replication: peer="{Voter server-0a992990-3048-d092-d3ea-841b8093122f 0a992990-3048-d092-d3ea-841b8093122f}" testing.go:149: server-b2275830-c7ad-916e-110f-f0fc4fe4f927: 2022-06-22T14:53:21.512Z [INFO] server-b2275830-c7ad-916e-110f-f0fc4fe4f927: aborting pipeline replication: peer="{Voter server-0a992990-3048-d092-d3ea-841b8093122f 0a992990-3048-d092-d3ea-841b8093122f}" --- PASS: TestRaft_VerifyLeader (0.19s) === RUN TestRaft_VerifyLeader_Single testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.513Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-40964a43-9209-ea5e-276e-f3d8ad3885da Address:40964a43-9209-ea5e-276e-f3d8ad3885da}]" testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.513Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-40964a43-9209-ea5e-276e-f3d8ad3885da Address:40964a43-9209-ea5e-276e-f3d8ad3885da}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.513Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:21.513Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.513Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering follower state: follower="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Follower]" leader= testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.513Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering follower state: follower="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Follower]" leader= testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.573Z [WARN] server-40964a43-9209-ea5e-276e-f3d8ad3885da: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.573Z [WARN] server-40964a43-9209-ea5e-276e-f3d8ad3885da: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering candidate state: node="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Candidate]" term=2 testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering candidate state: node="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Candidate]" term=2 testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: election won: tally=1 testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: election won: tally=1 testLoggerAdapter verbose: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering leader state: leader="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Leader]" testing.go:149: server-40964a43-9209-ea5e-276e-f3d8ad3885da: 2022-06-22T14:53:21.574Z [INFO] server-40964a43-9209-ea5e-276e-f3d8ad3885da: entering leader state: leader="Node at 40964a43-9209-ea5e-276e-f3d8ad3885da [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.680Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.574303487 +0000 UTC m=+14.389443033 (1 nodes), 60.39808ms from start of poll, 60.593951ms from cluster start. Timeout at 2022-06-22 14:53:21.680209594 +0000 UTC m=+14.495349160, 105.906127ms after stability testing.go:149: cluster: 2022-06-22T14:53:21.680Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.574303487 +0000 UTC m=+14.389443033 (1 nodes), 60.39808ms from start of poll, 60.593951ms from cluster start. Timeout at 2022-06-22 14:53:21.680209594 +0000 UTC m=+14.495349160, 105.906127ms after stability --- PASS: TestRaft_VerifyLeader_Single (0.17s) === RUN TestRaft_VerifyLeader_Fail testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.681Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20bf2255-148f-cf91-35e3-34655bfd13d3 Address:20bf2255-148f-cf91-35e3-34655bfd13d3} {Suffrage:Voter ID:server-a960f516-10d3-13dd-c10b-02251553ea6f Address:a960f516-10d3-13dd-c10b-02251553ea6f}]" testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.681Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20bf2255-148f-cf91-35e3-34655bfd13d3 Address:20bf2255-148f-cf91-35e3-34655bfd13d3} {Suffrage:Voter ID:server-a960f516-10d3-13dd-c10b-02251553ea6f Address:a960f516-10d3-13dd-c10b-02251553ea6f}]" testLoggerAdapter verbose: server-a960f516-10d3-13dd-c10b-02251553ea6f: 2022-06-22T14:53:21.681Z [INFO] server-a960f516-10d3-13dd-c10b-02251553ea6f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20bf2255-148f-cf91-35e3-34655bfd13d3 Address:20bf2255-148f-cf91-35e3-34655bfd13d3} {Suffrage:Voter ID:server-a960f516-10d3-13dd-c10b-02251553ea6f Address:a960f516-10d3-13dd-c10b-02251553ea6f}]" testing.go:149: server-a960f516-10d3-13dd-c10b-02251553ea6f: 2022-06-22T14:53:21.681Z [INFO] server-a960f516-10d3-13dd-c10b-02251553ea6f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-20bf2255-148f-cf91-35e3-34655bfd13d3 Address:20bf2255-148f-cf91-35e3-34655bfd13d3} {Suffrage:Voter ID:server-a960f516-10d3-13dd-c10b-02251553ea6f Address:a960f516-10d3-13dd-c10b-02251553ea6f}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.681Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:21.681Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a960f516-10d3-13dd-c10b-02251553ea6f: 2022-06-22T14:53:21.681Z [INFO] server-a960f516-10d3-13dd-c10b-02251553ea6f: entering follower state: follower="Node at a960f516-10d3-13dd-c10b-02251553ea6f [Follower]" leader= testing.go:149: server-a960f516-10d3-13dd-c10b-02251553ea6f: 2022-06-22T14:53:21.681Z [INFO] server-a960f516-10d3-13dd-c10b-02251553ea6f: entering follower state: follower="Node at a960f516-10d3-13dd-c10b-02251553ea6f [Follower]" leader= testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.681Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering follower state: follower="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Follower]" leader= testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.681Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering follower state: follower="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Follower]" leader= testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.754Z [WARN] server-20bf2255-148f-cf91-35e3-34655bfd13d3: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.754Z [WARN] server-20bf2255-148f-cf91-35e3-34655bfd13d3: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.754Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering candidate state: node="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Candidate]" term=2 testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.754Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering candidate state: node="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Candidate]" term=2 testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: election won: tally=2 testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: election won: tally=2 testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering leader state: leader="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Leader]" testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering leader state: leader="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Leader]" testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: added peer, starting replication: peer=server-a960f516-10d3-13dd-c10b-02251553ea6f testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: added peer, starting replication: peer=server-a960f516-10d3-13dd-c10b-02251553ea6f testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: pipelining replication: peer="{Voter server-a960f516-10d3-13dd-c10b-02251553ea6f a960f516-10d3-13dd-c10b-02251553ea6f}" testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.755Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: pipelining replication: peer="{Voter server-a960f516-10d3-13dd-c10b-02251553ea6f a960f516-10d3-13dd-c10b-02251553ea6f}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.860Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.7551387 +0000 UTC m=+14.570278496 (1 nodes), 73.676105ms from start of poll, 74.025516ms from cluster start. Timeout at 2022-06-22 14:53:21.860608832 +0000 UTC m=+14.675748388, 105.469892ms after stability testing.go:149: cluster: 2022-06-22T14:53:21.860Z [INFO] stable state for Leader reached at 2022-06-22 14:53:21.7551387 +0000 UTC m=+14.570278496 (1 nodes), 73.676105ms from start of poll, 74.025516ms from cluster start. Timeout at 2022-06-22 14:53:21.860608832 +0000 UTC m=+14.675748388, 105.469892ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.860Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:21.860Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.966Z [INFO] stable state for Follower reached at 2022-06-22 14:53:21.860765067 +0000 UTC m=+14.675904623 (1 nodes), 861ns from start of poll, 179.651643ms from cluster start. Timeout at 2022-06-22 14:53:21.966186988 +0000 UTC m=+14.781326534, 105.421911ms after stability testing.go:149: cluster: 2022-06-22T14:53:21.966Z [INFO] stable state for Follower reached at 2022-06-22 14:53:21.860765067 +0000 UTC m=+14.675904623 (1 nodes), 861ns from start of poll, 179.651643ms from cluster start. Timeout at 2022-06-22 14:53:21.966186988 +0000 UTC m=+14.781326534, 105.421911ms after stability testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [WARN] server-20bf2255-148f-cf91-35e3-34655bfd13d3: new leader elected, stepping down testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [WARN] server-20bf2255-148f-cf91-35e3-34655bfd13d3: new leader elected, stepping down testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering follower state: follower="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Follower]" leader= testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: entering follower state: follower="Node at 20bf2255-148f-cf91-35e3-34655bfd13d3 [Follower]" leader= testLoggerAdapter verbose: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: aborting pipeline replication: peer="{Voter server-a960f516-10d3-13dd-c10b-02251553ea6f a960f516-10d3-13dd-c10b-02251553ea6f}" testing.go:149: server-20bf2255-148f-cf91-35e3-34655bfd13d3: 2022-06-22T14:53:21.966Z [INFO] server-20bf2255-148f-cf91-35e3-34655bfd13d3: aborting pipeline replication: peer="{Voter server-a960f516-10d3-13dd-c10b-02251553ea6f a960f516-10d3-13dd-c10b-02251553ea6f}" --- PASS: TestRaft_VerifyLeader_Fail (0.29s) === RUN TestRaft_VerifyLeader_PartialConnect testLoggerAdapter verbose: server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: 2022-06-22T14:53:21.967Z [INFO] server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testing.go:149: server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: 2022-06-22T14:53:21.967Z [INFO] server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testLoggerAdapter verbose: server-74b5cf55-2ca7-1410-1d34-822bac923855: 2022-06-22T14:53:21.967Z [INFO] server-74b5cf55-2ca7-1410-1d34-822bac923855: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testing.go:149: server-74b5cf55-2ca7-1410-1d34-822bac923855: 2022-06-22T14:53:21.967Z [INFO] server-74b5cf55-2ca7-1410-1d34-822bac923855: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testLoggerAdapter verbose: server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: 2022-06-22T14:53:21.967Z [INFO] server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: entering follower state: follower="Node at 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 [Follower]" leader= testing.go:149: server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: 2022-06-22T14:53:21.967Z [INFO] server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652: entering follower state: follower="Node at 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 [Follower]" leader= testLoggerAdapter verbose: server-74b5cf55-2ca7-1410-1d34-822bac923855: 2022-06-22T14:53:21.968Z [INFO] server-74b5cf55-2ca7-1410-1d34-822bac923855: entering follower state: follower="Node at 74b5cf55-2ca7-1410-1d34-822bac923855 [Follower]" leader= testing.go:149: server-74b5cf55-2ca7-1410-1d34-822bac923855: 2022-06-22T14:53:21.968Z [INFO] server-74b5cf55-2ca7-1410-1d34-822bac923855: entering follower state: follower="Node at 74b5cf55-2ca7-1410-1d34-822bac923855 [Follower]" leader= testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:21.968Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:21.968Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 Address:7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652} {Suffrage:Voter ID:server-74b5cf55-2ca7-1410-1d34-822bac923855 Address:74b5cf55-2ca7-1410-1d34-822bac923855} {Suffrage:Voter ID:server-10335464-4c10-8074-aafc-4037f8235576 Address:10335464-4c10-8074-aafc-4037f8235576}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:21.968Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:21.968Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:21.968Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering follower state: follower="Node at 10335464-4c10-8074-aafc-4037f8235576 [Follower]" leader= testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:21.968Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering follower state: follower="Node at 10335464-4c10-8074-aafc-4037f8235576 [Follower]" leader= testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [WARN] server-10335464-4c10-8074-aafc-4037f8235576: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [WARN] server-10335464-4c10-8074-aafc-4037f8235576: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering candidate state: node="Node at 10335464-4c10-8074-aafc-4037f8235576 [Candidate]" term=2 testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering candidate state: node="Node at 10335464-4c10-8074-aafc-4037f8235576 [Candidate]" term=2 testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: election won: tally=2 testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: election won: tally=2 testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering leader state: leader="Node at 10335464-4c10-8074-aafc-4037f8235576 [Leader]" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: entering leader state: leader="Node at 10335464-4c10-8074-aafc-4037f8235576 [Leader]" testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: added peer, starting replication: peer=server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: added peer, starting replication: peer=server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: added peer, starting replication: peer=server-74b5cf55-2ca7-1410-1d34-822bac923855 testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.033Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: added peer, starting replication: peer=server-74b5cf55-2ca7-1410-1d34-822bac923855 testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.034Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: pipelining replication: peer="{Voter server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652}" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.034Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: pipelining replication: peer="{Voter server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652}" testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.034Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: pipelining replication: peer="{Voter server-74b5cf55-2ca7-1410-1d34-822bac923855 74b5cf55-2ca7-1410-1d34-822bac923855}" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.034Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: pipelining replication: peer="{Voter server-74b5cf55-2ca7-1410-1d34-822bac923855 74b5cf55-2ca7-1410-1d34-822bac923855}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.139Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.033822776 +0000 UTC m=+14.848962322 (1 nodes), 65.620065ms from start of poll, 66.264824ms from cluster start. Timeout at 2022-06-22 14:53:22.13958654 +0000 UTC m=+14.954726096, 105.763774ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.139Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.033822776 +0000 UTC m=+14.848962322 (1 nodes), 65.620065ms from start of poll, 66.264824ms from cluster start. Timeout at 2022-06-22 14:53:22.13958654 +0000 UTC m=+14.954726096, 105.763774ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.145Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:22.145Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.251Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.145325014 +0000 UTC m=+14.960464570 (2 nodes), 1.392µs from start of poll, 177.767072ms from cluster start. Timeout at 2022-06-22 14:53:22.251330455 +0000 UTC m=+15.066470011, 106.005441ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.251Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.145325014 +0000 UTC m=+14.960464570 (2 nodes), 1.392µs from start of poll, 177.767072ms from cluster start. Timeout at 2022-06-22 14:53:22.251330455 +0000 UTC m=+15.066470011, 106.005441ms after stability raft_test.go:1647: [INFO] Disconnecting Node at 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 [Follower] testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [ERROR] server-10335464-4c10-8074-aafc-4037f8235576: failed to heartbeat to: peer=7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 error="failed to connect to peer: 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [ERROR] server-10335464-4c10-8074-aafc-4037f8235576: failed to heartbeat to: peer=7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 error="failed to connect to peer: 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652" testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: aborting pipeline replication: peer="{Voter server-74b5cf55-2ca7-1410-1d34-822bac923855 74b5cf55-2ca7-1410-1d34-822bac923855}" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: aborting pipeline replication: peer="{Voter server-74b5cf55-2ca7-1410-1d34-822bac923855 74b5cf55-2ca7-1410-1d34-822bac923855}" testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: aborting pipeline replication: peer="{Voter server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652}" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.251Z [INFO] server-10335464-4c10-8074-aafc-4037f8235576: aborting pipeline replication: peer="{Voter server-7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652}" testLoggerAdapter verbose: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.252Z [ERROR] server-10335464-4c10-8074-aafc-4037f8235576: failed to heartbeat to: peer=7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 error="failed to connect to peer: 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652" testing.go:149: server-10335464-4c10-8074-aafc-4037f8235576: 2022-06-22T14:53:22.252Z [ERROR] server-10335464-4c10-8074-aafc-4037f8235576: failed to heartbeat to: peer=7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652 error="failed to connect to peer: 7ef4ae1c-2db6-aca9-f3bf-e33f7cd97652" --- PASS: TestRaft_VerifyLeader_PartialConnect (0.29s) === RUN TestRaft_StartAsLeader testLoggerAdapter verbose: server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: 2022-06-22T14:53:22.253Z [INFO] server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97 Address:1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97}]" testing.go:149: server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: 2022-06-22T14:53:22.253Z [INFO] server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97 Address:1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97}]" testLoggerAdapter verbose: server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: 2022-06-22T14:53:22.253Z [INFO] server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: entering leader state: leader="Node at 1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97 [Leader]" testing.go:149: server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: 2022-06-22T14:53:22.253Z [INFO] server-1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97: entering leader state: leader="Node at 1a53ccf7-3582-f2ea-af3d-ffc06a4ffc97 [Leader]" --- PASS: TestRaft_StartAsLeader (0.00s) === RUN TestRaft_NotifyCh testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.253Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5 Address:3bee2cbf-6716-afbf-c76f-dcf92149f7c5}]" testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.253Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5 Address:3bee2cbf-6716-afbf-c76f-dcf92149f7c5}]" testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.253Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering follower state: follower="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Follower]" leader= testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.253Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering follower state: follower="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Follower]" leader= testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [WARN] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [WARN] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering candidate state: node="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Candidate]" term=2 testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering candidate state: node="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Candidate]" term=2 testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: election won: tally=1 testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: election won: tally=1 testLoggerAdapter verbose: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering leader state: leader="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Leader]" testing.go:149: server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: 2022-06-22T14:53:22.327Z [INFO] server-3bee2cbf-6716-afbf-c76f-dcf92149f7c5: entering leader state: leader="Node at 3bee2cbf-6716-afbf-c76f-dcf92149f7c5 [Leader]" --- PASS: TestRaft_NotifyCh (0.07s) === RUN TestRaft_Voting testLoggerAdapter verbose: server-266dedfe-62d0-8e97-ab73-4168113f4259: 2022-06-22T14:53:22.328Z [INFO] server-266dedfe-62d0-8e97-ab73-4168113f4259: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testing.go:149: server-266dedfe-62d0-8e97-ab73-4168113f4259: 2022-06-22T14:53:22.328Z [INFO] server-266dedfe-62d0-8e97-ab73-4168113f4259: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.328Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.328Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testLoggerAdapter verbose: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.328Z [INFO] server-a1310bee-33c8-d99e-7474-f5258ab64600: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testing.go:149: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.328Z [INFO] server-a1310bee-33c8-d99e-7474-f5258ab64600: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-266dedfe-62d0-8e97-ab73-4168113f4259 Address:266dedfe-62d0-8e97-ab73-4168113f4259} {Suffrage:Voter ID:server-84efd11c-3f39-2649-b568-c3bd9c470419 Address:84efd11c-3f39-2649-b568-c3bd9c470419} {Suffrage:Voter ID:server-a1310bee-33c8-d99e-7474-f5258ab64600 Address:a1310bee-33c8-d99e-7474-f5258ab64600}]" testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.328Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering follower state: follower="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Follower]" leader= testLoggerAdapter verbose: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.328Z [INFO] server-a1310bee-33c8-d99e-7474-f5258ab64600: entering follower state: follower="Node at a1310bee-33c8-d99e-7474-f5258ab64600 [Follower]" leader= testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.328Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering follower state: follower="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.328Z [INFO] starting stability test: raft-state=Follower testing.go:149: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.328Z [INFO] server-a1310bee-33c8-d99e-7474-f5258ab64600: entering follower state: follower="Node at a1310bee-33c8-d99e-7474-f5258ab64600 [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:22.328Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-266dedfe-62d0-8e97-ab73-4168113f4259: 2022-06-22T14:53:22.328Z [INFO] server-266dedfe-62d0-8e97-ab73-4168113f4259: entering follower state: follower="Node at 266dedfe-62d0-8e97-ab73-4168113f4259 [Follower]" leader= testing.go:149: server-266dedfe-62d0-8e97-ab73-4168113f4259: 2022-06-22T14:53:22.328Z [INFO] server-266dedfe-62d0-8e97-ab73-4168113f4259: entering follower state: follower="Node at 266dedfe-62d0-8e97-ab73-4168113f4259 [Follower]" leader= testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.385Z [WARN] server-84efd11c-3f39-2649-b568-c3bd9c470419: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.385Z [WARN] server-84efd11c-3f39-2649-b568-c3bd9c470419: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.385Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering candidate state: node="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Candidate]" term=2 testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.385Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering candidate state: node="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Candidate]" term=2 testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: election won: tally=2 testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: election won: tally=2 testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering leader state: leader="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Leader]" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: entering leader state: leader="Node at 84efd11c-3f39-2649-b568-c3bd9c470419 [Leader]" testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: added peer, starting replication: peer=server-266dedfe-62d0-8e97-ab73-4168113f4259 testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: added peer, starting replication: peer=server-266dedfe-62d0-8e97-ab73-4168113f4259 testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: added peer, starting replication: peer=server-a1310bee-33c8-d99e-7474-f5258ab64600 testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: added peer, starting replication: peer=server-a1310bee-33c8-d99e-7474-f5258ab64600 testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: pipelining replication: peer="{Voter server-266dedfe-62d0-8e97-ab73-4168113f4259 266dedfe-62d0-8e97-ab73-4168113f4259}" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: pipelining replication: peer="{Voter server-266dedfe-62d0-8e97-ab73-4168113f4259 266dedfe-62d0-8e97-ab73-4168113f4259}" testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: pipelining replication: peer="{Voter server-a1310bee-33c8-d99e-7474-f5258ab64600 a1310bee-33c8-d99e-7474-f5258ab64600}" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.386Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: pipelining replication: peer="{Voter server-a1310bee-33c8-d99e-7474-f5258ab64600 a1310bee-33c8-d99e-7474-f5258ab64600}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.491Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.386153923 +0000 UTC m=+15.201293469 (2 nodes), 57.249794ms from start of poll, 57.733609ms from cluster start. Timeout at 2022-06-22 14:53:22.491238332 +0000 UTC m=+15.306377878, 105.084409ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.491Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.386153923 +0000 UTC m=+15.201293469 (2 nodes), 57.249794ms from start of poll, 57.733609ms from cluster start. Timeout at 2022-06-22 14:53:22.491238332 +0000 UTC m=+15.306377878, 105.084409ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.491Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:22.491Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.596Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.49133268 +0000 UTC m=+15.306472226 (1 nodes), 832ns from start of poll, 162.912366ms from cluster start. Timeout at 2022-06-22 14:53:22.596825421 +0000 UTC m=+15.411964968, 105.492742ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.596Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.49133268 +0000 UTC m=+15.306472226 (1 nodes), 832ns from start of poll, 162.912366ms from cluster start. Timeout at 2022-06-22 14:53:22.596825421 +0000 UTC m=+15.411964968, 105.492742ms after stability testLoggerAdapter verbose: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.596Z [WARN] server-a1310bee-33c8-d99e-7474-f5258ab64600: rejecting vote request since we have a leader: from=266dedfe-62d0-8e97-ab73-4168113f4259 leader=84efd11c-3f39-2649-b568-c3bd9c470419 testing.go:149: server-a1310bee-33c8-d99e-7474-f5258ab64600: 2022-06-22T14:53:22.596Z [WARN] server-a1310bee-33c8-d99e-7474-f5258ab64600: rejecting vote request since we have a leader: from=266dedfe-62d0-8e97-ab73-4168113f4259 leader=84efd11c-3f39-2649-b568-c3bd9c470419 testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.597Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: aborting pipeline replication: peer="{Voter server-a1310bee-33c8-d99e-7474-f5258ab64600 a1310bee-33c8-d99e-7474-f5258ab64600}" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.597Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: aborting pipeline replication: peer="{Voter server-a1310bee-33c8-d99e-7474-f5258ab64600 a1310bee-33c8-d99e-7474-f5258ab64600}" testLoggerAdapter verbose: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.597Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: aborting pipeline replication: peer="{Voter server-266dedfe-62d0-8e97-ab73-4168113f4259 266dedfe-62d0-8e97-ab73-4168113f4259}" testing.go:149: server-84efd11c-3f39-2649-b568-c3bd9c470419: 2022-06-22T14:53:22.597Z [INFO] server-84efd11c-3f39-2649-b568-c3bd9c470419: aborting pipeline replication: peer="{Voter server-266dedfe-62d0-8e97-ab73-4168113f4259 266dedfe-62d0-8e97-ab73-4168113f4259}" --- PASS: TestRaft_Voting (0.27s) === RUN TestRaft_ProtocolVersion_RejectRPC testLoggerAdapter verbose: server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: 2022-06-22T14:53:22.598Z [INFO] server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testing.go:149: server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: 2022-06-22T14:53:22.598Z [INFO] server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testLoggerAdapter verbose: server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: 2022-06-22T14:53:22.599Z [INFO] server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: entering follower state: follower="Node at 5e834eb5-dfd1-04b9-db8a-3f00371538bf [Follower]" leader= testing.go:149: server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: 2022-06-22T14:53:22.599Z [INFO] server-5e834eb5-dfd1-04b9-db8a-3f00371538bf: entering follower state: follower="Node at 5e834eb5-dfd1-04b9-db8a-3f00371538bf [Follower]" leader= testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.599Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.599Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.599Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering follower state: follower="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Follower]" leader= testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.599Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering follower state: follower="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Follower]" leader= testLoggerAdapter verbose: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.599Z [INFO] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testing.go:149: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.599Z [INFO] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-5e834eb5-dfd1-04b9-db8a-3f00371538bf Address:5e834eb5-dfd1-04b9-db8a-3f00371538bf} {Suffrage:Voter ID:server-0aaad77e-c195-13a0-ec18-903a9e8f2d66 Address:0aaad77e-c195-13a0-ec18-903a9e8f2d66} {Suffrage:Voter ID:server-9f33b900-db4f-3464-7b26-aae001bcb9e5 Address:9f33b900-db4f-3464-7b26-aae001bcb9e5}]" testLoggerAdapter verbose: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.599Z [INFO] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: entering follower state: follower="Node at 9f33b900-db4f-3464-7b26-aae001bcb9e5 [Follower]" leader= testing.go:149: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.599Z [INFO] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: entering follower state: follower="Node at 9f33b900-db4f-3464-7b26-aae001bcb9e5 [Follower]" leader= testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.599Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:22.599Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [WARN] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [WARN] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering candidate state: node="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Candidate]" term=2 testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering candidate state: node="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Candidate]" term=2 testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: election won: tally=2 testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: election won: tally=2 testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering leader state: leader="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Leader]" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: entering leader state: leader="Node at 0aaad77e-c195-13a0-ec18-903a9e8f2d66 [Leader]" testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: added peer, starting replication: peer=server-5e834eb5-dfd1-04b9-db8a-3f00371538bf testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: added peer, starting replication: peer=server-5e834eb5-dfd1-04b9-db8a-3f00371538bf testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: added peer, starting replication: peer=server-9f33b900-db4f-3464-7b26-aae001bcb9e5 testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: added peer, starting replication: peer=server-9f33b900-db4f-3464-7b26-aae001bcb9e5 testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: pipelining replication: peer="{Voter server-9f33b900-db4f-3464-7b26-aae001bcb9e5 9f33b900-db4f-3464-7b26-aae001bcb9e5}" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: pipelining replication: peer="{Voter server-9f33b900-db4f-3464-7b26-aae001bcb9e5 9f33b900-db4f-3464-7b26-aae001bcb9e5}" testLoggerAdapter verbose: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.658Z [WARN] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: rejecting vote request since our last term is greater: candidate=0aaad77e-c195-13a0-ec18-903a9e8f2d66 last-term=2 last-candidate-term=1 testing.go:149: server-9f33b900-db4f-3464-7b26-aae001bcb9e5: 2022-06-22T14:53:22.658Z [WARN] server-9f33b900-db4f-3464-7b26-aae001bcb9e5: rejecting vote request since our last term is greater: candidate=0aaad77e-c195-13a0-ec18-903a9e8f2d66 last-term=2 last-candidate-term=1 testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: pipelining replication: peer="{Voter server-5e834eb5-dfd1-04b9-db8a-3f00371538bf 5e834eb5-dfd1-04b9-db8a-3f00371538bf}" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.658Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: pipelining replication: peer="{Voter server-5e834eb5-dfd1-04b9-db8a-3f00371538bf 5e834eb5-dfd1-04b9-db8a-3f00371538bf}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.764Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.658685505 +0000 UTC m=+15.473825061 (2 nodes), 58.951823ms from start of poll, 59.931506ms from cluster start. Timeout at 2022-06-22 14:53:22.76417027 +0000 UTC m=+15.579309816, 105.484755ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.764Z [INFO] stable state for Follower reached at 2022-06-22 14:53:22.658685505 +0000 UTC m=+15.473825061 (2 nodes), 58.951823ms from start of poll, 59.931506ms from cluster start. Timeout at 2022-06-22 14:53:22.76417027 +0000 UTC m=+15.579309816, 105.484755ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.764Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:22.764Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.869Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.764454918 +0000 UTC m=+15.579594465 (1 nodes), 1.654µs from start of poll, 165.70091ms from cluster start. Timeout at 2022-06-22 14:53:22.86957748 +0000 UTC m=+15.684717026, 105.122561ms after stability testing.go:149: cluster: 2022-06-22T14:53:22.869Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.764454918 +0000 UTC m=+15.579594465 (1 nodes), 1.654µs from start of poll, 165.70091ms from cluster start. Timeout at 2022-06-22 14:53:22.86957748 +0000 UTC m=+15.684717026, 105.122561ms after stability testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.869Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: aborting pipeline replication: peer="{Voter server-9f33b900-db4f-3464-7b26-aae001bcb9e5 9f33b900-db4f-3464-7b26-aae001bcb9e5}" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.869Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: aborting pipeline replication: peer="{Voter server-9f33b900-db4f-3464-7b26-aae001bcb9e5 9f33b900-db4f-3464-7b26-aae001bcb9e5}" testLoggerAdapter verbose: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.869Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: aborting pipeline replication: peer="{Voter server-5e834eb5-dfd1-04b9-db8a-3f00371538bf 5e834eb5-dfd1-04b9-db8a-3f00371538bf}" testing.go:149: server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: 2022-06-22T14:53:22.869Z [INFO] server-0aaad77e-c195-13a0-ec18-903a9e8f2d66: aborting pipeline replication: peer="{Voter server-5e834eb5-dfd1-04b9-db8a-3f00371538bf 5e834eb5-dfd1-04b9-db8a-3f00371538bf}" --- PASS: TestRaft_ProtocolVersion_RejectRPC (0.27s) === RUN TestRaft_ProtocolVersion_Upgrade_1_2 testLoggerAdapter verbose: 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: 2022-06-22T14:53:22.871Z [INFO] 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: initial configuration: index=1 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testing.go:149: 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: 2022-06-22T14:53:22.871Z [INFO] 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: initial configuration: index=1 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.871Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: initial configuration: index=1 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.871Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: initial configuration: index=1 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testLoggerAdapter verbose: 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: 2022-06-22T14:53:22.871Z [INFO] 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: entering follower state: follower="Node at 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 [Follower]" leader= testing.go:149: 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: 2022-06-22T14:53:22.871Z [INFO] 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58: entering follower state: follower="Node at 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 [Follower]" leader= testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.871Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering follower state: follower="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Follower]" leader= testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.871Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering follower state: follower="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Follower]" leader= testLoggerAdapter verbose: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.871Z [INFO] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: initial configuration: index=0 servers=[] testing.go:149: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.871Z [INFO] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2022-06-22T14:53:22.871Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:22.871Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.871Z [INFO] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: entering follower state: follower="Node at 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 [Follower]" leader= testing.go:149: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.871Z [INFO] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: entering follower state: follower="Node at 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 [Follower]" leader= testLoggerAdapter verbose: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.927Z [WARN] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: no known peers, aborting election testing.go:149: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:22.927Z [WARN] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: no known peers, aborting election testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [WARN] 80a0da1e-2216-e653-1e35-a0fb416cefe6: heartbeat timeout reached, starting election: last-leader= testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [WARN] 80a0da1e-2216-e653-1e35-a0fb416cefe6: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering candidate state: node="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Candidate]" term=2 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering candidate state: node="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Candidate]" term=2 testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: election won: tally=2 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: election won: tally=2 testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering leader state: leader="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Leader]" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: entering leader state: leader="Node at 80a0da1e-2216-e653-1e35-a0fb416cefe6 [Leader]" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: added peer, starting replication: peer=9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: added peer, starting replication: peer=9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: pipelining replication: peer="{Voter 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58}" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:22.949Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: pipelining replication: peer="{Voter 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.055Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.949337559 +0000 UTC m=+15.764477105 (1 nodes), 77.693029ms from start of poll, 78.424162ms from cluster start. Timeout at 2022-06-22 14:53:23.055166374 +0000 UTC m=+15.870305920, 105.828815ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.055Z [INFO] stable state for Leader reached at 2022-06-22 14:53:22.949337559 +0000 UTC m=+15.764477105 (1 nodes), 77.693029ms from start of poll, 78.424162ms from cluster start. Timeout at 2022-06-22 14:53:23.055166374 +0000 UTC m=+15.870305920, 105.828815ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.055Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.055Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.161Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.055289086 +0000 UTC m=+15.870428632 (1 nodes), 1.362µs from start of poll, 184.375689ms from cluster start. Timeout at 2022-06-22 14:53:23.161364565 +0000 UTC m=+15.976504112, 106.07548ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.161Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.055289086 +0000 UTC m=+15.870428632 (1 nodes), 1.362µs from start of poll, 184.375689ms from cluster start. Timeout at 2022-06-22 14:53:23.161364565 +0000 UTC m=+15.976504112, 106.07548ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.161Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.161Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.266Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.16149888 +0000 UTC m=+15.976638426 (1 nodes), 1.353µs from start of poll, 290.585483ms from cluster start. Timeout at 2022-06-22 14:53:23.266693363 +0000 UTC m=+16.081832919, 105.194493ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.266Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.16149888 +0000 UTC m=+15.976638426 (1 nodes), 1.353µs from start of poll, 290.585483ms from cluster start. Timeout at 2022-06-22 14:53:23.266693363 +0000 UTC m=+16.081832919, 105.194493ms after stability testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.266Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: updating configuration: command=AddStaging server-id=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 server-addr=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6} {Suffrage:Voter ID:8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 Address:8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}]" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.266Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: updating configuration: command=AddStaging server-id=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 server-addr=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6} {Suffrage:Voter ID:8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 Address:8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}]" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.266Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: added peer, starting replication: peer=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.266Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: added peer, starting replication: peer=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 testLoggerAdapter verbose: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:23.267Z [WARN] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: 2022-06-22T14:53:23.267Z [WARN] 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.267Z [WARN] 80a0da1e-2216-e653-1e35-a0fb416cefe6: appendEntries rejected, sending older logs: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" next=1 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.267Z [WARN] 80a0da1e-2216-e653-1e35-a0fb416cefe6: appendEntries rejected, sending older logs: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" next=1 testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.267Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: pipelining replication: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.267Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: pipelining replication: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.267Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.267Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.373Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.26728909 +0000 UTC m=+16.082428636 (1 nodes), 431ns from start of poll, 396.375693ms from cluster start. Timeout at 2022-06-22 14:53:23.373300508 +0000 UTC m=+16.188440064, 106.011428ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.373Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.26728909 +0000 UTC m=+16.082428636 (1 nodes), 431ns from start of poll, 396.375693ms from cluster start. Timeout at 2022-06-22 14:53:23.373300508 +0000 UTC m=+16.188440064, 106.011428ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.373Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.373Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.479Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.373546012 +0000 UTC m=+16.188685558 (1 nodes), 1.302µs from start of poll, 502.632615ms from cluster start. Timeout at 2022-06-22 14:53:23.47915003 +0000 UTC m=+16.294289586, 105.604028ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.479Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.373546012 +0000 UTC m=+16.188685558 (1 nodes), 1.302µs from start of poll, 502.632615ms from cluster start. Timeout at 2022-06-22 14:53:23.47915003 +0000 UTC m=+16.294289586, 105.604028ms after stability testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: updating configuration: command=RemoveServer server-id=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 server-addr= servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: updating configuration: command=RemoveServer server-id=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 server-addr= servers="[{Suffrage:Voter ID:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 Address:9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58} {Suffrage:Voter ID:80a0da1e-2216-e653-1e35-a0fb416cefe6 Address:80a0da1e-2216-e653-1e35-a0fb416cefe6}]" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: removed peer, stopping replication: peer=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 last-index=4 testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: removed peer, stopping replication: peer=8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 last-index=4 testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: aborting pipeline replication: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: aborting pipeline replication: peer="{Voter 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2 8feec2a2-34ac-4d27-ac3b-a6e8ba5f58e2}" testLoggerAdapter verbose: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: aborting pipeline replication: peer="{Voter 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58}" testing.go:149: 80a0da1e-2216-e653-1e35-a0fb416cefe6: 2022-06-22T14:53:23.479Z [INFO] 80a0da1e-2216-e653-1e35-a0fb416cefe6: aborting pipeline replication: peer="{Voter 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58 9dea2d37-8efc-a1bc-fa4c-7e5b4cd3db58}" --- PASS: TestRaft_ProtocolVersion_Upgrade_1_2 (0.61s) === RUN TestRaft_ProtocolVersion_Upgrade_2_3 testLoggerAdapter verbose: ab3674c6-520b-e265-2164-1df8a8bc350d: 2022-06-22T14:53:23.481Z [INFO] ab3674c6-520b-e265-2164-1df8a8bc350d: initial configuration: index=1 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e}]" testing.go:149: ab3674c6-520b-e265-2164-1df8a8bc350d: 2022-06-22T14:53:23.481Z [INFO] ab3674c6-520b-e265-2164-1df8a8bc350d: initial configuration: index=1 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e}]" testLoggerAdapter verbose: ab3674c6-520b-e265-2164-1df8a8bc350d: 2022-06-22T14:53:23.481Z [INFO] ab3674c6-520b-e265-2164-1df8a8bc350d: entering follower state: follower="Node at ab3674c6-520b-e265-2164-1df8a8bc350d [Follower]" leader= testing.go:149: ab3674c6-520b-e265-2164-1df8a8bc350d: 2022-06-22T14:53:23.481Z [INFO] ab3674c6-520b-e265-2164-1df8a8bc350d: entering follower state: follower="Node at ab3674c6-520b-e265-2164-1df8a8bc350d [Follower]" leader= testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.481Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e}]" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.481Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.481Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:23.481Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.481Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering follower state: follower="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Follower]" leader= testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.481Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering follower state: follower="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Follower]" leader= testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [WARN] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: heartbeat timeout reached, starting election: last-leader= testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [WARN] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering candidate state: node="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Candidate]" term=2 testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering candidate state: node="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Candidate]" term=2 testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: election won: tally=2 testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: election won: tally=2 testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering leader state: leader="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Leader]" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: entering leader state: leader="Node at 6988e03a-0010-1501-4ca5-4df17a3dfe2e [Leader]" testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: added peer, starting replication: peer=ab3674c6-520b-e265-2164-1df8a8bc350d testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: added peer, starting replication: peer=ab3674c6-520b-e265-2164-1df8a8bc350d testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: pipelining replication: peer="{Voter ab3674c6-520b-e265-2164-1df8a8bc350d ab3674c6-520b-e265-2164-1df8a8bc350d}" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.541Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: pipelining replication: peer="{Voter ab3674c6-520b-e265-2164-1df8a8bc350d ab3674c6-520b-e265-2164-1df8a8bc350d}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.647Z [INFO] stable state for Follower reached at 2022-06-22 14:53:23.541570973 +0000 UTC m=+16.356710519 (1 nodes), 59.966451ms from start of poll, 60.471956ms from cluster start. Timeout at 2022-06-22 14:53:23.64725942 +0000 UTC m=+16.462398976, 105.688457ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.647Z [INFO] stable state for Follower reached at 2022-06-22 14:53:23.541570973 +0000 UTC m=+16.356710519 (1 nodes), 59.966451ms from start of poll, 60.471956ms from cluster start. Timeout at 2022-06-22 14:53:23.64725942 +0000 UTC m=+16.462398976, 105.688457ms after stability testLoggerAdapter verbose: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.647Z [INFO] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: initial configuration: index=0 servers=[] testing.go:149: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.647Z [INFO] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: initial configuration: index=0 servers=[] testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.647Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.647Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.647Z [INFO] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: entering follower state: follower="Node at e75c2666-3eb4-57a9-9d21-5dbd30f386b7 [Follower]" leader= testing.go:149: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.647Z [INFO] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: entering follower state: follower="Node at e75c2666-3eb4-57a9-9d21-5dbd30f386b7 [Follower]" leader= testLoggerAdapter verbose: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.701Z [WARN] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: no known peers, aborting election testing.go:149: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.701Z [WARN] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: no known peers, aborting election testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.753Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.647910271 +0000 UTC m=+16.463049817 (1 nodes), 280ns from start of poll, 166.811254ms from cluster start. Timeout at 2022-06-22 14:53:23.753095036 +0000 UTC m=+16.568234592, 105.184775ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.753Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.647910271 +0000 UTC m=+16.463049817 (1 nodes), 280ns from start of poll, 166.811254ms from cluster start. Timeout at 2022-06-22 14:53:23.753095036 +0000 UTC m=+16.568234592, 105.184775ms after stability testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: updating configuration: command=AddStaging server-id=server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 server-addr=e75c2666-3eb4-57a9-9d21-5dbd30f386b7 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e} {Suffrage:Voter ID:server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 Address:e75c2666-3eb4-57a9-9d21-5dbd30f386b7}]" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: updating configuration: command=AddStaging server-id=server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 server-addr=e75c2666-3eb4-57a9-9d21-5dbd30f386b7 servers="[{Suffrage:Voter ID:ab3674c6-520b-e265-2164-1df8a8bc350d Address:ab3674c6-520b-e265-2164-1df8a8bc350d} {Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e} {Suffrage:Voter ID:server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 Address:e75c2666-3eb4-57a9-9d21-5dbd30f386b7}]" testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: added peer, starting replication: peer=server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: added peer, starting replication: peer=server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 testLoggerAdapter verbose: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.753Z [WARN] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: failed to get previous log: previous-index=3 last-index=0 error="log not found" testing.go:149: server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: 2022-06-22T14:53:23.753Z [WARN] server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7: failed to get previous log: previous-index=3 last-index=0 error="log not found" testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [WARN] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: appendEntries rejected, sending older logs: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" next=1 testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [WARN] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: appendEntries rejected, sending older logs: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" next=1 testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: pipelining replication: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.753Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: pipelining replication: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.753Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.753Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.859Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.75392238 +0000 UTC m=+16.569061937 (1 nodes), 1.343µs from start of poll, 272.823374ms from cluster start. Timeout at 2022-06-22 14:53:23.859606038 +0000 UTC m=+16.674745595, 105.683658ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.859Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.75392238 +0000 UTC m=+16.569061937 (1 nodes), 1.343µs from start of poll, 272.823374ms from cluster start. Timeout at 2022-06-22 14:53:23.859606038 +0000 UTC m=+16.674745595, 105.683658ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.859Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.859Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.965Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.859842706 +0000 UTC m=+16.674982252 (1 nodes), 1.543µs from start of poll, 378.743689ms from cluster start. Timeout at 2022-06-22 14:53:23.965579754 +0000 UTC m=+16.780719310, 105.737058ms after stability testing.go:149: cluster: 2022-06-22T14:53:23.965Z [INFO] stable state for Leader reached at 2022-06-22 14:53:23.859842706 +0000 UTC m=+16.674982252 (1 nodes), 1.543µs from start of poll, 378.743689ms from cluster start. Timeout at 2022-06-22 14:53:23.965579754 +0000 UTC m=+16.780719310, 105.737058ms after stability testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.965Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: updating configuration: command=RemoveServer server-id=ab3674c6-520b-e265-2164-1df8a8bc350d server-addr= servers="[{Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e} {Suffrage:Voter ID:server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 Address:e75c2666-3eb4-57a9-9d21-5dbd30f386b7}]" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.965Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: updating configuration: command=RemoveServer server-id=ab3674c6-520b-e265-2164-1df8a8bc350d server-addr= servers="[{Suffrage:Voter ID:6988e03a-0010-1501-4ca5-4df17a3dfe2e Address:6988e03a-0010-1501-4ca5-4df17a3dfe2e} {Suffrage:Voter ID:server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 Address:e75c2666-3eb4-57a9-9d21-5dbd30f386b7}]" testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.965Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: removed peer, stopping replication: peer=ab3674c6-520b-e265-2164-1df8a8bc350d last-index=4 testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.965Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: removed peer, stopping replication: peer=ab3674c6-520b-e265-2164-1df8a8bc350d last-index=4 testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.966Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: aborting pipeline replication: peer="{Voter ab3674c6-520b-e265-2164-1df8a8bc350d ab3674c6-520b-e265-2164-1df8a8bc350d}" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.966Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: aborting pipeline replication: peer="{Voter ab3674c6-520b-e265-2164-1df8a8bc350d ab3674c6-520b-e265-2164-1df8a8bc350d}" testLoggerAdapter verbose: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.966Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: aborting pipeline replication: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" testing.go:149: 6988e03a-0010-1501-4ca5-4df17a3dfe2e: 2022-06-22T14:53:23.966Z [INFO] 6988e03a-0010-1501-4ca5-4df17a3dfe2e: aborting pipeline replication: peer="{Voter server-e75c2666-3eb4-57a9-9d21-5dbd30f386b7 e75c2666-3eb4-57a9-9d21-5dbd30f386b7}" --- PASS: TestRaft_ProtocolVersion_Upgrade_2_3 (0.49s) === RUN TestRaft_LeadershipTransferInProgress --- PASS: TestRaft_LeadershipTransferInProgress (0.00s) === RUN TestRaft_LeadershipTransferPickServer --- PASS: TestRaft_LeadershipTransferPickServer (0.00s) === RUN TestRaft_LeadershipTransfer testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:23.968Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:23.968Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:23.968Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering follower state: follower="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Follower]" leader= testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:23.968Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering follower state: follower="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Follower]" leader= testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:23.968Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:23.968Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:23.969Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering follower state: follower="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Follower]" leader= testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:23.969Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering follower state: follower="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Follower]" leader= testLoggerAdapter verbose: server-bb669090-33ea-ece3-06b5-7885fbfe6411: 2022-06-22T14:53:23.969Z [INFO] server-bb669090-33ea-ece3-06b5-7885fbfe6411: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testing.go:149: server-bb669090-33ea-ece3-06b5-7885fbfe6411: 2022-06-22T14:53:23.969Z [INFO] server-bb669090-33ea-ece3-06b5-7885fbfe6411: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0841d69e-2808-4435-1648-56ad4126d917 Address:0841d69e-2808-4435-1648-56ad4126d917} {Suffrage:Voter ID:server-f998619c-3403-1634-a11b-6f18efd3c485 Address:f998619c-3403-1634-a11b-6f18efd3c485} {Suffrage:Voter ID:server-bb669090-33ea-ece3-06b5-7885fbfe6411 Address:bb669090-33ea-ece3-06b5-7885fbfe6411}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:23.969Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:23.969Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-bb669090-33ea-ece3-06b5-7885fbfe6411: 2022-06-22T14:53:23.969Z [INFO] server-bb669090-33ea-ece3-06b5-7885fbfe6411: entering follower state: follower="Node at bb669090-33ea-ece3-06b5-7885fbfe6411 [Follower]" leader= testing.go:149: server-bb669090-33ea-ece3-06b5-7885fbfe6411: 2022-06-22T14:53:23.969Z [INFO] server-bb669090-33ea-ece3-06b5-7885fbfe6411: entering follower state: follower="Node at bb669090-33ea-ece3-06b5-7885fbfe6411 [Follower]" leader= testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [WARN] server-f998619c-3403-1634-a11b-6f18efd3c485: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [WARN] server-f998619c-3403-1634-a11b-6f18efd3c485: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering candidate state: node="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Candidate]" term=2 testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering candidate state: node="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Candidate]" term=2 testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: election won: tally=2 testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: election won: tally=2 testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering leader state: leader="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Leader]" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering leader state: leader="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Leader]" testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: added peer, starting replication: peer=server-0841d69e-2808-4435-1648-56ad4126d917 testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: added peer, starting replication: peer=server-0841d69e-2808-4435-1648-56ad4126d917 testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: added peer, starting replication: peer=server-bb669090-33ea-ece3-06b5-7885fbfe6411 testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.046Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: added peer, starting replication: peer=server-bb669090-33ea-ece3-06b5-7885fbfe6411 testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.047Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: pipelining replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.047Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: pipelining replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.047Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: pipelining replication: peer="{Voter server-0841d69e-2808-4435-1648-56ad4126d917 0841d69e-2808-4435-1648-56ad4126d917}" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.047Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: pipelining replication: peer="{Voter server-0841d69e-2808-4435-1648-56ad4126d917 0841d69e-2808-4435-1648-56ad4126d917}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.152Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.047176564 +0000 UTC m=+16.862316120 (1 nodes), 77.350959ms from start of poll, 79.285869ms from cluster start. Timeout at 2022-06-22 14:53:24.152771089 +0000 UTC m=+16.967910635, 105.594515ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.152Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.047176564 +0000 UTC m=+16.862316120 (1 nodes), 77.350959ms from start of poll, 79.285869ms from cluster start. Timeout at 2022-06-22 14:53:24.152771089 +0000 UTC m=+16.967910635, 105.594515ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.153Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:24.153Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.258Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.153060957 +0000 UTC m=+16.968200503 (1 nodes), 1.362µs from start of poll, 185.170252ms from cluster start. Timeout at 2022-06-22 14:53:24.25888087 +0000 UTC m=+17.074020416, 105.819913ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.258Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.153060957 +0000 UTC m=+16.968200503 (1 nodes), 1.362µs from start of poll, 185.170252ms from cluster start. Timeout at 2022-06-22 14:53:24.25888087 +0000 UTC m=+17.074020416, 105.819913ms after stability testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering candidate state: node="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Candidate]" term=3 testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering candidate state: node="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Candidate]" term=3 testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: election won: tally=2 testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: election won: tally=2 testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering leader state: leader="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Leader]" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: entering leader state: leader="Node at 0841d69e-2808-4435-1648-56ad4126d917 [Leader]" testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: added peer, starting replication: peer=server-f998619c-3403-1634-a11b-6f18efd3c485 testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: added peer, starting replication: peer=server-f998619c-3403-1634-a11b-6f18efd3c485 testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: added peer, starting replication: peer=server-bb669090-33ea-ece3-06b5-7885fbfe6411 testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: added peer, starting replication: peer=server-bb669090-33ea-ece3-06b5-7885fbfe6411 testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering follower state: follower="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Follower]" leader= testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: entering follower state: follower="Node at f998619c-3403-1634-a11b-6f18efd3c485 [Follower]" leader= testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: pipelining replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.259Z [INFO] starting stability test: raft-state=Leader testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: pipelining replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: aborting pipeline replication: peer="{Voter server-0841d69e-2808-4435-1648-56ad4126d917 0841d69e-2808-4435-1648-56ad4126d917}" testing.go:149: cluster: 2022-06-22T14:53:24.259Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: pipelining replication: peer="{Voter server-f998619c-3403-1634-a11b-6f18efd3c485 f998619c-3403-1634-a11b-6f18efd3c485}" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: aborting pipeline replication: peer="{Voter server-0841d69e-2808-4435-1648-56ad4126d917 0841d69e-2808-4435-1648-56ad4126d917}" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.259Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: pipelining replication: peer="{Voter server-f998619c-3403-1634-a11b-6f18efd3c485 f998619c-3403-1634-a11b-6f18efd3c485}" testLoggerAdapter verbose: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: aborting pipeline replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testing.go:149: server-f998619c-3403-1634-a11b-6f18efd3c485: 2022-06-22T14:53:24.259Z [INFO] server-f998619c-3403-1634-a11b-6f18efd3c485: aborting pipeline replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.365Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.259645867 +0000 UTC m=+17.074785413 (1 nodes), 22.773µs from start of poll, 291.755162ms from cluster start. Timeout at 2022-06-22 14:53:24.365523148 +0000 UTC m=+17.180662714, 105.877301ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.365Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.259645867 +0000 UTC m=+17.074785413 (1 nodes), 22.773µs from start of poll, 291.755162ms from cluster start. Timeout at 2022-06-22 14:53:24.365523148 +0000 UTC m=+17.180662714, 105.877301ms after stability testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.365Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: aborting pipeline replication: peer="{Voter server-f998619c-3403-1634-a11b-6f18efd3c485 f998619c-3403-1634-a11b-6f18efd3c485}" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.365Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: aborting pipeline replication: peer="{Voter server-f998619c-3403-1634-a11b-6f18efd3c485 f998619c-3403-1634-a11b-6f18efd3c485}" testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.365Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: aborting pipeline replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.365Z [INFO] server-0841d69e-2808-4435-1648-56ad4126d917: aborting pipeline replication: peer="{Voter server-bb669090-33ea-ece3-06b5-7885fbfe6411 bb669090-33ea-ece3-06b5-7885fbfe6411}" testLoggerAdapter verbose: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.416Z [ERROR] server-0841d69e-2808-4435-1648-56ad4126d917: failed to heartbeat to: peer=f998619c-3403-1634-a11b-6f18efd3c485 error="command timed out" testing.go:149: server-0841d69e-2808-4435-1648-56ad4126d917: 2022-06-22T14:53:24.416Z [ERROR] server-0841d69e-2808-4435-1648-56ad4126d917: failed to heartbeat to: peer=f998619c-3403-1634-a11b-6f18efd3c485 error="command timed out" --- PASS: TestRaft_LeadershipTransfer (0.45s) === RUN TestRaft_LeadershipTransferWithOneNode testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.417Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f18cca8-a38c-6fd8-85df-86153157536d Address:3f18cca8-a38c-6fd8-85df-86153157536d}]" testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.417Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f18cca8-a38c-6fd8-85df-86153157536d Address:3f18cca8-a38c-6fd8-85df-86153157536d}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.417Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:24.417Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.417Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering follower state: follower="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Follower]" leader= testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.417Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering follower state: follower="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Follower]" leader= testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [WARN] server-3f18cca8-a38c-6fd8-85df-86153157536d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [WARN] server-3f18cca8-a38c-6fd8-85df-86153157536d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering candidate state: node="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Candidate]" term=2 testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering candidate state: node="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Candidate]" term=2 testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: election won: tally=1 testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.470Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: election won: tally=1 testLoggerAdapter verbose: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.471Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering leader state: leader="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Leader]" testing.go:149: server-3f18cca8-a38c-6fd8-85df-86153157536d: 2022-06-22T14:53:24.471Z [INFO] server-3f18cca8-a38c-6fd8-85df-86153157536d: entering leader state: leader="Node at 3f18cca8-a38c-6fd8-85df-86153157536d [Leader]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.576Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.471080973 +0000 UTC m=+17.286220519 (1 nodes), 53.245304ms from start of poll, 53.53965ms from cluster start. Timeout at 2022-06-22 14:53:24.576137172 +0000 UTC m=+17.391276718, 105.056199ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.576Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.471080973 +0000 UTC m=+17.286220519 (1 nodes), 53.245304ms from start of poll, 53.53965ms from cluster start. Timeout at 2022-06-22 14:53:24.576137172 +0000 UTC m=+17.391276718, 105.056199ms after stability --- PASS: TestRaft_LeadershipTransferWithOneNode (0.16s) === RUN TestRaft_LeadershipTransferWithSevenNodes testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.577Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.577Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.578Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.578Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering follower state: follower="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Follower]" leader= testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.578Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.578Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering follower state: follower="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Follower]" leader= testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.578Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering follower state: follower="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Follower]" leader= testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.578Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering follower state: follower="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Follower]" leader= testLoggerAdapter verbose: server-b0483f50-4b33-ffee-0011-e0bf10655260: 2022-06-22T14:53:24.578Z [INFO] server-b0483f50-4b33-ffee-0011-e0bf10655260: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-b0483f50-4b33-ffee-0011-e0bf10655260: 2022-06-22T14:53:24.578Z [INFO] server-b0483f50-4b33-ffee-0011-e0bf10655260: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-abc38128-424c-8ad7-b840-2664a8960608: 2022-06-22T14:53:24.578Z [INFO] server-abc38128-424c-8ad7-b840-2664a8960608: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-b0483f50-4b33-ffee-0011-e0bf10655260: 2022-06-22T14:53:24.578Z [INFO] server-b0483f50-4b33-ffee-0011-e0bf10655260: entering follower state: follower="Node at b0483f50-4b33-ffee-0011-e0bf10655260 [Follower]" leader= testing.go:149: server-abc38128-424c-8ad7-b840-2664a8960608: 2022-06-22T14:53:24.578Z [INFO] server-abc38128-424c-8ad7-b840-2664a8960608: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-b0483f50-4b33-ffee-0011-e0bf10655260: 2022-06-22T14:53:24.578Z [INFO] server-b0483f50-4b33-ffee-0011-e0bf10655260: entering follower state: follower="Node at b0483f50-4b33-ffee-0011-e0bf10655260 [Follower]" leader= testLoggerAdapter verbose: server-abc38128-424c-8ad7-b840-2664a8960608: 2022-06-22T14:53:24.578Z [INFO] server-abc38128-424c-8ad7-b840-2664a8960608: entering follower state: follower="Node at abc38128-424c-8ad7-b840-2664a8960608 [Follower]" leader= testing.go:149: server-abc38128-424c-8ad7-b840-2664a8960608: 2022-06-22T14:53:24.578Z [INFO] server-abc38128-424c-8ad7-b840-2664a8960608: entering follower state: follower="Node at abc38128-424c-8ad7-b840-2664a8960608 [Follower]" leader= testLoggerAdapter verbose: server-14722325-85ff-c516-3101-c47a48167c38: 2022-06-22T14:53:24.578Z [INFO] server-14722325-85ff-c516-3101-c47a48167c38: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-14722325-85ff-c516-3101-c47a48167c38: 2022-06-22T14:53:24.578Z [INFO] server-14722325-85ff-c516-3101-c47a48167c38: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-14722325-85ff-c516-3101-c47a48167c38: 2022-06-22T14:53:24.578Z [INFO] server-14722325-85ff-c516-3101-c47a48167c38: entering follower state: follower="Node at 14722325-85ff-c516-3101-c47a48167c38 [Follower]" leader= testing.go:149: server-14722325-85ff-c516-3101-c47a48167c38: 2022-06-22T14:53:24.578Z [INFO] server-14722325-85ff-c516-3101-c47a48167c38: entering follower state: follower="Node at 14722325-85ff-c516-3101-c47a48167c38 [Follower]" leader= testLoggerAdapter verbose: server-51f705a0-252f-6742-d8ff-691b7e4c88b5: 2022-06-22T14:53:24.579Z [INFO] server-51f705a0-252f-6742-d8ff-691b7e4c88b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-51f705a0-252f-6742-d8ff-691b7e4c88b5: 2022-06-22T14:53:24.579Z [INFO] server-51f705a0-252f-6742-d8ff-691b7e4c88b5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: server-51f705a0-252f-6742-d8ff-691b7e4c88b5: 2022-06-22T14:53:24.579Z [INFO] server-51f705a0-252f-6742-d8ff-691b7e4c88b5: entering follower state: follower="Node at 51f705a0-252f-6742-d8ff-691b7e4c88b5 [Follower]" leader= testing.go:149: server-51f705a0-252f-6742-d8ff-691b7e4c88b5: 2022-06-22T14:53:24.579Z [INFO] server-51f705a0-252f-6742-d8ff-691b7e4c88b5: entering follower state: follower="Node at 51f705a0-252f-6742-d8ff-691b7e4c88b5 [Follower]" leader= testLoggerAdapter verbose: server-f43052e0-0dc3-beb0-397e-da3b36b5248c: 2022-06-22T14:53:24.579Z [INFO] server-f43052e0-0dc3-beb0-397e-da3b36b5248c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testing.go:149: server-f43052e0-0dc3-beb0-397e-da3b36b5248c: 2022-06-22T14:53:24.579Z [INFO] server-f43052e0-0dc3-beb0-397e-da3b36b5248c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d Address:a87bc61e-2e78-3bc7-2bc2-b0365393f99d} {Suffrage:Voter ID:server-75b32d65-c668-92e8-8016-f225b45b42a9 Address:75b32d65-c668-92e8-8016-f225b45b42a9} {Suffrage:Voter ID:server-b0483f50-4b33-ffee-0011-e0bf10655260 Address:b0483f50-4b33-ffee-0011-e0bf10655260} {Suffrage:Voter ID:server-abc38128-424c-8ad7-b840-2664a8960608 Address:abc38128-424c-8ad7-b840-2664a8960608} {Suffrage:Voter ID:server-14722325-85ff-c516-3101-c47a48167c38 Address:14722325-85ff-c516-3101-c47a48167c38} {Suffrage:Voter ID:server-51f705a0-252f-6742-d8ff-691b7e4c88b5 Address:51f705a0-252f-6742-d8ff-691b7e4c88b5} {Suffrage:Voter ID:server-f43052e0-0dc3-beb0-397e-da3b36b5248c Address:f43052e0-0dc3-beb0-397e-da3b36b5248c}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.579Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:24.579Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f43052e0-0dc3-beb0-397e-da3b36b5248c: 2022-06-22T14:53:24.579Z [INFO] server-f43052e0-0dc3-beb0-397e-da3b36b5248c: entering follower state: follower="Node at f43052e0-0dc3-beb0-397e-da3b36b5248c [Follower]" leader= testing.go:149: server-f43052e0-0dc3-beb0-397e-da3b36b5248c: 2022-06-22T14:53:24.579Z [INFO] server-f43052e0-0dc3-beb0-397e-da3b36b5248c: entering follower state: follower="Node at f43052e0-0dc3-beb0-397e-da3b36b5248c [Follower]" leader= testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [WARN] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [WARN] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering candidate state: node="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Candidate]" term=2 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering candidate state: node="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Candidate]" term=2 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: election won: tally=4 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: election won: tally=4 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering leader state: leader="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Leader]" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering leader state: leader="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Leader]" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-75b32d65-c668-92e8-8016-f225b45b42a9 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-75b32d65-c668-92e8-8016-f225b45b42a9 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-b0483f50-4b33-ffee-0011-e0bf10655260 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-b0483f50-4b33-ffee-0011-e0bf10655260 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-abc38128-424c-8ad7-b840-2664a8960608 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-abc38128-424c-8ad7-b840-2664a8960608 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-14722325-85ff-c516-3101-c47a48167c38 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-14722325-85ff-c516-3101-c47a48167c38 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-51f705a0-252f-6742-d8ff-691b7e4c88b5 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-51f705a0-252f-6742-d8ff-691b7e4c88b5 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-f43052e0-0dc3-beb0-397e-da3b36b5248c testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: added peer, starting replication: peer=server-f43052e0-0dc3-beb0-397e-da3b36b5248c testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-75b32d65-c668-92e8-8016-f225b45b42a9 75b32d65-c668-92e8-8016-f225b45b42a9}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.631Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-75b32d65-c668-92e8-8016-f225b45b42a9 75b32d65-c668-92e8-8016-f225b45b42a9}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.632Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.632Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: pipelining replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.737Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.631820449 +0000 UTC m=+17.446960005 (1 nodes), 52.419264ms from start of poll, 54.193279ms from cluster start. Timeout at 2022-06-22 14:53:24.737028432 +0000 UTC m=+17.552167978, 105.207973ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.737Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.631820449 +0000 UTC m=+17.446960005 (1 nodes), 52.419264ms from start of poll, 54.193279ms from cluster start. Timeout at 2022-06-22 14:53:24.737028432 +0000 UTC m=+17.552167978, 105.207973ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.737Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:24.737Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.842Z [INFO] stable state for Follower reached at 2022-06-22 14:53:24.73727525 +0000 UTC m=+17.552414796 (6 nodes), 2.976µs from start of poll, 159.64807ms from cluster start. Timeout at 2022-06-22 14:53:24.842782221 +0000 UTC m=+17.657921767, 105.506971ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.842Z [INFO] stable state for Follower reached at 2022-06-22 14:53:24.73727525 +0000 UTC m=+17.552414796 (6 nodes), 2.976µs from start of poll, 159.64807ms from cluster start. Timeout at 2022-06-22 14:53:24.842782221 +0000 UTC m=+17.657921767, 105.506971ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.842Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:24.842Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.948Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.842880396 +0000 UTC m=+17.658019952 (1 nodes), 992ns from start of poll, 265.253226ms from cluster start. Timeout at 2022-06-22 14:53:24.948585583 +0000 UTC m=+17.763725139, 105.705187ms after stability testing.go:149: cluster: 2022-06-22T14:53:24.948Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.842880396 +0000 UTC m=+17.658019952 (1 nodes), 992ns from start of poll, 265.253226ms from cluster start. Timeout at 2022-06-22 14:53:24.948585583 +0000 UTC m=+17.763725139, 105.705187ms after stability testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.948Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering candidate state: node="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Candidate]" term=3 testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.948Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering candidate state: node="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Candidate]" term=3 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:24.949Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:24.949Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering follower state: follower="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Follower]" leader= testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: entering follower state: follower="Node at a87bc61e-2e78-3bc7-2bc2-b0365393f99d [Follower]" leader= testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: election won: tally=4 testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: election won: tally=4 testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering leader state: leader="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Leader]" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: entering leader state: leader="Node at 75b32d65-c668-92e8-8016-f225b45b42a9 [Leader]" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-b0483f50-4b33-ffee-0011-e0bf10655260 testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-b0483f50-4b33-ffee-0011-e0bf10655260 testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-abc38128-424c-8ad7-b840-2664a8960608 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-abc38128-424c-8ad7-b840-2664a8960608 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-75b32d65-c668-92e8-8016-f225b45b42a9 75b32d65-c668-92e8-8016-f225b45b42a9}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-14722325-85ff-c516-3101-c47a48167c38 testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-75b32d65-c668-92e8-8016-f225b45b42a9 75b32d65-c668-92e8-8016-f225b45b42a9}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-14722325-85ff-c516-3101-c47a48167c38 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-51f705a0-252f-6742-d8ff-691b7e4c88b5 testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-51f705a0-252f-6742-d8ff-691b7e4c88b5 testLoggerAdapter verbose: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-f43052e0-0dc3-beb0-397e-da3b36b5248c testing.go:149: server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: 2022-06-22T14:53:24.949Z [INFO] server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d: aborting pipeline replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: added peer, starting replication: peer=server-f43052e0-0dc3-beb0-397e-da3b36b5248c testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d a87bc61e-2e78-3bc7-2bc2-b0365393f99d}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d a87bc61e-2e78-3bc7-2bc2-b0365393f99d}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.950Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.950Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.950Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.950Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:24.949Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: pipelining replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.054Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.949400334 +0000 UTC m=+17.764539880 (1 nodes), 316.549µs from start of poll, 371.773154ms from cluster start. Timeout at 2022-06-22 14:53:25.054411595 +0000 UTC m=+17.869551141, 105.011261ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.054Z [INFO] stable state for Leader reached at 2022-06-22 14:53:24.949400334 +0000 UTC m=+17.764539880 (1 nodes), 316.549µs from start of poll, 371.773154ms from cluster start. Timeout at 2022-06-22 14:53:25.054411595 +0000 UTC m=+17.869551141, 105.011261ms after stability testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-f43052e0-0dc3-beb0-397e-da3b36b5248c f43052e0-0dc3-beb0-397e-da3b36b5248c}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d a87bc61e-2e78-3bc7-2bc2-b0365393f99d}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-a87bc61e-2e78-3bc7-2bc2-b0365393f99d a87bc61e-2e78-3bc7-2bc2-b0365393f99d}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-b0483f50-4b33-ffee-0011-e0bf10655260 b0483f50-4b33-ffee-0011-e0bf10655260}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-abc38128-424c-8ad7-b840-2664a8960608 abc38128-424c-8ad7-b840-2664a8960608}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-14722325-85ff-c516-3101-c47a48167c38 14722325-85ff-c516-3101-c47a48167c38}" testLoggerAdapter verbose: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" testing.go:149: server-75b32d65-c668-92e8-8016-f225b45b42a9: 2022-06-22T14:53:25.054Z [INFO] server-75b32d65-c668-92e8-8016-f225b45b42a9: aborting pipeline replication: peer="{Voter server-51f705a0-252f-6742-d8ff-691b7e4c88b5 51f705a0-252f-6742-d8ff-691b7e4c88b5}" --- PASS: TestRaft_LeadershipTransferWithSevenNodes (0.48s) === RUN TestRaft_LeadershipTransferToInvalidID testLoggerAdapter verbose: server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: 2022-06-22T14:53:25.058Z [INFO] server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testing.go:149: server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: 2022-06-22T14:53:25.058Z [INFO] server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testLoggerAdapter verbose: server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: 2022-06-22T14:53:25.058Z [INFO] server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: entering follower state: follower="Node at ea350025-ebfd-a815-3ca4-6d951a7f71fd [Follower]" leader= testing.go:149: server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: 2022-06-22T14:53:25.058Z [INFO] server-ea350025-ebfd-a815-3ca4-6d951a7f71fd: entering follower state: follower="Node at ea350025-ebfd-a815-3ca4-6d951a7f71fd [Follower]" leader= testLoggerAdapter verbose: server-8378323d-1428-2efe-78bd-687da3d210ab: 2022-06-22T14:53:25.058Z [INFO] server-8378323d-1428-2efe-78bd-687da3d210ab: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testing.go:149: server-8378323d-1428-2efe-78bd-687da3d210ab: 2022-06-22T14:53:25.058Z [INFO] server-8378323d-1428-2efe-78bd-687da3d210ab: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testLoggerAdapter verbose: server-8378323d-1428-2efe-78bd-687da3d210ab: 2022-06-22T14:53:25.058Z [INFO] server-8378323d-1428-2efe-78bd-687da3d210ab: entering follower state: follower="Node at 8378323d-1428-2efe-78bd-687da3d210ab [Follower]" leader= testing.go:149: server-8378323d-1428-2efe-78bd-687da3d210ab: 2022-06-22T14:53:25.058Z [INFO] server-8378323d-1428-2efe-78bd-687da3d210ab: entering follower state: follower="Node at 8378323d-1428-2efe-78bd-687da3d210ab [Follower]" leader= testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.058Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.058Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea350025-ebfd-a815-3ca4-6d951a7f71fd Address:ea350025-ebfd-a815-3ca4-6d951a7f71fd} {Suffrage:Voter ID:server-8378323d-1428-2efe-78bd-687da3d210ab Address:8378323d-1428-2efe-78bd-687da3d210ab} {Suffrage:Voter ID:server-c5837b59-7718-6ea8-4bfb-5fcbc708b446 Address:c5837b59-7718-6ea8-4bfb-5fcbc708b446}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.058Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:25.058Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.058Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering follower state: follower="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Follower]" leader= testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.058Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering follower state: follower="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Follower]" leader= testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.123Z [WARN] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.123Z [WARN] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.123Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering candidate state: node="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Candidate]" term=2 testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.123Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering candidate state: node="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Candidate]" term=2 testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: election won: tally=2 testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: election won: tally=2 testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering leader state: leader="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Leader]" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: entering leader state: leader="Node at c5837b59-7718-6ea8-4bfb-5fcbc708b446 [Leader]" testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: added peer, starting replication: peer=server-ea350025-ebfd-a815-3ca4-6d951a7f71fd testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: added peer, starting replication: peer=server-ea350025-ebfd-a815-3ca4-6d951a7f71fd testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: added peer, starting replication: peer=server-8378323d-1428-2efe-78bd-687da3d210ab testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: added peer, starting replication: peer=server-8378323d-1428-2efe-78bd-687da3d210ab testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: pipelining replication: peer="{Voter server-8378323d-1428-2efe-78bd-687da3d210ab 8378323d-1428-2efe-78bd-687da3d210ab}" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: pipelining replication: peer="{Voter server-8378323d-1428-2efe-78bd-687da3d210ab 8378323d-1428-2efe-78bd-687da3d210ab}" testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: pipelining replication: peer="{Voter server-ea350025-ebfd-a815-3ca4-6d951a7f71fd ea350025-ebfd-a815-3ca4-6d951a7f71fd}" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.124Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: pipelining replication: peer="{Voter server-ea350025-ebfd-a815-3ca4-6d951a7f71fd ea350025-ebfd-a815-3ca4-6d951a7f71fd}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.229Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.124173958 +0000 UTC m=+17.939313504 (1 nodes), 65.233597ms from start of poll, 66.289805ms from cluster start. Timeout at 2022-06-22 14:53:25.229485685 +0000 UTC m=+18.044625241, 105.311737ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.229Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.124173958 +0000 UTC m=+17.939313504 (1 nodes), 65.233597ms from start of poll, 66.289805ms from cluster start. Timeout at 2022-06-22 14:53:25.229485685 +0000 UTC m=+18.044625241, 105.311737ms after stability testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.229Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: aborting pipeline replication: peer="{Voter server-8378323d-1428-2efe-78bd-687da3d210ab 8378323d-1428-2efe-78bd-687da3d210ab}" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.229Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: aborting pipeline replication: peer="{Voter server-8378323d-1428-2efe-78bd-687da3d210ab 8378323d-1428-2efe-78bd-687da3d210ab}" testLoggerAdapter verbose: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.229Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: aborting pipeline replication: peer="{Voter server-ea350025-ebfd-a815-3ca4-6d951a7f71fd ea350025-ebfd-a815-3ca4-6d951a7f71fd}" testing.go:149: server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: 2022-06-22T14:53:25.229Z [INFO] server-c5837b59-7718-6ea8-4bfb-5fcbc708b446: aborting pipeline replication: peer="{Voter server-ea350025-ebfd-a815-3ca4-6d951a7f71fd ea350025-ebfd-a815-3ca4-6d951a7f71fd}" --- PASS: TestRaft_LeadershipTransferToInvalidID (0.17s) === RUN TestRaft_LeadershipTransferToInvalidAddress testLoggerAdapter verbose: server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: 2022-06-22T14:53:25.230Z [INFO] server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testing.go:149: server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: 2022-06-22T14:53:25.230Z [INFO] server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testLoggerAdapter verbose: server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: 2022-06-22T14:53:25.231Z [INFO] server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: entering follower state: follower="Node at a4258dcd-37eb-1249-6d9b-0d19137b03fe [Follower]" leader= testing.go:149: server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: 2022-06-22T14:53:25.231Z [INFO] server-a4258dcd-37eb-1249-6d9b-0d19137b03fe: entering follower state: follower="Node at a4258dcd-37eb-1249-6d9b-0d19137b03fe [Follower]" leader= testLoggerAdapter verbose: server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: 2022-06-22T14:53:25.231Z [INFO] server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testing.go:149: server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: 2022-06-22T14:53:25.231Z [INFO] server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testLoggerAdapter verbose: server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: 2022-06-22T14:53:25.231Z [INFO] server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: entering follower state: follower="Node at 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 [Follower]" leader= testing.go:149: server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: 2022-06-22T14:53:25.231Z [INFO] server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251: entering follower state: follower="Node at 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 [Follower]" leader= testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.231Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.231Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a4258dcd-37eb-1249-6d9b-0d19137b03fe Address:a4258dcd-37eb-1249-6d9b-0d19137b03fe} {Suffrage:Voter ID:server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 Address:54fb77e4-bc8b-8ff1-a026-4bcfeefe6251} {Suffrage:Voter ID:server-faba4dad-e605-3961-d76f-53648d28ccf3 Address:faba4dad-e605-3961-d76f-53648d28ccf3}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.231Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:25.231Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.231Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering follower state: follower="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Follower]" leader= testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.231Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering follower state: follower="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Follower]" leader= testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [WARN] server-faba4dad-e605-3961-d76f-53648d28ccf3: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [WARN] server-faba4dad-e605-3961-d76f-53648d28ccf3: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering candidate state: node="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Candidate]" term=2 testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering candidate state: node="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Candidate]" term=2 testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: election won: tally=2 testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: election won: tally=2 testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering leader state: leader="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Leader]" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.292Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: entering leader state: leader="Node at faba4dad-e605-3961-d76f-53648d28ccf3 [Leader]" testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: added peer, starting replication: peer=server-a4258dcd-37eb-1249-6d9b-0d19137b03fe testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: added peer, starting replication: peer=server-a4258dcd-37eb-1249-6d9b-0d19137b03fe testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: added peer, starting replication: peer=server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: added peer, starting replication: peer=server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: pipelining replication: peer="{Voter server-a4258dcd-37eb-1249-6d9b-0d19137b03fe a4258dcd-37eb-1249-6d9b-0d19137b03fe}" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: pipelining replication: peer="{Voter server-a4258dcd-37eb-1249-6d9b-0d19137b03fe a4258dcd-37eb-1249-6d9b-0d19137b03fe}" testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: pipelining replication: peer="{Voter server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251}" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.293Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: pipelining replication: peer="{Voter server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.399Z [INFO] stable state for Follower reached at 2022-06-22 14:53:25.293131188 +0000 UTC m=+18.108270734 (2 nodes), 61.605645ms from start of poll, 62.29007ms from cluster start. Timeout at 2022-06-22 14:53:25.399107282 +0000 UTC m=+18.214246828, 105.976094ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.399Z [INFO] stable state for Follower reached at 2022-06-22 14:53:25.293131188 +0000 UTC m=+18.108270734 (2 nodes), 61.605645ms from start of poll, 62.29007ms from cluster start. Timeout at 2022-06-22 14:53:25.399107282 +0000 UTC m=+18.214246828, 105.976094ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.399Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:25.399Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.504Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.399352386 +0000 UTC m=+18.214491932 (1 nodes), 1.312µs from start of poll, 168.511268ms from cluster start. Timeout at 2022-06-22 14:53:25.504694781 +0000 UTC m=+18.319834337, 105.342405ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.504Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.399352386 +0000 UTC m=+18.214491932 (1 nodes), 1.312µs from start of poll, 168.511268ms from cluster start. Timeout at 2022-06-22 14:53:25.504694781 +0000 UTC m=+18.319834337, 105.342405ms after stability testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.505Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: aborting pipeline replication: peer="{Voter server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251}" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.505Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: aborting pipeline replication: peer="{Voter server-54fb77e4-bc8b-8ff1-a026-4bcfeefe6251 54fb77e4-bc8b-8ff1-a026-4bcfeefe6251}" testLoggerAdapter verbose: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.505Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: aborting pipeline replication: peer="{Voter server-a4258dcd-37eb-1249-6d9b-0d19137b03fe a4258dcd-37eb-1249-6d9b-0d19137b03fe}" testing.go:149: server-faba4dad-e605-3961-d76f-53648d28ccf3: 2022-06-22T14:53:25.505Z [INFO] server-faba4dad-e605-3961-d76f-53648d28ccf3: aborting pipeline replication: peer="{Voter server-a4258dcd-37eb-1249-6d9b-0d19137b03fe a4258dcd-37eb-1249-6d9b-0d19137b03fe}" --- PASS: TestRaft_LeadershipTransferToInvalidAddress (0.28s) === RUN TestRaft_LeadershipTransferToBehindServer testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.506Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.506Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.506Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.506Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.506Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering follower state: follower="Node at f0c11821-7874-74cb-0513-370c28d0859f [Follower]" leader= testLoggerAdapter verbose: server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: 2022-06-22T14:53:25.507Z [INFO] server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.506Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering follower state: follower="Node at f0c11821-7874-74cb-0513-370c28d0859f [Follower]" leader= testing.go:149: server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: 2022-06-22T14:53:25.507Z [INFO] server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0c11821-7874-74cb-0513-370c28d0859f Address:f0c11821-7874-74cb-0513-370c28d0859f} {Suffrage:Voter ID:server-c611dc2e-0443-3dce-8c68-73dac7169edf Address:c611dc2e-0443-3dce-8c68-73dac7169edf} {Suffrage:Voter ID:server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e Address:7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.507Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:25.507Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: 2022-06-22T14:53:25.507Z [INFO] server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: entering follower state: follower="Node at 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e [Follower]" leader= testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.507Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering follower state: follower="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Follower]" leader= testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.507Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering follower state: follower="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Follower]" leader= testing.go:149: server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: 2022-06-22T14:53:25.507Z [INFO] server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e: entering follower state: follower="Node at 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e [Follower]" leader= testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.580Z [WARN] server-f0c11821-7874-74cb-0513-370c28d0859f: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.580Z [WARN] server-f0c11821-7874-74cb-0513-370c28d0859f: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering candidate state: node="Node at f0c11821-7874-74cb-0513-370c28d0859f [Candidate]" term=2 testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering candidate state: node="Node at f0c11821-7874-74cb-0513-370c28d0859f [Candidate]" term=2 testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: election won: tally=2 testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: election won: tally=2 testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering leader state: leader="Node at f0c11821-7874-74cb-0513-370c28d0859f [Leader]" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering leader state: leader="Node at f0c11821-7874-74cb-0513-370c28d0859f [Leader]" testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: added peer, starting replication: peer=server-c611dc2e-0443-3dce-8c68-73dac7169edf testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: added peer, starting replication: peer=server-c611dc2e-0443-3dce-8c68-73dac7169edf testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: added peer, starting replication: peer=server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: added peer, starting replication: peer=server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: pipelining replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: pipelining replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: pipelining replication: peer="{Voter server-c611dc2e-0443-3dce-8c68-73dac7169edf c611dc2e-0443-3dce-8c68-73dac7169edf}" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.581Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: pipelining replication: peer="{Voter server-c611dc2e-0443-3dce-8c68-73dac7169edf c611dc2e-0443-3dce-8c68-73dac7169edf}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.687Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.581469387 +0000 UTC m=+18.396608943 (1 nodes), 73.754403ms from start of poll, 75.167196ms from cluster start. Timeout at 2022-06-22 14:53:25.68715459 +0000 UTC m=+18.502294146, 105.685203ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.687Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.581469387 +0000 UTC m=+18.396608943 (1 nodes), 73.754403ms from start of poll, 75.167196ms from cluster start. Timeout at 2022-06-22 14:53:25.68715459 +0000 UTC m=+18.502294146, 105.685203ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.687Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:25.687Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.792Z [INFO] stable state for Follower reached at 2022-06-22 14:53:25.687418459 +0000 UTC m=+18.502558005 (2 nodes), 1.683µs from start of poll, 181.116258ms from cluster start. Timeout at 2022-06-22 14:53:25.792516553 +0000 UTC m=+18.607656099, 105.098094ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.792Z [INFO] stable state for Follower reached at 2022-06-22 14:53:25.687418459 +0000 UTC m=+18.502558005 (2 nodes), 1.683µs from start of poll, 181.116258ms from cluster start. Timeout at 2022-06-22 14:53:25.792516553 +0000 UTC m=+18.607656099, 105.098094ms after stability testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering candidate state: node="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Candidate]" term=3 testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering candidate state: node="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Candidate]" term=3 testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: election won: tally=2 testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: election won: tally=2 testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering leader state: leader="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Leader]" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: entering leader state: leader="Node at c611dc2e-0443-3dce-8c68-73dac7169edf [Leader]" testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: added peer, starting replication: peer=server-f0c11821-7874-74cb-0513-370c28d0859f testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: added peer, starting replication: peer=server-f0c11821-7874-74cb-0513-370c28d0859f testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: added peer, starting replication: peer=server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: added peer, starting replication: peer=server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.815Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering follower state: follower="Node at f0c11821-7874-74cb-0513-370c28d0859f [Follower]" leader= testing.go:149: cluster: 2022-06-22T14:53:25.815Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: pipelining replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: pipelining replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: entering follower state: follower="Node at f0c11821-7874-74cb-0513-370c28d0859f [Follower]" leader= testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: aborting pipeline replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: aborting pipeline replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: pipelining replication: peer="{Voter server-f0c11821-7874-74cb-0513-370c28d0859f f0c11821-7874-74cb-0513-370c28d0859f}" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.815Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: pipelining replication: peer="{Voter server-f0c11821-7874-74cb-0513-370c28d0859f f0c11821-7874-74cb-0513-370c28d0859f}" testLoggerAdapter verbose: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: aborting pipeline replication: peer="{Voter server-c611dc2e-0443-3dce-8c68-73dac7169edf c611dc2e-0443-3dce-8c68-73dac7169edf}" testing.go:149: server-f0c11821-7874-74cb-0513-370c28d0859f: 2022-06-22T14:53:25.815Z [INFO] server-f0c11821-7874-74cb-0513-370c28d0859f: aborting pipeline replication: peer="{Voter server-c611dc2e-0443-3dce-8c68-73dac7169edf c611dc2e-0443-3dce-8c68-73dac7169edf}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.921Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.815696303 +0000 UTC m=+18.630836000 (1 nodes), 57.108µs from start of poll, 309.394253ms from cluster start. Timeout at 2022-06-22 14:53:25.921190685 +0000 UTC m=+18.736330251, 105.494251ms after stability testing.go:149: cluster: 2022-06-22T14:53:25.921Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.815696303 +0000 UTC m=+18.630836000 (1 nodes), 57.108µs from start of poll, 309.394253ms from cluster start. Timeout at 2022-06-22 14:53:25.921190685 +0000 UTC m=+18.736330251, 105.494251ms after stability testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.921Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: aborting pipeline replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.921Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: aborting pipeline replication: peer="{Voter server-7dc4c397-4a71-817a-9b5d-f2ff8e745e9e 7dc4c397-4a71-817a-9b5d-f2ff8e745e9e}" testLoggerAdapter verbose: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.921Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: aborting pipeline replication: peer="{Voter server-f0c11821-7874-74cb-0513-370c28d0859f f0c11821-7874-74cb-0513-370c28d0859f}" testing.go:149: server-c611dc2e-0443-3dce-8c68-73dac7169edf: 2022-06-22T14:53:25.921Z [INFO] server-c611dc2e-0443-3dce-8c68-73dac7169edf: aborting pipeline replication: peer="{Voter server-f0c11821-7874-74cb-0513-370c28d0859f f0c11821-7874-74cb-0513-370c28d0859f}" --- PASS: TestRaft_LeadershipTransferToBehindServer (0.42s) === RUN TestRaft_LeadershipTransferToItself testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.923Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.923Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.923Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering follower state: follower="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Follower]" leader= testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.923Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering follower state: follower="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Follower]" leader= testLoggerAdapter verbose: server-b2bce67a-ce7e-6de2-07ab-5302b189579c: 2022-06-22T14:53:25.923Z [INFO] server-b2bce67a-ce7e-6de2-07ab-5302b189579c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testing.go:149: server-b2bce67a-ce7e-6de2-07ab-5302b189579c: 2022-06-22T14:53:25.923Z [INFO] server-b2bce67a-ce7e-6de2-07ab-5302b189579c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testLoggerAdapter verbose: server-b2bce67a-ce7e-6de2-07ab-5302b189579c: 2022-06-22T14:53:25.923Z [INFO] server-b2bce67a-ce7e-6de2-07ab-5302b189579c: entering follower state: follower="Node at b2bce67a-ce7e-6de2-07ab-5302b189579c [Follower]" leader= testing.go:149: server-b2bce67a-ce7e-6de2-07ab-5302b189579c: 2022-06-22T14:53:25.923Z [INFO] server-b2bce67a-ce7e-6de2-07ab-5302b189579c: entering follower state: follower="Node at b2bce67a-ce7e-6de2-07ab-5302b189579c [Follower]" leader= testLoggerAdapter verbose: server-49512bef-be19-b26c-4fed-c29a9e72c357: 2022-06-22T14:53:25.924Z [INFO] server-49512bef-be19-b26c-4fed-c29a9e72c357: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testing.go:149: server-49512bef-be19-b26c-4fed-c29a9e72c357: 2022-06-22T14:53:25.924Z [INFO] server-49512bef-be19-b26c-4fed-c29a9e72c357: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 Address:f0151ae9-6c8a-09cd-04d9-8bc12eade8e8} {Suffrage:Voter ID:server-b2bce67a-ce7e-6de2-07ab-5302b189579c Address:b2bce67a-ce7e-6de2-07ab-5302b189579c} {Suffrage:Voter ID:server-49512bef-be19-b26c-4fed-c29a9e72c357 Address:49512bef-be19-b26c-4fed-c29a9e72c357}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:25.924Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:25.924Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-49512bef-be19-b26c-4fed-c29a9e72c357: 2022-06-22T14:53:25.924Z [INFO] server-49512bef-be19-b26c-4fed-c29a9e72c357: entering follower state: follower="Node at 49512bef-be19-b26c-4fed-c29a9e72c357 [Follower]" leader= testing.go:149: server-49512bef-be19-b26c-4fed-c29a9e72c357: 2022-06-22T14:53:25.924Z [INFO] server-49512bef-be19-b26c-4fed-c29a9e72c357: entering follower state: follower="Node at 49512bef-be19-b26c-4fed-c29a9e72c357 [Follower]" leader= testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [WARN] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [WARN] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering candidate state: node="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Candidate]" term=2 testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering candidate state: node="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Candidate]" term=2 testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: election won: tally=2 testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: election won: tally=2 testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering leader state: leader="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Leader]" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: entering leader state: leader="Node at f0151ae9-6c8a-09cd-04d9-8bc12eade8e8 [Leader]" testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: added peer, starting replication: peer=server-b2bce67a-ce7e-6de2-07ab-5302b189579c testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: added peer, starting replication: peer=server-b2bce67a-ce7e-6de2-07ab-5302b189579c testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: added peer, starting replication: peer=server-49512bef-be19-b26c-4fed-c29a9e72c357 testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: added peer, starting replication: peer=server-49512bef-be19-b26c-4fed-c29a9e72c357 testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: pipelining replication: peer="{Voter server-49512bef-be19-b26c-4fed-c29a9e72c357 49512bef-be19-b26c-4fed-c29a9e72c357}" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: pipelining replication: peer="{Voter server-49512bef-be19-b26c-4fed-c29a9e72c357 49512bef-be19-b26c-4fed-c29a9e72c357}" testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: pipelining replication: peer="{Voter server-b2bce67a-ce7e-6de2-07ab-5302b189579c b2bce67a-ce7e-6de2-07ab-5302b189579c}" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:25.989Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: pipelining replication: peer="{Voter server-b2bce67a-ce7e-6de2-07ab-5302b189579c b2bce67a-ce7e-6de2-07ab-5302b189579c}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.095Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.989608114 +0000 UTC m=+18.804747660 (1 nodes), 65.404289ms from start of poll, 66.339819ms from cluster start. Timeout at 2022-06-22 14:53:26.095665848 +0000 UTC m=+18.910805404, 106.057744ms after stability testing.go:149: cluster: 2022-06-22T14:53:26.095Z [INFO] stable state for Leader reached at 2022-06-22 14:53:25.989608114 +0000 UTC m=+18.804747660 (1 nodes), 65.404289ms from start of poll, 66.339819ms from cluster start. Timeout at 2022-06-22 14:53:26.095665848 +0000 UTC m=+18.910805404, 106.057744ms after stability testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.095Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: cannot transfer leadership to itself testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.095Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: cannot transfer leadership to itself testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.095Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: aborting pipeline replication: peer="{Voter server-49512bef-be19-b26c-4fed-c29a9e72c357 49512bef-be19-b26c-4fed-c29a9e72c357}" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.095Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: aborting pipeline replication: peer="{Voter server-49512bef-be19-b26c-4fed-c29a9e72c357 49512bef-be19-b26c-4fed-c29a9e72c357}" testLoggerAdapter verbose: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.096Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: aborting pipeline replication: peer="{Voter server-b2bce67a-ce7e-6de2-07ab-5302b189579c b2bce67a-ce7e-6de2-07ab-5302b189579c}" testing.go:149: server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: 2022-06-22T14:53:26.096Z [INFO] server-f0151ae9-6c8a-09cd-04d9-8bc12eade8e8: aborting pipeline replication: peer="{Voter server-b2bce67a-ce7e-6de2-07ab-5302b189579c b2bce67a-ce7e-6de2-07ab-5302b189579c}" --- PASS: TestRaft_LeadershipTransferToItself (0.17s) === RUN TestRaft_LeadershipTransferLeaderRejectsClientRequests raft_test.go:2064: Disabled unreliable test --- SKIP: TestRaft_LeadershipTransferLeaderRejectsClientRequests (0.00s) === RUN TestRaft_LeadershipTransferLeaderReplicationTimeout testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.097Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.097Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testLoggerAdapter verbose: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.097Z [INFO] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testing.go:149: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.097Z [INFO] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testLoggerAdapter verbose: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.097Z [INFO] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: entering follower state: follower="Node at 04a20b56-bbd9-1858-2a30-03a8917e35ac [Follower]" leader= testing.go:149: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.097Z [INFO] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: entering follower state: follower="Node at 04a20b56-bbd9-1858-2a30-03a8917e35ac [Follower]" leader= testLoggerAdapter verbose: server-bdd06344-d466-02b8-7097-1b1e913bdcb8: 2022-06-22T14:53:26.097Z [INFO] server-bdd06344-d466-02b8-7097-1b1e913bdcb8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testing.go:149: server-bdd06344-d466-02b8-7097-1b1e913bdcb8: 2022-06-22T14:53:26.097Z [INFO] server-bdd06344-d466-02b8-7097-1b1e913bdcb8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-16ba464f-0678-e12d-63e6-c0ab58dec1ce Address:16ba464f-0678-e12d-63e6-c0ab58dec1ce} {Suffrage:Voter ID:server-04a20b56-bbd9-1858-2a30-03a8917e35ac Address:04a20b56-bbd9-1858-2a30-03a8917e35ac} {Suffrage:Voter ID:server-bdd06344-d466-02b8-7097-1b1e913bdcb8 Address:bdd06344-d466-02b8-7097-1b1e913bdcb8}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.097Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:26.097Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: server-bdd06344-d466-02b8-7097-1b1e913bdcb8: 2022-06-22T14:53:26.097Z [INFO] server-bdd06344-d466-02b8-7097-1b1e913bdcb8: entering follower state: follower="Node at bdd06344-d466-02b8-7097-1b1e913bdcb8 [Follower]" leader= testing.go:149: server-bdd06344-d466-02b8-7097-1b1e913bdcb8: 2022-06-22T14:53:26.097Z [INFO] server-bdd06344-d466-02b8-7097-1b1e913bdcb8: entering follower state: follower="Node at bdd06344-d466-02b8-7097-1b1e913bdcb8 [Follower]" leader= testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.097Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering follower state: follower="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Follower]" leader= testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.097Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering follower state: follower="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Follower]" leader= testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [WARN] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [WARN] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering candidate state: node="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Candidate]" term=2 testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering candidate state: node="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Candidate]" term=2 testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: election won: tally=2 testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: election won: tally=2 testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering leader state: leader="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Leader]" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.157Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: entering leader state: leader="Node at 16ba464f-0678-e12d-63e6-c0ab58dec1ce [Leader]" testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: added peer, starting replication: peer=server-04a20b56-bbd9-1858-2a30-03a8917e35ac testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: added peer, starting replication: peer=server-04a20b56-bbd9-1858-2a30-03a8917e35ac testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: added peer, starting replication: peer=server-bdd06344-d466-02b8-7097-1b1e913bdcb8 testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: added peer, starting replication: peer=server-bdd06344-d466-02b8-7097-1b1e913bdcb8 testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: pipelining replication: peer="{Voter server-bdd06344-d466-02b8-7097-1b1e913bdcb8 bdd06344-d466-02b8-7097-1b1e913bdcb8}" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: pipelining replication: peer="{Voter server-bdd06344-d466-02b8-7097-1b1e913bdcb8 bdd06344-d466-02b8-7097-1b1e913bdcb8}" testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: pipelining replication: peer="{Voter server-04a20b56-bbd9-1858-2a30-03a8917e35ac 04a20b56-bbd9-1858-2a30-03a8917e35ac}" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.158Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: pipelining replication: peer="{Voter server-04a20b56-bbd9-1858-2a30-03a8917e35ac 04a20b56-bbd9-1858-2a30-03a8917e35ac}" testLoggerAdapter verbose: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.158Z [WARN] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: rejecting vote request since our last term is greater: candidate=16ba464f-0678-e12d-63e6-c0ab58dec1ce last-term=2 last-candidate-term=1 testing.go:149: server-04a20b56-bbd9-1858-2a30-03a8917e35ac: 2022-06-22T14:53:26.158Z [WARN] server-04a20b56-bbd9-1858-2a30-03a8917e35ac: rejecting vote request since our last term is greater: candidate=16ba464f-0678-e12d-63e6-c0ab58dec1ce last-term=2 last-candidate-term=1 testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.264Z [INFO] stable state for Leader reached at 2022-06-22 14:53:26.158448297 +0000 UTC m=+18.973587863 (1 nodes), 60.696875ms from start of poll, 61.300096ms from cluster start. Timeout at 2022-06-22 14:53:26.264482286 +0000 UTC m=+19.079621842, 106.033979ms after stability testing.go:149: cluster: 2022-06-22T14:53:26.264Z [INFO] stable state for Leader reached at 2022-06-22 14:53:26.158448297 +0000 UTC m=+18.973587863 (1 nodes), 60.696875ms from start of poll, 61.300096ms from cluster start. Timeout at 2022-06-22 14:53:26.264482286 +0000 UTC m=+19.079621842, 106.033979ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.264Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:26.264Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.369Z [INFO] stable state for Follower reached at 2022-06-22 14:53:26.264795879 +0000 UTC m=+19.079935425 (2 nodes), 1.683µs from start of poll, 167.647658ms from cluster start. Timeout at 2022-06-22 14:53:26.369907795 +0000 UTC m=+19.185047351, 105.111926ms after stability testing.go:149: cluster: 2022-06-22T14:53:26.369Z [INFO] stable state for Follower reached at 2022-06-22 14:53:26.264795879 +0000 UTC m=+19.079935425 (2 nodes), 1.683µs from start of poll, 167.647658ms from cluster start. Timeout at 2022-06-22 14:53:26.369907795 +0000 UTC m=+19.185047351, 105.111926ms after stability testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.588Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: aborting pipeline replication: peer="{Voter server-bdd06344-d466-02b8-7097-1b1e913bdcb8 bdd06344-d466-02b8-7097-1b1e913bdcb8}" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.588Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: aborting pipeline replication: peer="{Voter server-bdd06344-d466-02b8-7097-1b1e913bdcb8 bdd06344-d466-02b8-7097-1b1e913bdcb8}" testLoggerAdapter verbose: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.588Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: aborting pipeline replication: peer="{Voter server-04a20b56-bbd9-1858-2a30-03a8917e35ac 04a20b56-bbd9-1858-2a30-03a8917e35ac}" testing.go:149: server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: 2022-06-22T14:53:26.588Z [INFO] server-16ba464f-0678-e12d-63e6-c0ab58dec1ce: aborting pipeline replication: peer="{Voter server-04a20b56-bbd9-1858-2a30-03a8917e35ac 04a20b56-bbd9-1858-2a30-03a8917e35ac}" --- PASS: TestRaft_LeadershipTransferLeaderReplicationTimeout (0.49s) === RUN TestRaft_LeadershipTransferStopRightAway --- PASS: TestRaft_LeadershipTransferStopRightAway (0.00s) === RUN TestRaft_GetConfigurationNoBootstrap testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.590Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.590Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testLoggerAdapter verbose: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.590Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testing.go:149: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.590Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.590Z [INFO] starting stability test: raft-state=Follower testing.go:149: cluster: 2022-06-22T14:53:26.590Z [INFO] starting stability test: raft-state=Follower testLoggerAdapter verbose: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.590Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: entering follower state: follower="Node at a09eb517-d544-38b6-86b2-23e94ab325ba [Follower]" leader= testing.go:149: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.590Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: entering follower state: follower="Node at a09eb517-d544-38b6-86b2-23e94ab325ba [Follower]" leader= testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.590Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering follower state: follower="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Follower]" leader= testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.590Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering follower state: follower="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Follower]" leader= testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [WARN] server-08a1963d-afff-04c1-b193-f4bfa535da7b: heartbeat timeout reached, starting election: last-leader= testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [WARN] server-08a1963d-afff-04c1-b193-f4bfa535da7b: heartbeat timeout reached, starting election: last-leader= testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering candidate state: node="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Candidate]" term=2 testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering candidate state: node="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Candidate]" term=2 testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: election won: tally=2 testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: election won: tally=2 testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering leader state: leader="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Leader]" testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: entering leader state: leader="Node at 08a1963d-afff-04c1-b193-f4bfa535da7b [Leader]" testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: added peer, starting replication: peer=server-a09eb517-d544-38b6-86b2-23e94ab325ba testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: added peer, starting replication: peer=server-a09eb517-d544-38b6-86b2-23e94ab325ba testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: pipelining replication: peer="{Voter server-a09eb517-d544-38b6-86b2-23e94ab325ba a09eb517-d544-38b6-86b2-23e94ab325ba}" testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.657Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: pipelining replication: peer="{Voter server-a09eb517-d544-38b6-86b2-23e94ab325ba a09eb517-d544-38b6-86b2-23e94ab325ba}" testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.763Z [INFO] stable state for Follower reached at 2022-06-22 14:53:26.657428696 +0000 UTC m=+19.472568533 (1 nodes), 67.24537ms from start of poll, 67.533073ms from cluster start. Timeout at 2022-06-22 14:53:26.763048993 +0000 UTC m=+19.578188549, 105.620016ms after stability testing.go:149: cluster: 2022-06-22T14:53:26.763Z [INFO] stable state for Follower reached at 2022-06-22 14:53:26.657428696 +0000 UTC m=+19.472568533 (1 nodes), 67.24537ms from start of poll, 67.533073ms from cluster start. Timeout at 2022-06-22 14:53:26.763048993 +0000 UTC m=+19.578188549, 105.620016ms after stability testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.763Z [INFO] starting stability test: raft-state=Leader testing.go:149: cluster: 2022-06-22T14:53:26.763Z [INFO] starting stability test: raft-state=Leader testLoggerAdapter verbose: cluster: 2022-06-22T14:53:26.868Z [INFO] stable state for Leader reached at 2022-06-22 14:53:26.763353901 +0000 UTC m=+19.578493447 (1 nodes), 2.074µs from start of poll, 173.457987ms from cluster start. Timeout at 2022-06-22 14:53:26.868540177 +0000 UTC m=+19.683679733, 105.186286ms after stability testing.go:149: cluster: 2022-06-22T14:53:26.868Z [INFO] stable state for Leader reached at 2022-06-22 14:53:26.763353901 +0000 UTC m=+19.578493447 (1 nodes), 2.074µs from start of poll, 173.457987ms from cluster start. Timeout at 2022-06-22 14:53:26.868540177 +0000 UTC m=+19.683679733, 105.186286ms after stability testLoggerAdapter verbose: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.879Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testing.go:149: server-a09eb517-d544-38b6-86b2-23e94ab325ba: 2022-06-22T14:53:26.879Z [INFO] server-a09eb517-d544-38b6-86b2-23e94ab325ba: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-08a1963d-afff-04c1-b193-f4bfa535da7b Address:08a1963d-afff-04c1-b193-f4bfa535da7b} {Suffrage:Voter ID:server-a09eb517-d544-38b6-86b2-23e94ab325ba Address:a09eb517-d544-38b6-86b2-23e94ab325ba}]" testLoggerAdapter verbose: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.879Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: aborting pipeline replication: peer="{Voter server-a09eb517-d544-38b6-86b2-23e94ab325ba a09eb517-d544-38b6-86b2-23e94ab325ba}" testing.go:149: server-08a1963d-afff-04c1-b193-f4bfa535da7b: 2022-06-22T14:53:26.879Z [INFO] server-08a1963d-afff-04c1-b193-f4bfa535da7b: aborting pipeline replication: peer="{Voter server-a09eb517-d544-38b6-86b2-23e94ab325ba a09eb517-d544-38b6-86b2-23e94ab325ba}" --- PASS: TestRaft_GetConfigurationNoBootstrap (0.29s) === RUN TestTCPTransport_BadAddr --- PASS: TestTCPTransport_BadAddr (0.00s) === RUN TestTCPTransport_WithAdvertise --- PASS: TestTCPTransport_WithAdvertise (0.00s) === RUN TestTransport_StartStop --- PASS: TestTransport_StartStop (0.00s) === RUN TestTransport_AppendEntries --- PASS: TestTransport_AppendEntries (0.00s) === RUN TestTransport_AppendEntriesPipeline --- PASS: TestTransport_AppendEntriesPipeline (0.00s) === RUN TestTransport_RequestVote --- PASS: TestTransport_RequestVote (0.00s) === RUN TestTransport_InstallSnapshot --- PASS: TestTransport_InstallSnapshot (0.00s) === RUN TestTransport_EncodeDecode --- PASS: TestTransport_EncodeDecode (0.00s) === RUN TestRandomTimeout --- PASS: TestRandomTimeout (0.00s) === RUN TestNewSeed --- PASS: TestNewSeed (0.00s) === RUN TestRandomTimeout_NoTime --- PASS: TestRandomTimeout_NoTime (0.00s) === RUN TestMin --- PASS: TestMin (0.00s) === RUN TestMax --- PASS: TestMax (0.00s) === RUN TestGenerateUUID --- PASS: TestGenerateUUID (0.00s) === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok github.com/hashicorp/raft 19.712s ? github.com/hashicorp/raft/bench [no test files] create-stamp debian/debhelper-build-stamp fakeroot debian/rules binary dh binary --buildsystem=golang --with=golang dh_testroot -O--buildsystem=golang dh_prep -O--buildsystem=golang dh_auto_install --destdir=debian/golang-github-hashicorp-raft-dev/ -O--buildsystem=golang dh_installdocs -O--buildsystem=golang dh_installchangelogs -O--buildsystem=golang dh_installinit -O--buildsystem=golang dh_installsystemduser -O--buildsystem=golang dh_perl -O--buildsystem=golang dh_link -O--buildsystem=golang dh_strip_nondeterminism -O--buildsystem=golang dh_compress -O--buildsystem=golang dh_fixperms -O--buildsystem=golang dh_missing -O--buildsystem=golang dh_installdeb -O--buildsystem=golang dh_golang -O--buildsystem=golang dh_gencontrol -O--buildsystem=golang dpkg-gencontrol: warning: Depends field of package golang-github-hashicorp-raft-dev: substitution variable ${shlibs:Depends} used, but is not defined dh_md5sums -O--buildsystem=golang dh_builddeb -O--buildsystem=golang INFO: pkgstriptranslations version 149 pkgstriptranslations: processing golang-github-hashicorp-raft-dev (in debian/golang-github-hashicorp-raft-dev); do_strip: , oemstrip: pkgmaintainermangler: Maintainer field overridden to "Ubuntu Developers " pkgstripfiles: processing control file: debian/golang-github-hashicorp-raft-dev/DEBIAN/control, package golang-github-hashicorp-raft-dev, directory debian/golang-github-hashicorp-raft-dev pkgstripfiles: Truncating usr/share/doc/golang-github-hashicorp-raft-dev/changelog.Debian.gz to topmost ten records pkgstripfiles: Running PNG optimization (using 4 cpus) for package golang-github-hashicorp-raft-dev ... pkgstripfiles: No PNG files. dpkg-deb: building package 'golang-github-hashicorp-raft-dev' in '../golang-github-hashicorp-raft-dev_1.1.2-1_all.deb'. dpkg-genbuildinfo --build=binary -O../golang-github-hashicorp-raft_1.1.2-1_amd64.buildinfo dpkg-genchanges --build=binary -mLaunchpad Build Daemon -O../golang-github-hashicorp-raft_1.1.2-1_amd64.changes dpkg-genchanges: info: binary-only upload (no source code included) dpkg-source --after-build . dpkg-buildpackage: info: binary-only upload (no source included) -------------------------------------------------------------------------------- Build finished at 2022-06-22T14:53:30Z Finished -------- I: Built successfully +------------------------------------------------------------------------------+ | Changes | +------------------------------------------------------------------------------+ golang-github-hashicorp-raft_1.1.2-1_amd64.changes: --------------------------------------------------- Format: 1.8 Date: Sun, 19 Jan 2020 18:15:42 +1100 Source: golang-github-hashicorp-raft Binary: golang-github-hashicorp-raft-dev Built-For-Profiles: noudeb Architecture: all Version: 1.1.2-1 Distribution: kinetic Urgency: medium Maintainer: Launchpad Build Daemon Changed-By: Dmitry Smirnov Description: golang-github-hashicorp-raft-dev - implementation of the Raft consensus protocol Changes: golang-github-hashicorp-raft (1.1.2-1) unstable; urgency=medium . * New upstream release. Checksums-Sha1: 9e346ef2b0dd4f0c624a6a5e447e2b851a02be46 90020 golang-github-hashicorp-raft-dev_1.1.2-1_all.deb 1bf9aa0d85f9c321b1d82016ed60b2062e9b9bdf 9306 golang-github-hashicorp-raft_1.1.2-1_amd64.buildinfo Checksums-Sha256: 50a7b825aa08d190d0c9b91ba502a7e01f7d9f9a77af074ac5a218c312104fc9 90020 golang-github-hashicorp-raft-dev_1.1.2-1_all.deb 4acb7c0067089e29f6f0c4c51c3154704b8f30377bbc2d66e4dd00dee1a87904 9306 golang-github-hashicorp-raft_1.1.2-1_amd64.buildinfo Files: f4536af40db20b9374cb65bc5756f4e7 90020 devel optional golang-github-hashicorp-raft-dev_1.1.2-1_all.deb 8ff6a052f8db7f6209848109154fe0c0 9306 devel optional golang-github-hashicorp-raft_1.1.2-1_amd64.buildinfo /<>/golang-github-hashicorp-raft_1.1.2-1_amd64.changes.new could not be renamed to /<>/golang-github-hashicorp-raft_1.1.2-1_amd64.changes: Illegal seek Distribution field may be wrong!!! +------------------------------------------------------------------------------+ | Buildinfo | +------------------------------------------------------------------------------+ Format: 1.0 Source: golang-github-hashicorp-raft Binary: golang-github-hashicorp-raft-dev Architecture: all Version: 1.1.2-1 Checksums-Md5: f4536af40db20b9374cb65bc5756f4e7 90020 golang-github-hashicorp-raft-dev_1.1.2-1_all.deb Checksums-Sha1: 9e346ef2b0dd4f0c624a6a5e447e2b851a02be46 90020 golang-github-hashicorp-raft-dev_1.1.2-1_all.deb Checksums-Sha256: 50a7b825aa08d190d0c9b91ba502a7e01f7d9f9a77af074ac5a218c312104fc9 90020 golang-github-hashicorp-raft-dev_1.1.2-1_all.deb Build-Origin: Ubuntu Build-Architecture: amd64 Build-Date: Wed, 22 Jun 2022 14:53:30 +0000 Build-Path: /<> Build-Tainted-By: merged-usr-via-aliased-dirs usr-local-has-programs Installed-Build-Depends: autoconf (= 2.71-2), automake (= 1:1.16.5-1.3), autopoint (= 0.21-6), autotools-dev (= 20220109.1), base-files (= 12.2ubuntu1), base-passwd (= 3.5.52build1), bash (= 5.1-6ubuntu1), binutils (= 2.38.50.20220615-4ubuntu1), binutils-common (= 2.38.50.20220615-4ubuntu1), binutils-x86-64-linux-gnu (= 2.38.50.20220615-4ubuntu1), bsdextrautils (= 2.38-4ubuntu1), bsdutils (= 1:2.38-4ubuntu1), build-essential (= 12.9ubuntu3), bzip2 (= 1.0.8-5build1), coreutils (= 8.32-4.1ubuntu1), cpp (= 4:11.2.0-1ubuntu1), cpp-11 (= 11.3.0-3ubuntu1), dash (= 0.5.11+git20210903+057cd650a4ed-3build1), debconf (= 1.5.79ubuntu1), debhelper (= 13.7.1ubuntu1), debianutils (= 5.7-0.2), debugedit (= 1:5.0-4build1), dh-autoreconf (= 20), dh-golang (= 1.56), dh-strip-nondeterminism (= 1.13.0-1), diffutils (= 1:3.8-0ubuntu2), dpkg (= 1.21.8ubuntu1), dpkg-dev (= 1.21.8ubuntu1), dwz (= 0.14-1build2), file (= 1:5.41-4), findutils (= 4.8.0-1ubuntu3), g++ (= 4:11.2.0-1ubuntu1), g++-11 (= 11.3.0-3ubuntu1), gcc (= 4:11.2.0-1ubuntu1), gcc-11 (= 11.3.0-3ubuntu1), gcc-11-base (= 11.3.0-3ubuntu1), gcc-12-base (= 12.1.0-2ubuntu1), gettext (= 0.21-6), gettext-base (= 0.21-6), golang-1.18-go (= 1.18.3-1ubuntu1), golang-1.18-src (= 1.18.3-1ubuntu1), golang-any (= 2:1.18~0ubuntu2), golang-github-alecthomas-units-dev (= 0.0~git20211218.b94a6e3-1), golang-github-armon-go-metrics-dev (= 0.3.4-1), golang-github-aws-aws-sdk-go-dev (= 1.41.14-1ubuntu1), golang-github-beorn7-perks-dev (= 1.0.1-1), golang-github-cespare-xxhash-dev (= 2.1.1-2), golang-github-circonus-labs-circonus-gometrics-dev (= 2.3.1-3), golang-github-circonus-labs-circonusllhist-dev (= 0.0~git20191022.ec08cde-1), golang-github-creack-pty-dev (= 1.1.18-1), golang-github-datadog-datadog-go-dev (= 2.1.0-3), golang-github-davecgh-go-spew-dev (= 1.1.1-2), golang-github-fatih-color-dev (= 1.7.0-1), golang-github-go-kit-log-dev (= 0.2.0-3), golang-github-go-logfmt-logfmt-dev (= 0.5.0-2), golang-github-hashicorp-go-cleanhttp-dev (= 0.5.2-1), golang-github-hashicorp-go-hclog-dev (= 0.11.0-1), golang-github-hashicorp-go-immutable-radix-dev (= 1.3.1-1), golang-github-hashicorp-go-msgpack-dev (= 0.5.5-1), golang-github-hashicorp-go-retryablehttp-dev (= 0.7.0-1), golang-github-hashicorp-go-uuid-dev (= 1.0.2-1), golang-github-hashicorp-golang-lru-dev (= 0.5.4-2), golang-github-jmespath-go-jmespath-dev (= 0.4.0-2), golang-github-jpillora-backoff-dev (= 1.0.0-1.1), golang-github-json-iterator-go-dev (= 1.1.12-1), golang-github-julienschmidt-httprouter-dev (= 1.3.0-1), golang-github-kr-pretty-dev (= 0.3.0-1), golang-github-kr-text-dev (= 0.2.0-1), golang-github-mattn-go-colorable-dev (= 0.1.12-1), golang-github-mattn-go-isatty-dev (= 0.0.14-1), golang-github-modern-go-concurrent-dev (= 1.0.3-1.1), golang-github-modern-go-reflect2-dev (= 1.0.2-2), golang-github-mwitkow-go-conntrack-dev (= 0.0~git20190716.2f06839-2), golang-github-pascaldekloe-goe-dev (= 0.1.0-3), golang-github-pkg-diff-dev (= 0.0~git20210226.20ebb0f-1), golang-github-pkg-errors-dev (= 0.9.1-2), golang-github-pmezard-go-difflib-dev (= 1.0.0-3), golang-github-prometheus-client-golang-dev (= 1.11.1-1), golang-github-prometheus-client-model-dev (= 0.2.0-3), golang-github-prometheus-common-dev (= 0.32.1-8), golang-github-prometheus-procfs-dev (= 0.7.3-2), golang-github-rogpeppe-go-internal-dev (= 1.8.1-1), golang-github-stretchr-objx-dev (= 0.3.0-1), golang-github-stretchr-testify-dev (= 1.7.2-1), golang-github-tv42-httpunix-dev (= 0.0~git20150427.b75d861-2), golang-github-vmihailenco-tagparser-dev (= 0.1.1-3), golang-go (= 2:1.18~0ubuntu2), golang-golang-x-net-dev (= 1:0.0+git20211209.491a49a+dfsg-1), golang-golang-x-oauth2-dev (= 0.0~git20211104.d3ed0bb-1), golang-golang-x-sync-dev (= 0.0~git20210220.036812b-1), golang-golang-x-sys-dev (= 0.0~git20220520.bc2c85a-1), golang-golang-x-text-dev (= 0.3.7-1), golang-gopkg-alecthomas-kingpin.v2-dev (= 2.2.6-3), golang-gopkg-check.v1-dev (= 0.0+git20200902.038fdea-1), golang-gopkg-errgo.v2-dev (= 2.1.0-2), golang-gopkg-mgo.v2-dev (= 2016.08.01-7), golang-gopkg-tomb.v2-dev (= 0.0~git20161208.d5d1b58-3), golang-gopkg-vmihailenco-msgpack.v2-dev (= 4.3.1-2), golang-gopkg-yaml.v2-dev (= 2.4.0-3), golang-gopkg-yaml.v3-dev (= 3.0.1-1), golang-goprotobuf-dev (= 1.3.4-2), golang-protobuf-extensions-dev (= 1.0.1-1), golang-src (= 2:1.18~0ubuntu2), grep (= 3.7-1build1), groff-base (= 1.22.4-8build1), gzip (= 1.10-4ubuntu4), hostname (= 3.23ubuntu2), init-system-helpers (= 1.63), intltool-debian (= 0.35.0+20060710.5), libacl1 (= 2.3.1-1), libarchive-zip-perl (= 1.68-1), libasan6 (= 11.3.0-3ubuntu1), libatomic1 (= 12.1.0-2ubuntu1), libattr1 (= 1:2.5.1-1build1), libaudit-common (= 1:3.0.7-1build1), libaudit1 (= 1:3.0.7-1build1), libbinutils (= 2.38.50.20220615-4ubuntu1), libblkid1 (= 2.38-4ubuntu1), libbz2-1.0 (= 1.0.8-5build1), libc-bin (= 2.35-0ubuntu3), libc-dev-bin (= 2.35-0ubuntu3), libc6 (= 2.35-0ubuntu3), libc6-dev (= 2.35-0ubuntu3), libcap-ng0 (= 0.7.9-4), libcap2 (= 1:2.44-1build3), libcc1-0 (= 12.1.0-2ubuntu1), libcom-err2 (= 1.46.5-2ubuntu2), libcrypt-dev (= 1:4.4.27-1), libcrypt1 (= 1:4.4.27-1), libctf-nobfd0 (= 2.38.50.20220615-4ubuntu1), libctf0 (= 2.38.50.20220615-4ubuntu1), libdb5.3 (= 5.3.28+dfsg1-0.9), libdebconfclient0 (= 0.261ubuntu1), libdebhelper-perl (= 13.7.1ubuntu1), libdpkg-perl (= 1.21.8ubuntu1), libdw1 (= 0.187-1), libelf1 (= 0.187-1), libfile-stripnondeterminism-perl (= 1.13.0-1), libgcc-11-dev (= 11.3.0-3ubuntu1), libgcc-s1 (= 12.1.0-2ubuntu1), libgcrypt20 (= 1.10.1-2ubuntu1), libgdbm-compat4 (= 1.23-1), libgdbm6 (= 1.23-1), libgmp10 (= 2:6.2.1+dfsg-3ubuntu1), libgomp1 (= 12.1.0-2ubuntu1), libgpg-error0 (= 1.45-2), libgprofng0 (= 2.38.50.20220615-4ubuntu1), libgssapi-krb5-2 (= 1.19.2-2), libicu71 (= 71.1-3), libisl23 (= 0.24-2build1), libitm1 (= 12.1.0-2ubuntu1), libk5crypto3 (= 1.19.2-2), libkeyutils1 (= 1.6.1-3ubuntu1), libkrb5-3 (= 1.19.2-2), libkrb5support0 (= 1.19.2-2), liblsan0 (= 12.1.0-2ubuntu1), liblz4-1 (= 1.9.3-2build2), liblzma5 (= 5.2.5-2.1), libmagic-mgc (= 1:5.41-4), libmagic1 (= 1:5.41-4), libmount1 (= 2.38-4ubuntu1), libmpc3 (= 1.2.1-2build1), libmpfr6 (= 4.1.0-3build3), libnsl-dev (= 1.3.0-2build2), libnsl2 (= 1.3.0-2build2), libpam-modules (= 1.4.0-13ubuntu1), libpam-modules-bin (= 1.4.0-13ubuntu1), libpam-runtime (= 1.4.0-13ubuntu1), libpam0g (= 1.4.0-13ubuntu1), libpcre2-8-0 (= 10.40-1), libpcre3 (= 2:8.39-14), libperl5.34 (= 5.34.0-3ubuntu1), libpipeline1 (= 1.5.6-1), libprotobuf-dev (= 3.12.4-1ubuntu7), libprotobuf-lite23 (= 3.12.4-1ubuntu7), libprotobuf23 (= 3.12.4-1ubuntu7), libprotoc23 (= 3.12.4-1ubuntu7), libquadmath0 (= 12.1.0-2ubuntu1), libsasl2-2 (= 2.1.28+dfsg-6ubuntu1), libsasl2-dev (= 2.1.28+dfsg-6ubuntu1), libsasl2-modules-db (= 2.1.28+dfsg-6ubuntu1), libseccomp2 (= 2.5.4-1ubuntu1), libselinux1 (= 3.4-1), libsigsegv2 (= 2.13-1ubuntu3), libsmartcols1 (= 2.38-4ubuntu1), libssl3 (= 3.0.3-5ubuntu3), libstdc++-11-dev (= 11.3.0-3ubuntu1), libstdc++6 (= 12.1.0-2ubuntu1), libsub-override-perl (= 0.09-3), libsystemd0 (= 249.11-0ubuntu4), libtinfo6 (= 6.3+20220423-2), libtirpc-common (= 1.3.2-2build1), libtirpc-dev (= 1.3.2-2build1), libtirpc3 (= 1.3.2-2build1), libtool (= 2.4.7-4), libtsan0 (= 11.3.0-3ubuntu1), libubsan1 (= 12.1.0-2ubuntu1), libuchardet0 (= 0.0.7-1build2), libudev1 (= 249.11-0ubuntu4), libunistring2 (= 1.0-1), libuuid1 (= 2.38-4ubuntu1), libxml2 (= 2.9.14+dfsg-1), libzstd1 (= 1.5.2+dfsg-1), linux-libc-dev (= 5.15.0-27.28), login (= 1:4.11.1+dfsg1-2ubuntu1), lsb-base (= 11.1.0ubuntu4), lto-disabled-list (= 27), m4 (= 1.4.18-5ubuntu2), make (= 4.3-4.1build1), man-db (= 2.10.2-1), mawk (= 1.3.4.20200120-3.1), ncurses-base (= 6.3+20220423-2), ncurses-bin (= 6.3+20220423-2), patch (= 2.7.6-7build2), perl (= 5.34.0-3ubuntu1), perl-base (= 5.34.0-3ubuntu1), perl-modules-5.34 (= 5.34.0-3ubuntu1), po-debconf (= 1.0.21+nmu1), protobuf-compiler (= 3.12.4-1ubuntu7), rpcsvc-proto (= 1.4.2-0ubuntu6), sed (= 4.8-1ubuntu2), sensible-utils (= 0.0.17), sysvinit-utils (= 3.01-1ubuntu1), tar (= 1.34+dfsg-1build3), util-linux (= 2.38-4ubuntu1), util-linux-extra (= 2.38-4ubuntu1), xz-utils (= 5.2.5-2.1), zlib1g (= 1:1.2.11.dfsg-2ubuntu9), zlib1g-dev (= 1:1.2.11.dfsg-2ubuntu9) Environment: DEB_BUILD_OPTIONS="noautodbgsym parallel=4" DEB_BUILD_PROFILES="noudeb" LANG="C.UTF-8" LC_ALL="C.UTF-8" SOURCE_DATE_EPOCH="1579418142" +------------------------------------------------------------------------------+ | Package contents | +------------------------------------------------------------------------------+ golang-github-hashicorp-raft-dev_1.1.2-1_all.deb ------------------------------------------------ new Debian package, version 2.0. size 90020 bytes: control archive=1824 bytes. 738 bytes, 15 lines control 4502 bytes, 47 lines md5sums Package: golang-github-hashicorp-raft-dev Source: golang-github-hashicorp-raft Version: 1.1.2-1 Architecture: all Maintainer: Ubuntu Developers Original-Maintainer: Debian Go Packaging Team Installed-Size: 446 Depends: golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~) Section: devel Priority: optional Homepage: https://github.com/hashicorp/raft Description: implementation of the Raft consensus protocol raft is a Go library that manages a replicated log and can be used with an FSM to manage replicated state machines. It is a library for providing consensus. drwxr-xr-x root/root 0 2020-01-19 07:15 ./ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/doc/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/doc/golang-github-hashicorp-raft-dev/ -rw-r--r-- root/root 738 2020-01-19 07:15 ./usr/share/doc/golang-github-hashicorp-raft-dev/changelog.Debian.gz -rw-r--r-- root/root 17485 2019-12-25 07:24 ./usr/share/doc/golang-github-hashicorp-raft-dev/copyright drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/src/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/src/github.com/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/ drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/ -rw-r--r-- root/root 38656 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/api.go drwxr-xr-x root/root 0 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/bench/ -rw-r--r-- root/root 4424 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/bench/bench.go -rw-r--r-- root/root 4437 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/commands.go -rw-r--r-- root/root 3253 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/commitment.go -rw-r--r-- root/root 6005 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/commitment_test.go -rw-r--r-- root/root 12657 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/config.go -rw-r--r-- root/root 12046 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/configuration.go -rw-r--r-- root/root 9192 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/configuration_test.go -rw-r--r-- root/root 2038 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/discard_snapshot.go -rw-r--r-- root/root 426 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/discard_snapshot_test.go -rw-r--r-- root/root 13176 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/file_snapshot.go -rw-r--r-- root/root 7564 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/file_snapshot_test.go -rw-r--r-- root/root 6614 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/fsm.go -rw-r--r-- root/root 7182 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/future.go -rw-r--r-- root/root 923 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/future_test.go -rw-r--r-- root/root 410 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/go.mod -rw-r--r-- root/root 4420 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/go.sum -rw-r--r-- root/root 2783 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/inmem_snapshot.go -rw-r--r-- root/root 4089 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/inmem_snapshot_test.go -rw-r--r-- root/root 2827 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/inmem_store.go -rw-r--r-- root/root 8939 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/inmem_transport.go -rw-r--r-- root/root 2035 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/inmem_transport_test.go -rw-r--r-- root/root 8797 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/integ_test.go -rw-r--r-- root/root 3120 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/log.go -rw-r--r-- root/root 1734 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/log_cache.go -rw-r--r-- root/root 1584 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/log_cache_test.go -rw-r--r-- root/root 19883 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/net_transport.go -rw-r--r-- root/root 19148 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/net_transport_test.go -rw-r--r-- root/root 3813 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/observer.go -rw-r--r-- root/root 2851 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/peersjson.go -rw-r--r-- root/root 3049 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/peersjson_test.go -rw-r--r-- root/root 54544 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/raft.go -rw-r--r-- root/root 59998 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/raft_test.go -rw-r--r-- root/root 18351 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/replication.go -rw-r--r-- root/root 7681 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/snapshot.go -rw-r--r-- root/root 443 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/stable.go -rw-r--r-- root/root 3868 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/state.go -rw-r--r-- root/root 3104 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/tcp_transport.go -rw-r--r-- root/root 569 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/tcp_transport_test.go -rw-r--r-- root/root 21548 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/testing.go -rw-r--r-- root/root 598 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/testing_batch.go -rw-r--r-- root/root 4796 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/transport.go -rw-r--r-- root/root 6932 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/transport_test.go -rw-r--r-- root/root 2972 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/util.go -rw-r--r-- root/root 1883 2020-01-19 07:15 ./usr/share/gocode/src/github.com/hashicorp/raft/util_test.go +------------------------------------------------------------------------------+ | Post Build | +------------------------------------------------------------------------------+ +------------------------------------------------------------------------------+ | Cleanup | +------------------------------------------------------------------------------+ Purging /<> Not removing build depends: as requested +------------------------------------------------------------------------------+ | Summary | +------------------------------------------------------------------------------+ Build Architecture: amd64 Build Type: binary Build-Space: 119308 Build-Time: 54 Distribution: kinetic Host Architecture: amd64 Install-Time: 15 Job: golang-github-hashicorp-raft_1.1.2-1.dsc Machine Architecture: amd64 Package: golang-github-hashicorp-raft Package-Time: 71 Source-Version: 1.1.2-1 Space: 119308 Status: successful Version: 1.1.2-1 -------------------------------------------------------------------------------- Finished at 2022-06-22T14:53:30Z Build needed 00:01:11, 119308k disk space Adding user buildd to group lxd RUN: /usr/share/launchpad-buildd/bin/in-target scan-for-processes --backend=chroot --series=kinetic --arch=amd64 PACKAGEBUILD-24000232 Scanning for processes to kill in build PACKAGEBUILD-24000232