Resolve failed tests

Asked by Vindicator

My ideal is that every test passes.
I know there are tests that WILL fail because the hardware doesn't exist (firewire).
There are tests that fail that I think SHOULD pass.

It began with my 5 year old system that I just transitioned to Ubuntu from Vista. There were other issues that made me pull the trigger and upgrade my system from H55 to H97.
With a fresh install and working through my notes for setting it up, I ran System Testing, perused the results and saw failures mentioned due to packages not being installed, so I updated my notes, added the packages (sudo apt-get install fwts python-opencv checkbox wmctrl obexftp fswebcam iperf) and reran the test. (looks like I missed the mesa-utils package so will rerun the test again)

There was 1 test that was causing the entire process to "hang", except that I could cancel it. "Test FSV screenshot for 4th Generation Core Processor Family Integrated Graphics Controller"

Failed:
cpu/clocktest
*****
Testing for clock jitter on 4 cpus sched_setaffinity: Invalid argument
*****
//"Invalid argument" looks to be an issue in the code.

Test FSV screenshot for 4th Generation Core Processor Family Integrated Graphics Controller
//Causes the testing to hang. I let it go longer than the memory-check test before I cancel it.

Test that a screenshot is valid, automatically
*****
ERROR: video source not found: /dev/external_webcam
*****
//I have no webcam installed. I have a USB TV tuner (auvitek/mygica/geniatech) attached, but I'd have to manually build a module for that.

mediacard/sd-preinserted
*****
ERROR:root:Unable to locate drive associated with /org/freedesktop/UDisks2/block_devices/dm_2d1
ERROR:root:No removable drives were detected, aborting
*****
//I doubt this is really an issue. It sounds like it's saying I didn't have the sdcard already inserted when starting the testing.

mediacard/sdhc-insert
*****
You have failed to perform the required manipulation in time
*****
//Not an issue with the testing I don't think since the card isn't being read whatsoever for some reason. It was fine in Vista and when I used Ubuntu in a virtual machine with Vista as the host.

miscellanea/fwts_test
*****
High Failures: 1
WARNING: The following test cases were reported as high
level failures by fwts.
 - uefirtvariable
Medium Failures: 2
WARNING: The following test cases were reported as medium
level failures by fwts.
 - method
 - aspm
*****
// uefirtvariable:
"FAILED [HIGH] UEFIRuntimeGetNextVariableName: Test 2, Expected EFI_BUFFER_TOO_SMALL with small VariableNameSize. Return status: EFI_INVALID_PARAMETER. A parameter was incorrect."
//method:
"FAILED [MEDIUM] MethodHIDInvalidInteger: Test 8, \NFC_._HID returned a integer 0x00000000 (EISA ID @@@0000) but the this is not a valid EISA ID encoded PNP ID."
//aspm:
"FAILED [MEDIUM] PCIEASPM_Unmatched: Test 2, PCIe ASPM setting was not matched. RP 00h:1Ch.07h has ASPM = 00h. Device 03h:00h.00h has ASPM = 03h."

networking/http
*****
http://: Invalid host name.
*****
//Looks like a coding issue

suspend/audio_before_suspend
*****
Traceback (most recent call last):
  File "/tmp/nest-bqpisetd.53da2f0d5daa59cf2d6aa81031bd900071c971e89e30e8d5dd45c710404c5805/audio_settings", line 9, in
    sys.exit(main())
  File "/usr/lib/python3/dist-packages/checkbox_support/scripts/audio_settings.py", line 363, in main
    store_audio_settings(settings_file)
  File "/usr/lib/python3/dist-packages/checkbox_support/scripts/audio_settings.py", line 217, in store_audio_settings
    default = default_regex.search(pactl_status).group()
AttributeError: 'NoneType' object has no attribute 'group'
*****
//Looks like a coding issue

suspend/bluetooth_obex_browse_after_suspend_auto
*****
btdevaddr option not set to device address of Bluetooth target in checkbox.ini
*****
//Same for all similar obex tests. Looks like something the user is supposed to set?

suspend/wireless_before_suspend
*****
Usage: nmcli connection { COMMAND | help }

COMMAND := { show | up | down | add | modify | edit | delete | reload | load }

  show [--active] [[id | uuid | path | apath] ] ...

  up [[id | uuid | path] ] [ifname ] [ap ]

  down [id | uuid | path | apath]

  add COMMON_OPTIONS TYPE_SPECIFIC_OPTIONS IP_OPTIONS

  modify [--temporary] [id | uuid | path] ([+|-]. )+

  edit [id | uuid | path]
  edit [type ] [con-name ]

  delete [id | uuid | path]

  reload

  load [ ... ]

Error: 'status' is not valid 'connection' command.
*****
//Looks like coding issue

Not supported:
suspend/wireless_connection_after_suspend_open_ac_auto
*****
job cannot be started: resource expression "environment.ROUTERS == 'multiple'" evaluates to false
*****
//Same for all similar wireless tests. Looks like something that SHOULD be supported.

Side notes:
There is an audio check that BLASTS very loudly. It is an automatic test and not an interactive one which behaves correctly.
My SDHC is not being read, but that is unrelated to the test.

Question information

Language:
English Edit question
Status:
Expired
For:
Checkbox Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Po-Hsu Lin (cypressyew) said :
#1

Hello,

I would like to suggest you to test it with the latest version of Checkbox:
https://launchpad.net/~checkbox-dev/+archive/ubuntu/ppa
It will be easier to exclude some potential coding issues.

Also, you might need to setup your test environment [1] to get over the "environment.ROUTERS == 'multiple'" issue for your wireless / bluetooth tests.

[1] http://certification-static.canonical.com/docs/guides/14.04/Ubuntu_Desktop_Certified_Hardware_Self-Testing_Guide.pdf

Revision history for this message
Vindicator (vindicator) said :
#2

Well that didn't work at all:
[May29 14:47] checkbox-gui[8358]: segfault at 0 ip 00007f6b856b2192 sp 00007ffe163447d0 error 4 in libgui-engine.so[7f6b85699000+52000]

I only have Default testing selected, click OK and the window disappears and that error is in dmesg.

Good to have that pdf to refer to for customization. I went ahead and created that .ini.

Revision history for this message
Vindicator (vindicator) said :
#3

The screenshot hang still occurs.
Bluetooth:
  /usr/share/checkbox/configs/checkbox.ini
   [checkbox/plugins/environment_info]
   btdevaddr = <hciconfig result>
   [checkbox/plugins/environment_info]
   routers = single
   router_ssid = <ssid of your router>
   router_psk = <psk used for wpa2 on your router>
 Bluetooth obex tests still fail with the same error as before. I had even tested it with "$ obexftpd -c ~/Downloads -bv" which worked.

Revision history for this message
Po-Hsu Lin (cypressyew) said :
#4

Hi there,
if certain case still fail for you, I would suggest you to turn it into a bug report here:
https://bugs.launchpad.net/checkbox-project

It will be easier to track and fix.

When opening a bug, please run this script: http://pastebin.ubuntu.com/11578990/ to get some basic information for your checkbox. (You might be asked to install the plainbox tool)

Revision history for this message
Launchpad Janitor (janitor) said :
#5

This question was expired because it remained in the 'Open' state without activity for the last 15 days.