Graphite

WSP files not being created

Asked by Colin Small

I am using Icinga2 with the check_wmi_plus plugin. All checks with this plugin work fine. I see the data in the icinga2 classic UI, I see it if I run the commands manually and all of the service checks work and write wsp files to /var/lib/graphite/whisper/icinga/servername/ except for two.

So everything works great between Icinga2, check_wmi_plus and Graphite... except for two of the service checks.

If I use checkmem or checkpage... no wsp files are every create for them. I can't figure out why. The commands work as expected in Icinga2 and also if I manually run them at the command line.. but no wsp files get generated.

Why is this? Where can I look?

These are the only wsp files that are generated for the two services (checkmem & checkpage):

current_attempt.wsp max_check_attempts.wsp state.wsp
execution_time.wsp reachable.wsp
latency.wsp state_type.wsp

All the ones that are working have the proper wsp files created. (checkcpudetail shown as an example)

Avg_Utilisation_CPU0.wsp execution_time.wsp reachable.wsp
Avg_Utilisation_CPU_Total.wsp latency.wsp state_type.wsp
current_attempt.wsp max_check_attempts.wsp state.wsp

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Colin Small
Solved:
Last query:
Last reply:
Revision history for this message
Jason Dixon (jason-dixongroup) said : 		#1

Hi Colin,

I haven't run Icinga2 myself, but it sounds like they've integrated Graphite in the product. That being the case, I don't know what their default setup looks like or how they're transmitting metrics from Icinga to Carbon (or somehow writing directly to Whisper).

At the risk of sounding like I'm passing the buck, I'd suggest asking within the Icinga2 community. Otherwise my suggestions would be the typical "check your logs, sniff your Carbon socket" and troubleshoot along the chain. I'm sorry this isn't more help but I simply have no idea how Icinga2 interacts with Graphite natively.

Thanks,
Jason

Revision history for this message
Colin Small (colin-small) said : 		#2

Thanks Jason,

I've sort of gone down that road already with little success. I'll see what I can do with logs but I'm not even sure what logs to be looking at with so many things in play.

What strikes me as odd is the check plugin data that is passed to Icinga works and all that data is passed to graphite and that works with the exception of two checks. There seems to be some sort of breakdown between Icinga and graphite for just these two particular checks.

I'll keep poking around and if I figure it out I'll post about it.

Revision history for this message
Colin Small (colin-small) said : 		#3

It seems that it could be related to this bug report in Icinga2.

https://dev.icinga.org/issues/6550