PXC - Node Eviction Failure

Asked by ilanchezhian murugan

Environment - Server version: 5.6.28-76.1-56-log Percona XtraDB Cluster (GPL), Release rel76.1
Cluster - 6 nodes
Issue - We had incident where 2 out of 6 nodes had storage issues and the two trouble node was not evicted completely so we had scenario where the entire cluster is stuck and not accepting any more write request. Errors like below.

user: 'mysql' host: 'localhost' (Lock wait timeout exceeded; try restarting transaction)
user: 'mysql' host: 'localhost' (Lock wait timeout exceeded; try restarting transaction)

So how do we prevent such scenario in galera cluster as any cluster responsibility the nodes which are having issues should be evicted rather causing the entire cluster to stop accepting request from end users.

Should this be consider as enhancement in galera cluster or is there an setting which we are missing could have prevented such scenarios.

It would be great if we can get response on this. Thank you.

Question information

Language:
English Edit question
Status:
Expired
For:
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Launchpad Janitor (janitor) said :
#1

This question was expired because it remained in the 'Open' state without activity for the last 15 days.