[bionic] samba PANIC, INTERNAL ERROR: Signal 11

Thanks for filing this bug in Ubuntu.

Could you please attach:
- /etc/samba/smb.conf
- the core dump or crash file (look in /var/crash)

So all these samba servers are joined to an AD realm domain, and have shares and whatnot, but when the share on a 18.04 server is accessed you get this crash? Could you please also attach the smb.conf file from one of these client computers that try to access the share on the 18.04 one?

Can you also try to access the 18.04 share with smbclient from:
- itself? smbclient //localhost/share -U somevaliduser
- the same computer where accessing it via nautilus/dolphin triggers the crash: smbclient //18.04server/share -U somevaliduser

Thanks

crash file on 18.04 when accessing smb share with 17.10

smb.conf (Ubuntu 17.10) where smb share is working and not crashing smbd if another client accesses this share.
That 17.10 client for example accesses 18.04 where smbd crashes afterwards.

smb.conf (18.04) where smbd crashes after a client accesses its share

all our clients should have equal or similar smbd settings

Trying to access a share on 18.04 with smbclient from 17.10 lets smbd crash too.
The other way round is working - Accessing a share on 17.10 with 18.04 and smbclient shows me the shared folder content.

The smb.conf file for the 18.04 box shows it as being a standalone server, not a domain member. Is that expected? Are you managing its users locally via smbpasswd?

Was this 18.04 box a fresh install of samba 4.7.6, or did you at some point have 4.7.4 or earlier and upgrade?

Ok, I can reproduce this with a simple "smbclient -L localhost -N" and this smb.conf:

[global]
 dns proxy = No
 domain master = No
 kerberos method = secrets and keytab
 local master = No
 log file = /var/log/samba/log.%m
 map to guest = Bad User
 max log size = 1000
 obey pam restrictions = Yes
 pam password change = Yes
 panic action = /usr/share/samba/panic-action %d
 passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
 passwd program = /usr/bin/passwd %u
 security = USER
 server role = standalone server
 server string = %h %a
 syslog = 0
 unix password sync = Yes
 usershare allow guests = Yes
 idmap config * : backend = tdb

The moment I remove your "kerberos method" option (i.e., comment it), the crash no longer happens.

Can you elaborate on how this 18.04 machine is supposed to authenticate users and give them access or not to a share, since it's not part of the AD realm, at least according to smb.conf? In the meantime I'll check with upstream.

> The smb.conf file for the 18.04 box shows it as being a standalone server, not a domain member. Is that expected? Are you managing its users locally via smbpasswd?

After uploading I noticed that too. No it is not intended. I changed it to

   security = ADS

again and added same settings as in 17.10. Unfortunately smbd is still crashing after accessing the share on 18.04.

> Was this 18.04 box a fresh install of samba 4.7.6, or did you at some point have 4.7.4 or earlier and upgrade?

I upgraded from 16.04 to the development release of 18.04 earlier this year. It is very possible that I had samba 4.7.4 at some point earlier this year.
I have another system with a fresh install of 18.04. smbd also crashes there.

> The moment I remove your "kerberos method" option (i.e., comment it), the crash no longer happens.

Hm, it still keeps crashing for me.
Now I changed smb.conf on 18.04 to this still crashing configuration:

[global]
        dns proxy = No
        domain master = No
        kerberos method = secrets and keytab
        local master = No
        log file = /var/log/samba/log.%m
        map to guest = Bad User
        max log size = 1000
        obey pam restrictions = Yes
        pam password change = Yes
        panic action = /usr/share/samba/panic-action %d
        passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
        passwd program = /usr/bin/passwd %u
        realm = MPI-DORTMUND.MPG.DE
        security = ADS
        server role = member server
        server string = %h %a
        syslog = 0
        unix password sync = Yes
        usershare allow guests = Yes
        workgroup = MPI-DORTMUND
        idmap config * : backend = tdb

> Can you elaborate on how this 18.04 machine is supposed to authenticate users and give them access or not to a share, since it's not part of the AD realm, at least according to smb.conf?

The 18.04 machine should prefer kerberos for authenticating users. Local authentication using sssd for AD is working fine. Kerberos authentication is working fine too.

There is a shared directory users should have access. It is working the other way round - on 17.10 with same settings:

[share]
        create mask = 0640
        directory mask = 0750
        force group = "Domain Users"
        invalid users = root
        path = /mnt/share
        read only = No
        valid users = +ntwsadmins "+Domain Users"

After changing security to ADS, did you join the realm/domain again? You might have some incorrect local databases. Can you start fresh with 4.7.6 on this box?

Also, even on a fresh 4.7.6, I couldn't get "kerberos method = secrets and keytab" to work without crashing, that's the samba bug I filed upstream. I think there is something wrong when it attempts "secrets". I was able to setup a standalone samba server and authenticate to it using plain kerberos (smbclient -k) just fine, but I had to set the dedicated keytab option to /etc/krb5.keytab (which is the system keytab file anyway).

Do you really need to specify "kerberos method"? The default value (not specify it) doesn't work for you case?

The bug in 4.7.4 is only when samba seems to only affect samba when used as a directory controller itself:
o BUG 13228: This is a major issue in Samba's ActiveDirectory domain
   controller code. It might happen that AD objects have missing or broken
   linked attributes. This could lead to broken group memberships e.g.
   All Samba AD domain controllers set up with Samba 4.6 or lower and then
   upgraded to 4.7 are affected. The corrupt database can be fixed with
   'samba-tool dbcheck --cross-ncs --fix'.

Do I really have to rejoin the client to AD after changing samba security to ADS? I'm not using samba "net join" and no winbind for AD binding. I've created the AD machine account with realm and I'm using sssd for authentication to AD DC.
BTW "realm" changed my "security = ADS" in smb.conf to "security = user"....

However, I could reproduce the smb crash anytime when
  security = ADS
is set. It doesn't matter if I specify "kerberos method" or not.

When set to
  security = user
and disabled
  # kerberos method = secrets and keytab
smb is not crashing anymore but I also cannot authenticate with my AD user account (using sssd).

Enabling
  kerberos method = secrets and keytab
and
  security = user
let's smb crash too.

Ok, so to summarize:
- sssd is providing user and groups from AD (via /etc/nsswitch.conf)
- realmd was used to join the machine to AD for the above
- local user authentication is done via pam_sss and using kerberos. Shell users get a ticket upon login
- samba is not using winbind

I have a feeling samba is missing it's account with the AD server. I don't know if the sssd join works for samba's "security = ADS", I have never tested that. I always used net ads join. Is this how you configured the non-18.04 samba member servers? With just sssd, no "net ads join"?

The crash also seems to indicate that the "secrets" bit of "secrets and keytab" is returning a null pointer to the code, so maybe samba isn't finding the secret.

Do you have a populated /etc/krb5.keytab?

Can you try these commands:
net ads testjoin -k
net ads status -k

After having acquired a kerberos ticket most likely (for -k to work).

> Ok, so to summarize:
> - sssd is providing user and groups from AD (via /etc/nsswitch.conf)
> - realmd was used to join the machine to AD for the above
> - local user authentication is done via pam_sss and using kerberos. Shell users get a ticket upon login
> - samba is not using winbind

that's right

> I have a feeling samba is missing it's account with the AD server.

The machine account on the AD server does exist.

> I don't know if the sssd join works for samba's "security = ADS", I have never tested that.

Up to 17.10 it is working using realm to join the client to the AD and smb is working too.

> I always used net ads join. Is this how you configured the non-18.04 samba member servers? With just sssd, no "net ads join"?

Yes, all our clients and servers are not joined to AD by "net ads join". These are all joined by realm and use sssd.

> The crash also seems to indicate that the "secrets" bit of "secrets and keytab" is returning a null pointer to the code, so maybe samba isn't finding the secret.
> Do you have a populated /etc/krb5.keytab?

local /etc/krb5.keytab is generated by realm when AD machine account is created on the server.

> Can you try these commands:
> net ads testjoin -k

Join to domain is not valid: NT code 0xfffffff6

I also get this message on 17.10, where smb is not crashing.

> net ads status -k

objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
objectClass: computer
cn: m15015-vm-lin3
distinguishedName: CN=m15015-vm-lin3,OU=Linux-Clients,OU=Client Computer,OU=alle Computer,DC=mpi-dortmund,DC=mpg,DC=de
instanceType: 4
whenCreated: 20180412075138.0Z
whenChanged: 20180413071746.0Z
uSNCreated: 99733897
uSNChanged: 99802204
name: m15015-vm-lin3
objectGUID: cc30fbce-545d-4dfb-b28c-e973059857a0
userAccountControl: 69632
codePage: 0
countryCode: 0
lastLogon: 131680786856152060
localPolicyFlags: 0
pwdLastSet: 131679930989191696
primaryGroupID: 515
objectSid: S-1-5-21-3772173984-4185860275-536710523-2741741
accountExpires: 9223372036854775807
logonCount: 148
sAMAccountName: m15015-vm-lin3$
sAMAccountType: 805306369
operatingSystem: Ubuntu
operatingSystemVersion: 18.04
dNSHostName: m15015-vm-lin3
userPrincipalName: <email address hidden>
servicePrincipalName: host/m15015-vm-lin3
servicePrincipalName: host/m15015-vm-lin3.client.mpi-dortmund.mpg.de
objectCategory: CN=Computer,CN=Schema,CN=Configuration,DC=mpi-dortmund,DC=mpg,DC=de
isCriticalSystemObject: FALSE
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 131679931011068668
msDS-SupportedEncryptionTypes: 31

What happens in terms of accessing the share in the 18.04 server when you use these settings:

a)
kerberos method = system keytab

b)
kerberos method = dedicated keytab
dedicated keytab file = /etc/krb5.keytab

c) kerberos method = default

a)
  security = ADS
  kerberos method = system keytab

no smb crash, but I cannot authenticate with AD users:
SPNEGO login failed: NT_STATUS_NO_LOGON_SERVERS

b)
  security = ADS
  kerberos method = dedicated keytab
  dedicated keytab file = /etc/krb5.keytab

same as in a)

c)
  security = ADS
  kerberos method = default

smb crashes on access

Ok

The smb.conf(5) manpage does state that for "security = ads" or "server role = member server" to work, the machine must have been joined to the domain via "net ads join". This is what creates the necessary secrets in the local secrets tdb database.

My hypothesis is that there was a change in 4.7.x and that when the secrets are not found, it crashes. Definitely a bug, but we might be in an unsupported configuration. I have yet to hear from upstream in their bug.

Here is what we could try:

a) Samba as a standalone server, but using kerberos for authentication. The users will exist "locally" via sssd, and samba will be just like any other kerberized service authenticating the users via the kdc. For that it will need an appropriate service key in /etc/krb5.keytab. I think realm (the tool) only extracts host/* keys, not cifs/* keys, and samba might want cifs/* ones.
Note that the realm tool does not change smb.conf as far as I can see, that's why you still had "security = user" or "server role = stanalone server" in your smb.conf before. That might be a hint.

Also, we have to be careful in this configuration to use the same username format. SSSD by default likes "<email address hidden>", and samba might expect just "username", or "username@WORKGROUP". That kind of thing.

b) Samba as a normal member server. For this you would have to use "net ads join". I'm not sure if this would require winbind, probably not.

I can try both scenarios in a clean VM, but I'm a bit out of time and can't commit to it just yet. If we can't address this for the release, then an SRU is in order.

I also just tried 4.7.7 quickly and can still reproduce the crash with the minimal smb.conf I showed in the upstream bug at https://bugzilla.samba.org/show_bug.cgi?id=13376.

> a) Samba as a standalone server, but using kerberos for authentication. The users will exist "locally" via sssd, and samba will be just like any other kerberized service authenticating the users via the kdc. For that it will need an appropriate service key in /etc/krb5.keytab. I think realm (the tool) only extracts host/* keys, not cifs/* keys, and samba might want cifs/* ones.

yes, the krb5.keytab created by realm does not contain cifs/* and contains

# klist -e -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (aes256-cts-hmac-sha1-96)
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (aes128-cts-hmac-sha1-96)
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (des3-cbc-sha1)
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (arcfour-hmac)
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (des-cbc-md5)
   2 m15015-vm-lin3$@MPI-DORTMUND.MPG.DE (des-cbc-crc)
   2 <email address hidden> (aes256-cts-hmac-sha1-96)
   2 <email address hidden> (aes128-cts-hmac-sha1-96)
   2 <email address hidden> (des3-cbc-sha1)
   2 <email address hidden> (arcfour-hmac)
   2 <email address hidden> (des-cbc-md5)
   2 <email address hidden> (des-cbc-crc)
   2 <email address hidden> (aes256-cts-hmac-sha1-96)
   2 <email address hidden> (aes128-cts-hmac-sha1-96)
   2 <email address hidden> (des3-cbc-sha1)
   2 <email address hidden> (arcfour-hmac)
   2 <email address hidden> (des-cbc-md5)
   2 <email address hidden> (des-cbc-crc)

But in previous samba version there was no cifs/* in keytab and smb didn't crash on access. So is it really necessary?

> Note that the realm tool does not change smb.conf as far as I can see, that's why you still had "security = user" or "server role = stanalone server" in your smb.conf before. That might be a hint.

Hm, I'm sure it did change the smb.conf previously (maybe this changed recently?). That's why I had "security = user" instead of "security = ADS" in my smb.conf. But now I cannot see any changes in smb.conf too after joining to AD with realm.

So you mean in a) I should try his, right?
  security = auto
  server role = standalone server
  kerberos method = secrets and keytab

smbd crashes here.
What is the best way to add the correct cifs/* in /etc/krb5.keytab?

> SSSD by default likes "<email address hidden>", and samba might expect just "username", or "username@WORKGROUP"

Ok, what is the recommended configuration in sssd.conf and smb.conf?

> b)

So you mean in b) I should try his, right?
  security = auto
  kerberos method = secrets and keytab
  server role = member server
afterwards "net ads join" gives me:

# net ads join -U ntfieroch
Enter ntfieroch's password:
Using short domain name -- MPI-DORTMUND
Joined 'M15015-VM-LIN3' to dns domain 'mpi-dortmund.mpg.de'
DNS Update for m15015-vm-lin3.client.mpi-dortmund.mpg.de fai...

(sorry if I'm telling you something you already know: the text below is also for my own benefit and thought process)

Joining a domain means basically creating a computer account in the AD. That is what allows the computer to query the domain for information like usernames, uid numbers, and even authenticate users.

sssd can do that, for its own benefit. It installs a pam module, a nss module, configures files accordingly, and you get a machine where users can login to the linux system and be treated almost like local users, as if they were in /etc/{passwd,shadow}. sssd can largely do that because of kerberos.

Samba can also join a domain, of course, and it stores the credentials for that locally somewhere. I believe that's ultimately what the "kerberos method" setting controls: if it's in the secrets.tdb database, or in a normal kerberos keytab. I believe when you use "net ads join", it uses secrets.tdb. You can check the /etc/krb5.keytab to see if it changed after you ran "net ads join".

Now, the question is how to take advantage of the already running sssd (for your linux users to login on the box via ssh, login, gdm, etc) for samba. As we know, for samba to authenticate and recognize a windows user, that user also needs to appear as a linux user, as if it existed in /etc/passwd. That's one of the functions of winbind, or nss_ldap, or even sssd. But samba also needs to contact the kerberos server (AD in this case) to authenticate the user and obtain a TGT, and for that it needs to have its own account. An account that sssd created, not "net ads join" in your case. Samba should be able to use the system keytab (that's /etc/krb5.keytab), where apparently sssd did all the work for us, but we are seeing segfaults in our way when messing with that parameter.

In the release notes for samba 4.8.0, for example, they state that having winbind is required for domain membership, because the rpc calls were delegated to it (https://github.com/samba-team/samba/blob/v4-8-stable/WHATSNEW.txt#L24). In 4.7.x that doesn't seem to be the case yet, but maybe they were on that path already.

You have evidence that in previous ubuntu releases it is possible: using only sssd, and having samba authenticate domain users. I don't know if by design, or by accident. Or maybe you are using just a subset of all the possible rpc calls and it works.

I have documentation that says "net ads join" is necessary for this to work (it's in the smb.conf manpage). It doesn't elaborate if winbind is needed, though. Above when you said "it works" after trying "net ads join", did you mean just the join, or that samba started to authenticate domain users normally?

Bottom line is, I don't know if you can use sssd for samba, or if you need both sssd and winbind. I would have to experiment with it. The segfault is a bug, and shouldn't happen even with invalid configurations, so that has to be fixed. But it might be unrelated to the big question.

What I suggest:
- try the net ads join way. It's what the samba documentation recommends
- check if "net ads join" creates another entry in the keytab file
- subscribe to https://lists.samba.org/mailman/listinfo/samba and post this question...

This is https://bugzilla.samba.org/show_bug.cgi?id=13393

Does changing 'secrets and keytab' to 'keytab' help?

I just noticed https://bugzilla.samba.org/show_bug.cgi?id=13376 and closed
https://bugzilla.samba.org/show_bug.cgi?id=13393 again...

The "kerberos method" options that were tried are in https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1761737/comments/16. There is no crash when it's set to "system keytab" or "dedicated keytab" plus pointing the keytab at /etc/krb5.keytab

Can someone try what happens with
https://attachments.samba.org/attachment.cgi?id=14155
together with "kerberos method = secrets and keytab"?

I'd guess it should behave like "system keytab" or "dedicated keytab",
but it would be good to have this verified.

I have it building in a ppa and will try shortly

Packages from https://launchpad.net/~ahasenack/+archive/ubuntu/samba-kerberos-method-1761737 have the patch and fix the crash test case.

After a lot of experimentation, I got my samba server, with "security = ads" but no winbind and no "net ads join" command, to authenticate an AD user using kerberos.

What nailed it was to use setspn on the windows side to add cifs/<hostname> to the computer account, like this (for a "bionic-sssd" computer account):

setspn -S cifs/bionic-sssd bionic-sssd

After that, this worked:
<email address hidden>@bionic-sssd:~$ smbclient //bionic-sssd/myshare -k
WARNING: The "syslog" option is deprecated
Try "help" to get a list of possible commands.
smb: \> dir
  . D 0 Wed Apr 18 20:29:20 2018
  .. D 0 Wed Apr 18 20:50:25 2018
  hello.txt N 13 Wed Apr 18 20:29:20 2018

  7950756 blocks of size 1024. 6300604 blocks available
smb: \> <email address hidden>@bionic-sssd:~$ klist
Ticket cache: FILE:/tmp/krb5cc_45001119_1zpGGU
Default principal: <email address hidden>

Valid starting Expires Service principal
04/18/18 20:51:05 04/19/18 06:51:05 <email address hidden>
 renew until 04/19/18 20:51:05
04/18/18 20:51:49 04/19/18 06:51:05 <email address hidden>

<email address hidden>@bionic-sssd:~$ id
uid=45001119(<email address hidden>) gid=45000513(domain <email address hidden>) groups=45000513(domain <email address hidden>)

<email address hidden>@bionic-sssd:~$ grep testuser /etc/passwd
<email address hidden>@bionic-sssd:~$

My smb.conf has:
[global]
    workgroup = LOWTECH
    realm = LOWTECH.INTERNAL
    kerberos method = system keytab
    server role = member server
    security = ads
...

Ah, and I didn't have to use the updated packages from my ppa, because I set "kerberos method = system keytab", so it wasn't trying "secrets" which is where the crash happens.

At some point I also installed libwbclient-sssd, during the experimentation. I can't say if it was essential now.

> Above when you said "it works" after trying "net ads join", did you mean just the join, or that samba started to authenticate domain users normally?

After additionally trying "net ads join" samba started to authenticate domain users normally. I can access a shared directory with a domain user without smb crash.

> check if "net ads join" creates another entry in the keytab file
Yes, "net ads join" additionally adds cifs/* entries in the keytab file.

I'm asking <email address hidden> if an additional "net ads join" is necessary when joining to AD by realm and use sssd for authentication.

> After a lot of experimentation, I got my samba server, with "security = ads" but no winbind and no "net ads join" command, to authenticate an AD user using kerberos.
> What nailed it was to use setspn on the windows side to add cifs/<hostname> to the computer account, like this (for a "bionic-sssd" computer account):
>
> setspn -S cifs/bionic-sssd bionic-sssd

Same here! It is also working with adding SPN host/ instead of cifs/.

Is there any linux tool that can rpc and create SPNs on the DC?

after adding cifs/ entries on Windows DC to the machine account with setspn there are no cifs/ entries in local keytab file what "net ads join" alternatively has added and samba shares still are accessible.

For the release team: this fixes a crash bug, but in a not very common scenario: domain was joined via sssd and not samba's net join command, and the config tells samba to look first at the secrets database which is only populated via net join.

The MP at https://code.launchpad.net/~ahasenack/ubuntu/+source/samba/+git/samba/+merge/343614 has a simple test case.

There is no need to respin isos because of this, but if it happens for some other reason, it would be cool if this could get in. Otherwise, I can transform it into an SRU for bionic once CC is opened for development.

This bug was fixed in the package samba - 2:4.7.6+dfsg~ubuntu-0ubuntu2

---------------
samba (2:4.7.6+dfsg~ubuntu-0ubuntu2) bionic; urgency=medium

  * debian/patches/passdb_dont_return_ok_if_pinfo_not_filled.patch:
    [PATCH] s3:passdb: Do not return OK if we don't have pinfo filled.
    Thanks to Andreas Schneider <email address hidden>. (LP: #1761737)

 -- Andreas Hasenack <email address hidden> Wed, 18 Apr 2018 11:49:55 -0300