At every introduction to a new office environment, I spend (waste?) a certain
period of time to connect to network shares through CIFS/SMB. This was not
different at my new work place either. After spending almost two days to
troubleshoot the problem, I figured out that the problem was due to the
/sbin/request-key binary provided by
keyutils package in Xubuntu
14.04 LTS. So how did I really get it solved?
It is really sick that
cifs kernel module which is responsible for loading
CIFS/SMB filesystems returns
EINVAL (that is,
error(22): Invalid argument)
for an entire family of errors and makes it impossible for the user to spot
the problem. This was also the case for me.
$ sudo mount \ -t cifs //path/to/network/shares /local/mount/point \ -o credentials=$HOME/.smbcredentials,iocharset=utf8,rw,uid=$USER,serverino mount error(22): Invalid argument Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
So given that the UNC is correct, the local mount point does exist and
accessible, and mount options are valid,
invalid argument error does not
provide any context for the problem.
Further searching on the internet leaded me to LinuxCIFS
at Samba Wiki. There it is told that I can enable debug messages for
module by echoing a non-zero value to
/proc/fs/cifs/cifsFYI, hence I did so:
$ echo 1 | sudo tee /proc/fs/cifs/cifsFYI
And tried to mount the network share again. And observed the following snippet
CIFS VFS: dns_resolve_server_name_to_ip: unable to resolve: path CIFS VFS: compose_mount_options: Failed to resolve server part of \\path\to\network\shares to IP
dns_resolve_server_name_to_ip could not resolve
path for some reason.
So how does kernel really resolve domain names to IP addresses? After reading
mount.cifs(8) a couple of times, I
saw See Also section telling that
in the linux kernel source tree may contain additional options and
information. I checked that too and spotted the following lines:
DFS support allows transparent redirection to shares in an MS-DFS name space. In addition, DFS support for target shares which are specified as UNC names which begin with host names (rather than IP addresses) requires a user space helper (such as
cifs.upcall) to be present in order to translate host names to ip address, and the user space helper must also be configured in the file
/etc/request-key.conf. Samba, Windows servers and many NAS appliances support DFS as a way of constructing a global name space to ease network configuration and improve reliability.
To use cifs Kerberos and DFS support, the Linux
keyutilspackage should be installed and something like the following lines should be added to the
create cifs.spnego * * /usr/local/sbin/cifs.upcall %k create dns_resolver * * /usr/local/sbin/cifs.upcall %k
This was getting interesting. I spotted the
dns_resolver keyword at the
bottom and started reading
The DNS resolver module provides a way for kernel services to make DNS queries by way of requesting a key of key type
dns_resolver. These queries are upcalled to userspace through
The long story short, CIFS module employs DNS resolver module, which
internally upcalls the queries to userspace through
/sbin/request-key. I got
further curious and checked the
This program is invoked by the kernel when the kernel is asked for a key that it doesn’t have immediately available. The kernel creates a partially set up key and then calls out to this program to instantiate it. It is not intended to be called directly.
Everything was pointing that something was wrong with my
Ooops! The binary was not there at all.
apt-file search /sbin/request-key
told that the binary was provided by
keyutils package, which was missing in
my system. After installing
keyutils, I did not observe any DNS resolver
issues in the
dmesg logs and
mount.cifs worked without a problem.