Basic Filesystem operations

The information returned is very similar to that returned by the Unix command ls l, with a few minor differences.

The first column shows the file mode. The secord column is the replication factor of the file (something a traditional Unix filesystem do not have). Remember we set the default replication factor in the site-wide configuration.

File Permissions in HDFS

HDFS has a permissions model for files and directories that is much like the POSIX model.

There are three types of permission: the read permission (r), the write permission (w), and the execute permission (x).

The read permission is required to read files or list the contents of a directory.

The write permission is required to write a file or, for a directory, to create or delete files or directories in it.

The execute permission is ignored for a file because you can’t execute a file on HDFS (unlike POSIX), and for a directory this permission is required to access its children.

Each file and directory has an owner, a group, and a mode.

The mode is made up of the permissions for the user who is the owner, the permissions for the users who are members of the group, and the permissions for users who are neither the owners nor members of the group.

By default, Hadoop runs with security disabled, which means that a client’s identity is not authenticated. Because clients are remote, it is possible for a client to become an arbitrary user simply by creating an account of that name on the remote system.

This is not possible if security is turned on; see “Security” on page 309. Either way, it is worth- while having permissions enabled (as they are by default; see the dfs.permis sions.enabled property) to avoid accidental modification or deletion of substantial parts of the filesystem, either by users or by automated tools or programs.