Linux basic command collection from YouTube videos

These days, knowing Linux commands is inevitable to learn big data tools. Here I am sharing useful basic Linux commands and some basic stuffs which were taken from YouTube tutorials.

20 Important Linux commands in a nutshell:

Moving page by page or row by row

To get help : man ls = ls –help

Delete file recursively and forcefully

Cat command is read or concatenate files

Nano command is used for Edit text

Ctrl + x -> Exit

Mv used for rename the file as well

File format no matter in Linux 1.txt and 1 are same in Linux

To find / search the file

Remove Directory : sudo rm -r DirectryName

Print or echo command :

The following image shows the echo command used to display JAVA_HOME environmental variable

Download and Unzip files:
The following example show how to download Hadoop and Unzip

Copy file content to another file:
you can use cp command to copy data or content from one file to another
In this example shows how to get copy content from template file in Hadoop

Give permission to all sub folders:
The following example shows that how to grant permission to all subfolder

Change to Root

[1]	The user account added during installation is set an administrative account with Sudo, so it's easy to get root account's shell like follows. trusty@dlp:~$ sudo -s [sudo] password for trusty: # own password root@dlp:~# # switched

[2]

Or it's possible to switch to root account with su command to set root account's password. trusty@dlp:~$ sudo passwd root [sudo] password for trusty: # own password Enter new UNIX password: # set root password Retype new UNIX password: # confirm passwd: password updated successfully trusty@dlp:~$ su - Password: # input root password root@dlp:~# # switched

How to locate directry

locate '*/data'

How to quickly create a file without a context

touch file1.txt

touch file1.txt file2.txt file3.txt file4.txt

Main directories and its purpose

The standard Ubuntu directory structure mostly follows the Filesystem Hierarchy Standard, which can be referred to for more detailed information.

Here, only the most important directories in the system will be presented.

/bin is a place for most commonly used terminal commands, like ls, mount, rm, etc.

/boot contains files needed to start up the system, including the Linux kernel, a RAM disk image and bootloader configuration files.

/dev contains all device files, which are not regular files but instead refer to various hardware devices on the system, including hard drives.

/etc contains system-global configuration files, which affect the system's behavior for all users.

/home home sweet home, this is the place for users' home directories.

/lib contains very important dynamic libraries and kernel modules

/media is intended as a mount point for external devices, such as hard drives or removable media (floppies, CDs, DVDs).

/mnt is also a place for mount points, but dedicated specifically to "temporarily mounted" devices, such as network filesystems.

/opt can be used to store additional software for your system, which is not handled by the package manager.

/proc is a virtual filesystem that provides a mechanism for kernel to send information to processes.

/root is the superuser's home directory, not in /home/ to allow for booting the system even if /home/ is not available.

/run is a tmpfs (temporary file system) available early in the boot process where ephemeral run-time data is stored. Files under this directory are removed or truncated at the beginning of the boot process. 
(It deprecates various legacy locations such as /var/run, /var/lock, /lib/init/rw in otherwise non-ephemeral directory trees as well as/dev/.* and /dev/shm  which are not device files.)

/sbin contains important administrative commands that should generally only be employed by the superuser.

/srv can contain data directories of services such as HTTP (/srv/www/) or FTP.

/sys is a virtual filesystem that can be accessed to set or obtain information about the kernel's view of the system.

/tmp is a place for temporary files used by applications.

/usr contains the majority of user utilities and applications, and partly replicates the root directory structure, containing for instance, among others,/usr/bin/ and /usr/lib.

/var is dedicated to variable data, such as logs, databases, websites, and temporary spool (e-mail etc.) files that persist from one boot to the next. A notable directory it contains is /var/log where system log files are kept.

More details: https://help.ubuntu.com/community/LinuxFilesystemTreeOverview

Cheers:
Uma

How to move a table from one schema to another Schema

In ETL, one of the strategy load data to Staging schema and then move to Live schema.

The following example shows that how to move a table from one schema to another schema.

CREATE SCHEMA Staging;

CREATE SCHEMA Live;

CREATE SCHEMA Tmp;

CREATE TABLE Staging.Department

(

   DepartmentNumber char(10) NOT NULL PRIMARY KEY CLUSTERED,

   DepartmentName varchar(50) NOT NULL,

   ManagerID INT  NULL,

   ParentDepartmentNumber char(10) NULL

);

CREATE TABLE Live.Department

(

   DepartmentNumber char(10) NOT NULL PRIMARY KEY CLUSTERED,

   DepartmentName varchar(50) NOT NULL,

   ManagerID INT  NULL,

   ParentDepartmentNumber char(10) NULL

);

--Switch Live to Tmp

ALTER SCHEMA Tmp TRANSFER Live.Department;

--Switch Staging to Live

ALTER SCHEMA Live TRANSFER Staging.Department;

--Switch Tmp to Staging

ALTER SCHEMA Staging TRANSFER Tmp.Department;

To get rid of risk, we can use this as SP with transaction.

Sample stored procedure for Switch schema with transaction.

CREATE PROCEDURE dbo.SwitchSchema

AS

BEGIN

BEGIN TRANSACTION;

BEGIN TRY

--Switch Live to Tmp

ALTER SCHEMA Tmp TRANSFER Live.Department;

--Switch Staging to Live

ALTER SCHEMA Live TRANSFER Staging.Department;

--Switch Tmp to Staging

ALTER SCHEMA Staging TRANSFER Tmp.Department;

END TRY

BEGIN CATCH

IF @@TRANCOUNT > 0

       ROLLBACK TRANSACTION;

END CATCH;

IF @@TRANCOUNT > 0

   COMMIT TRANSACTION;

END;

Cheers!
Uma