Linux / Unix pv Command: Monitor Progress of Data Sent Via a Pipe

The pv command allows you to see the progress of data through a pipeline. It provides the following info:

Time elapsedPercentage completed (with progress bar)Current throughput rateTotal data transferredETA

From the project home page:

It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

By default pv command is not installed. Type the following apt-get command under Debian / Ubuntu Linux:
# apt-get install pv
Sample outputs:

 Reading package lists... DoneBuilding dependency treeReading state information... DoneSuggested packages: doc-baseThe following NEW packages will be installed: pv0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.Need to get 28.9 kB of archives.After this operation, 143 kB of additional disk space will be used.Get:1 http://mirror.anl.gov/debian/ squeeze/main pv amd64 1.1.4-1 [28.9 kB]Fetched 28.9 kB in 1s (16.1 kB/s)Selecting previously deselected package pv.(Reading database ... 32240 files and directories currently installed.)Unpacking pv (from .../archives/pv_1.1.4-1_amd64.deb) ...Processing triggers for man-db ...Setting up pv (1.1.4-1) . 

RHEL / CentOS / SL / Fedora Linux users, turn on EPEL repo and type the following yum command to install pv:
# yum install pv
Sample outputs:

 Loaded plugins: product-id, protectbase, rhnplugin, subscription-managerUpdating certificate-based repositories.Unable to read consumer identity0 packages excluded due to repository protectionsSetting up Install ProcessResolving Dependencies--> Running transaction check---> Package pv.x86_64 0:1.1.4-3.el6 will be installed--> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size====================================================================================================Installing: pv x86_64 1.1.4-3.el6 epel 34 k Transaction Summary====================================================================================================Install 1 Package(s) Total download size: 34 kInstalled size: 67 kIs this ok [y/N]: yDownloading Packages:pv-1.1.4-3.el6.x86_64.rpm | 34 kB 00:00Running rpm_check_debugRunning Transaction TestTransaction Test SucceededRunning Transaction Installing : pv-1.1.4-3.el6.x86_64 1/1Installed products updated. Verifying : pv-1.1.4-3.el6.x86_64 1/1 Installed: pv.x86_64 0:1.1.4-3.el6 Complete! 

FreeBSD users can use the port to install pv, enter:
# cd /usr/ports/sysutils/pv/
# make install clean
OR add the binary package, run:
# pkg_add -r pv

The syntax is

 pv filenamepv filename > /path/to/outputpv options filename | command1pv options filename | command1 > output.filepv filename | command1command1 | pv | command2pv -options input.file | command1 | pv -options > output.file 

In this example copy a file called origin-cdn.cyberciti.org_access.log to /tmp/origin-cdn-access.log and show progress:
# pv origin-cdn.cyberciti.org_access.log > /tmp/origin-cdn-access.log
OR just send it to /dev/null:
# pv origin-cdn.cyberciti.org_access.log > /dev/null

1.41GB 0:00:21 [66.3MB/s] [=====================================================>] 100%

Use nc to create a network port # 2000. Type the following command:
$ nc -l -v -w 30 -p 2000 > /tmp/data.bin
Open another terminal and type:
$ pv firmware33921.bin | nc -w 1 127.0.0.1 2000
Sample outputs:

Fig.01: pv command in action

In this example you can see progress of both pipes:  pv -cN rawlogfile origin-cdn.cyberciti.org_access.log | gzip | pv -cN gziplogfile > access.log.gz 

Sample outputs:

Fig.02: pv displaying output of complicated pipes and commands

Where, -c : Use cursor positioning escape sequences instead of just using carriage returns. This is useful in conjunction with -N (name) if you are using multiple pv invocations in a single, long, pipeline.N rawlogfile : Prefix the output information with NAME. Useful in conjunction with -c if you have a complicated pipeline and you want to be able to tell different parts of it apart.

You can create a progress bar (progress indicator) when copying/moving/extracting files or making backups using the gauge box:

 echo percentage | dialog --gauge "text" height width percentecho "10" | dialog --gauge "Please wait" 10 70 0echo "50" | dialog --gauge "Please wait" 10 70 0echo "100" | dialog --gauge "Please wait" 10 70 0 

In this example, extract tar ball and show progress using the dialog command:

 (pv -n backup.tar.gz | tar xzf - -C path/to/data ) 2>&1 | dialog --gauge "Running tar, please wait..." 10 70 0 

Sample outputs:

Fig.03: pv and dialog command in action

Finally, you can always use the rsync command to display progress bar for both local and remote files.

You should follow me on twitter here or grab rss feed to keep track of new changes.

View the original article here