Shell命令学习笔记(1)- linux三剑客

为什么想学shell?

shell命令还是蛮重要的,因为windows、mac、安卓、ios等系统都拥有同一个祖先,Unix。因此,学会shell命令可以通吃服务端、客户端的测试。

最近在学习Linux三剑客,grepawksed。整理了一下学习笔记,方便以后查看。

linux三剑客与SQL的类比

grep  相当于  select table like '%%'  
awk   相当于  select field from table
sed   相当于  update tabele set field = new  where field = old

grep

grep扩展正则{}的用法

{}表示范围限定

[indeyo@VM_0_14_centos indeyo]$ echo '23456' | grep -oE '[2-5]' # 默认只匹配 1个
2
3
4
5
[indeyo@VM_0_14_centos indeyo]$ echo '23456' | grep -oE '[2-5]{1}'
2
3
4
5
[indeyo@VM_0_14_centos indeyo]$ echo '23456' | grep -oE '[2-5]{1,2}'
23
45
[indeyo@VM_0_14_centos indeyo]$ echo '23456' | grep -oE '[2-5]{1,3}' # 最大匹配 3个,剩下不够的就匹配 1个
234
5
[indeyo@VM_0_14_centos indeyo]$ echo '23456' | grep -oE '[2-5]{2,3}' # 最大匹配 3个,剩下的不足以匹配 2个则匹配完成
234

grep[]的用法

[indeyo@VM_0_14_centos ~]$ echo '
> 22
> 23
> 24' | grep -E '(23|24)'
23
24
[indeyo@VM_0_14_centos ~]$ echo '
22
23
24' | grep '2[24]'
22
24
[indeyo@VM_0_14_centos ~]$ echo '
22
23
24' | grep '[24]'
22
23
24

awk

awk原理

awk 'BEGIN{ print "start" } pattern{ commands } END{ print "end" }' file

awk执行顺序

  1. 加载输入流之前,运行BEGIN代码块
  2. 执行pattern代码块
  3. 语句运行完成后,执行END代码块

awk pattern 语法

[86413302@shell.testing-studio.com indeyo]$cat 4.txt
6656
123456
[86413302@shell.testing-studio.com indeyo]$awk '/123/' 4.txt  # 正则用法
123456
[86413302@shell.testing-studio.com indeyo]$cat 8.txt 
hello world
I gona eat korean roast meat .
Thinking of this , I feel excitd.


2390840928905825789

jskfjlsjfldkfjkdkfdjkddddddddddddddddd
fdsf 


[86413302@shell.testing-studio.com indeyo]$awk '/this/,/89/' 8.txt  # 取一个区间
Thinking of this , I feel excitd.


2390840928905825789
[86413302@shell.testing-studio.com indeyo]$awk '$2~/of/' 8.txt  # 某个字段等于某个值
Thinking of this , I feel excitd.
[86413302@shell.testing-studio.com indeyo]$awk 'NR==2' 8.txt  # 取第二行
I gona eat korean roast meat .
[86413302@shell.testing-studio.com indeyo]$awk 'NR>1' 8.txt  # 去掉第一行
I gona eat korean roast meat .
Thinking of this , I feel excitd.


2390840928905825789

jskfjlsjfldkfjkdkfdjkddddddddddddddddd
fdsf 



awk有正则表达的作用,一定程度可以代替grep

[indeyo@VM_0_14_centos indeyo]$ awk '/111/' 1.sh
gogog1111111111abczzzzzzz
11111111sdfsdf222

NR 和 END语句块

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print NR,$0}'
1 /usr/local/bin
2 /usr/bin
3 /usr/local/sbin
4 /usr/sbin
5 /home/86413302/.local/bin
6 /home/86413302/bin
[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}END{print NR}'
6

NF 一般用来取倒数的字段

[indeyo@VM_0_14_centos indeyo]$ echo '1 2    3' | awk '{print $3}'
3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2    3' | awk '{print $NF}' # 相当最后一个字段
3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2    3' | awk '{print $(NF-1)}'  相当于倒数第二个字段
2

NR 表示行序号

[indeyo@VM_0_14_centos indeyo]$ cat 2.txt
dddddddd
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ awk 'NR>1' 2.txt |cat  # 默认输出$0
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ awk 'NR>1{print $0}' 2.txt |cat
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ awk 'NR>2' 2.txt |cat
hello  world

hey


NR 和 NF 的不同

  • NR表示每一行的序号,NF表示这一行的字段数。
  • 计算行数要用 wc -lEND{print NR}也可用来输出行数
[indeyo@VM_0_14_centos indeyo]$ cat 2.txt
dddddddd
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ awk '{print $NR}' 2.txt  # 每一行的行数不同,第一行$1有值,直接打印出来,第二行$2没有值,所以显示空
dddddddd





[indeyo@VM_0_14_centos indeyo]$ awk '{print NR}' 2.txt  # 把每一行行号打印出来
1
2
3
4
5
6
[indeyo@VM_0_14_centos indeyo]$ awk 'NR>1{print $0}' 2.txt
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ sort 2.txt | uniq -c
      2 
      1 dddddddd
      1 hello  world
      1 hey
      1 ttttttttttttttt
[indeyo@VM_0_14_centos indeyo]$ wc -l 2.txt
6 2.txt
[indeyo@VM_0_14_centos indeyo]$ awk '{print $NF}' 2.txt
dddddddd
ttttttttttttttt
world

hey

[indeyo@VM_0_14_centos indeyo]$ awk '{print NF}' 2.txt  # 计算每一行的字段数
1
1
2
0
1
0

awk 单行变多行

[indeyo@VM_0_14_centos ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/indeyo/.local/bin:/home/indeyo/bin
[indeyo@VM_0_14_centos ~]$ echo $PATH | awk 'BEGIN{RS=":"}{print $0}'
/usr/local/bin
/usr/bin
/usr/local/sbin
/usr/sbin
/home/indeyo/.local/bin
/home/indeyo/bin

awk -F 参数,分隔符

[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk -F ' ' '{print $1}'
1
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk -F ' ' '{print $2}'
2
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk -F ' ' '{print $3}'
3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk -F '2' '{print $1}'
1 
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk -F '2' '{print $2}'
 3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk '{print $2}' # 不填分隔符,默认空格分隔
2
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk '{print $1}'
1
[indeyo@VM_0_14_centos indeyo]$ echo '1 2 3' | awk '{print $3}'
3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2       3' | awk '{print $3}' # 无论多少空格都当做分隔符
3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2       3' | awk '{print $0}' # $0表示匹配到的字符串本身
1 2       3

awk OFS 的用法

[indeyo@VM_0_14_centos indeyo]$ echo '1 2    3' | awk '{OFS="-"}{print $1,$2,$3}'
1-2-3
[indeyo@VM_0_14_centos indeyo]$ echo '1 2    3' | awk '{OFS="-";print $1,$2,$3}'
1-2-3

ORS 和 OFS

疑问:为什么不是OFS???

可能是因为打印出来的是很多条记录,OFS无法对多条记录起作用,只对一条记录。(自问自答)

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}' | awk 'BEGIN{OFS=":"}{print $0}'
/usr/local/bin
/usr/bin
/usr/local/sbin
/usr/sbin
/home/86413302/.local/bin
/home/86413302/bin

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}' | awk 'BEGIN{RS="\n";OFS=":"}{print $0}'
/usr/local/bin
/usr/bin
/usr/local/sbin
/usr/sbin
/home/86413302/.local/bin
/home/86413302/bin

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}' | awk 'BEGIN{ORS=":"}{print $0}'
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/86413302/.local/bin:/home/86413302/bin::[86413302@shell.testing-studio.com indeyo]$

awk 字段分割

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}'  
/usr/local/bin
/usr/bin
/usr/local/sbin
/usr/sbin
/home/86413302/.local/bin
/home/86413302/bin

[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}'| awk -F / '{print $1,$2,$3,$4}'
 usr local bin
 usr bin 
 usr local sbin
 usr sbin 
 home 86413302 .local
 home 86413302 bin
   
[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{RS=":"}{print $0}'| awk 'BEGIN{FS="/"}{print $1,$2,$3,$4}'
 usr local bin
 usr bin 
 usr local sbin
 usr sbin 
 home 86413302 .local
 home 86413302 bin
 
[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{FS=":";OFS=" | "}{$1=$1;print $0}'
/usr/local/bin | /usr/bin | /usr/local/sbin | /usr/sbin | /home/86413302/.local/bin | /home/86413302/bin
[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{FS=":";OFS=" | "}{$0=$0;print $0}'
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/86413302/.local/bin:/home/86413302/bin
[86413302@shell.testing-studio.com indeyo]$echo $PATH | awk 'BEGIN{FS=":";OFS=" | "}{$2=$2;print $0}'
/usr/local/bin | /usr/bin | /usr/local/sbin | /usr/sbin | /home/86413302/.local/bin | /home/86413302/bin

$1=$1 是什么意思?和 $0=$0 有什么不同?

看了chinaunix帖子,发现2011那个时候的网络用语很有意思。大家喜欢用“呵呵”,帅旋说那个时候用这个词可是很酷的。还有“浮云”,啊,多有年代感的词啊哈哈。

而且奖品只是鼠标、文化衫和马克杯,但是却吸引了很多大神,想不通。最后一题是写一个五子棋的实现,真的有人写出来,250+行代码,好像只有一个人答了这题。可能真的很爱技术吧?

在帖子中看到一个大神的解释,非常易懂。

http://bbs.chinaunix.net/thread-2319120-1-1.html

命令 作用 使用场景
$1 = $1 重新计算$0 用了OFS,且需输出$0
$0 = $0 重新计算$NNF 改变了FS,且需输出$1NF

实践了一下,确实如此。

[indeyo@VM_0_14_centos indeyo]$ echo 'A B C D' | awk '{OFS="|";print $0;$1=$1;print $0}' # $1=$1 重新计算了 $0
A B C D
A|B|C|D
[indeyo@VM_0_14_centos indeyo]$ echo 'A B C D' | awk 'BEGIN{OFS="|"}{print $0;$1=$1;print $0}'  # BEGIN 对 OFS 不起作用
A B C D
A|B|C|D
[indeyo@VM_0_14_centos indeyo]$ echo 'A|B|C|D' | awk '{FS="|";print $1,NF;$0=$0;print $1,NF}'  # $0=$0 改变了 $1 和 NF
A|B|C|D 1
A 4
[indeyo@VM_0_14_centos indeyo]$ echo 'A|B|C|D' | awk 'BEGIN{FS="|"}{print $1,NF;$0=$0;print $1,NF}'  # 用 BEGIN 可以代替 $0=$0 的效果
A 4
A 4

awk 数据计算

[indeyo@VM_0_14_centos indeyo]$ echo '1,10
> 2,20
> 3,30' | awk 'BEGIN{a=0;FS=","}{a+=$2}END{print a/NR}' # 计算平均值
20
[indeyo@VM_0_14_centos ~]$ echo '
1
2
3
4' | awk '/^..*/{t+=$1;print NR,$1,t,t/NR}' # /^..*/ 过滤掉第一行空白行。这个有点累赘,可以用/^./,表示有任意字符就行,相当于去掉空白行
2 1 1 0.5
3 2 3 1
4 3 6 1.5
5 4 10 2
[86413302@shell.testing-studio.com indeyo]$echo ' # /^./表示有任意字符就行,相当于去掉空白行


1
2
3
4 ' | awk '/^./{print $0}'
1
2
3
4 

sed

sed的主要功能是替换,类似于sql里面的update语句。

sed 的一些常见用法

[indeyo@VM_0_14_centos ~]$ echo '1
> 2
> 3
> 4' | sed -n 1p   # 打印
1
[indeyo@VM_0_14_centos ~]$ echo '1
2
3
4' | sed -n 2p
2
[indeyo@VM_0_14_centos ~]$ echo '1
2
3
4' | sed 's/2/xxxx/'  # 替换
1
xxxx
3
4
[indeyo@VM_0_14_centos ~]$ echo '1
2
3
4' | sed -e 's/2/xxxx/' -e 's&3&33333&'  # 多处替换
1
xxxx
33333
4
[indeyo@VM_0_14_centos ~]$ echo '1
2
3
4' | sed -e 's/2/xxxx/' -e 's#3#3334444#'   # 可以用除了单引号以外的符号作为分隔符
1
xxxx
3334444
4
[indeyo@VM_0_14_centos ~]$ echo 12341234 | sed 's/2/7/'
17341234
[indeyo@VM_0_14_centos ~]$ echo 12341234 | sed 's^2^7^g'
17341734
[indeyo@VM_0_14_centos ~]$ echo 12341234 | sed 's/2/7/g'  # g 表示全部替换
17341734

其他linux命令

  • /^$/ 表示空行

  • which 当前环境变量是否包含这个文件

    [86413302@shell.testing-studio.com indeyo]$which 1.sh
    /usr/bin/which: no 1.sh in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/86413302/.local/bin:/home/86413302/bin)
    [86413302@shell.testing-studio.com indeyo]$which python
    /usr/bin/python
    
  • echo 打印

  • a=1 赋值

  • $PATH, $PWD, $USER, $HOME 预定义变量

  • export path=$PATH:$PWD 添加环境变量

  • $变量名 引用变量

  • 字符串中引用变量,需要用双引号

    [86413302@shell.testing-studio.com indeyo]$a=1
    [86413302@shell.testing-studio.com indeyo]$echo $a
    1
    [86413302@shell.testing-studio.com indeyo]$echo '$a'
    $a
    [86413302@shell.testing-studio.com indeyo]$echo "$a"
    1
    
  • ` 可引用命令

    [86413302@shell.testing-studio.com indeyo]$echo `ll`
    total 124 -rw-rw-r-- 1 86413302 86413302 104 Mar 4 14:37 1 -rwxrwxr-x 1 86413302 86413302 18 Dec 7 22:30 1.sh -rw-rw-r-- 1 86413302 86413302 478 Dec 12 16:40 20191212.txt -rw-rw-r-- 1 86413302 86413302 104 Dec 13 12:30 20191213.txt -rw-rw-r-- 1 86413302 86413302 259 Dec 10 12:41 2.txt lrwxrwxrwx 1 86413302 86413302 5 Dec 8 11:01 3.txt -> 2.txt -rw-rw-r-- 1 86413302 86413302 12 Dec 8 11:35 4.txt -rw-rw-r-- 1 86413302 86413302 147 Dec 13 12:36 8.txt -rw-rw-r-- 1 86413302 86413302 97646 Dec 8 16:23 bing_shell.txt
    [86413302@shell.testing-studio.com indeyo]$ll
    total 124
    -rw-rw-r-- 1 86413302 86413302   104 Mar  4 14:37 1
    -rwxrwxr-x 1 86413302 86413302    18 Dec  7 22:30 1.sh
    -rw-rw-r-- 1 86413302 86413302   478 Dec 12 16:40 20191212.txt
    -rw-rw-r-- 1 86413302 86413302   104 Dec 13 12:30 20191213.txt
    -rw-rw-r-- 1 86413302 86413302   259 Dec 10 12:41 2.txt
    lrwxrwxrwx 1 86413302 86413302     5 Dec  8 11:01 3.txt -> 2.txt
    -rw-rw-r-- 1 86413302 86413302    12 Dec  8 11:35 4.txt
    -rw-rw-r-- 1 86413302 86413302   147 Dec 13 12:36 8.txt
    -rw-rw-r-- 1 86413302 86413302 97646 Dec  8 16:23 bing_shell.txt
    
  • a=(1 2 3 4 5) 数组赋值

  • ${a[下标]}引用数组

  • ${a[*]} , ${a[#]} 输出所有数组元素

  • ${#a[*]} 计算元素个数

    [86413302@shell.testing-studio.com indeyo]$b=(1 2 3)
    [86413302@shell.testing-studio.com indeyo]$echo b
    b
    [86413302@shell.testing-studio.com indeyo]$echo $b
    1
    [86413302@shell.testing-studio.com indeyo]$echo ${b[1]}
    2
    [86413302@shell.testing-studio.com indeyo]$echo ${b[0]}
    1
    [86413302@shell.testing-studio.com indeyo]$echo ${b[2]}
    3
    [86413302@shell.testing-studio.com indeyo]$echo ${b[*]}
    1 2 3
    [86413302@shell.testing-studio.com indeyo]$echo ${#b[*]}
    3
    
  • $?表示上一条语句运行结果,0表示真,非0表示假

  • vim 文件名编辑文件,输入i,insert表示输入状态,ESC一下, :q表示退出,:wq保存并退出

[indeyo@VM_0_14_centos indeyo]$ ll # 查看当前文件列表
total 112
-rw-rw-r-- 1 indeyo indeyo   167 Dec 16 07:21 1.sh
-rw-rw-r-- 1 indeyo indeyo    44 Dec 27 08:28 2.txt
-rw-rw-r-- 1 indeyo indeyo 98808 Dec 11 22:59 shell_bing.txt
[indeyo@VM_0_14_centos indeyo]$ cat 2.txt # 打开一个文件
dddddddd
ttttttttttttttt
hello  world

hey

[indeyo@VM_0_14_centos indeyo]$ vim 2.txt # 编辑文件
[indeyo@VM_0_14_centos indeyo]$ less shell_bing.txt # 页数很多的时候用less查看文件,可翻页

掐头去尾

${string#匹配字符串} 掐头

${string##*匹配字符串} 贪婪匹配掐头

${string%匹配字符串} 去尾

${string%%匹配字符串*} 贪婪匹配去尾

[86413302@izuf60jasqavbxb9efockpz indeyo]$ s="hello testerhome"
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s#h}
ello testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s##*h}
ome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s%terhome}
hello tes
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s%%t*}
hello

不带双引号则默认忽略空格

[86413302@izuf60jasqavbxb9efockpz indeyo]$ s="hello from testerhome"
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s#hello}
from testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo "${s#hello}"
 from testerhome

字符串替换

[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s/testerhome/indeyo}
hello from indeyo
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s/from/to}
hello to testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo ${s//h/xxx}
xxxello from testerxxxome

逻辑判断

[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ 3 -eq 2 ]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ ((3>2));echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ ((3>20));echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ a="testerhome"
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $a
testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ b="hello from testerhome"
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $b
hello from testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ "$a" = "$b" ]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $b
hello from testerhome
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ "$a" = "$b" ]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ "$a" != "$b" ]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ -n "$a" ]  # 非空判断
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ -z "$a" ]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [[ "$b" == h* ]]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [[ "$b" == m* ]]
[86413302@izuf60jasqavbxb9efockpz indeyo]$ echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ 2 -ge 1 ];echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ 2 -le 1 ];echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ 2 -ge 1 -a 3 -ge 4 ];echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ 2 -ge 1 -o 3 -ge 4 ];echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [[ 2 -ge 1 && 3 -ge 4 ]];echo $?
1
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [[ 2 -ge 1 || 3 -ge 4 ]];echo $?
0
[86413302@izuf60jasqavbxb9efockpz indeyo]$ [ ! 2 -ge 1 ];echo $?

cp 移动文件

在文件目录下,执行cp 文件名 目标目录

[86413302@shell.testing-studio.com tmp]$cp nginx.log /home/86413302
[86413302@shell.testing-studio.com tmp]$cd /home/86413302
[86413302@shell.testing-studio.com ~]$ll
total 556
-rw-rw-r-- 1 86413302 86413302      0 Dec 10 07:34 b
-rw-rw-r-- 1 86413302 86413302 135583 Dec 10 07:25 bing_shell.txt
drwxrwxr-x 2 86413302 86413302   4096 Dec 13 12:36 indeyo
-rw-r--r-- 1 86413302 86413302 424573 Feb  4 09:55 nginx.log

sort 和 uniq

统计每条记录的数量,先用sort聚合,再用uniq -c统计,要用sort 待计量数据| uniq -c

[indeyo@VM_0_14_centos indeyo]$ cat 1.sh
gogog1111111111abczzzzzzz
q
11111111sdfsdf222

q
q
q


Today is Wednessday.
I wake up at 6.20.
I prepare for the soap.
It must be delicious.


2019-12-11
2019-12-12

[indeyo@VM_0_14_centos indeyo]$ sort 1.sh | uniq -c
      5 
      1  
      1 11111111sdfsdf222
      1 2019-12-11
      1 2019-12-12
      1 gogog1111111111abczzzzzzz
      1 I prepare for the soap.
      1 It must be delicious.
      1 I wake up at 6.20.
      4 q
      1 Today is Wednessday.

问题清单

这个清单是学习过程中的灵魂发问。有些问题也许不能立马就有答案,后面熟悉以后,有些是自己解答的。有种穿越过去回答问题的感觉,哈哈。

  1. BEGIN到底是啥用法?不明白

    awk加载输入流文件之前,运行的代码块,一般用于变量初始化、打印表格表头之类的。

    [86413302@shell.testing-studio.com indeyo]$awk 'BEGIN{ print "begin"}{ print } END{ print "end"}' 4.txt
    begin
    6656
    123456
    end
    
  2. fieldrecord 到底是哪个层面区分的?什么情况下是record

    一行数据表示record记录,一行数据里面的字段是field。通过正则取数据,取出来的就是一个record。通过awk的$2取出第二个field

    [indeyo@VM_0_14_centos indeyo]$ cat 2.txt
    dddddddd
    ttttttttttttttt
    hello  world
    
    hey
    
    [indeyo@VM_0_14_centos indeyo]$ grep hello 2.txt
    hello  world
    [indeyo@VM_0_14_centos indeyo]$ grep hello 2.txt | awk '{print $2}'
    world
    [indeyo@VM_0_14_centos indeyo]$ grep hello 2.txt | awk '{print $1}'
    hello
    [indeyo@VM_0_14_centos indeyo]$ grep hello 2.txt | awk '{print $0}'
    hello  world
    
  3. (GNU grep) 3.1上面,grep的贪婪匹配不能工作的,为啥?

    grep -V查看版本号。可能是操作系统版本不支持。

    ?贪婪匹配在GNU2.20版本上,该功能无效,mac2.50上是可以的。

    [indeyo@VM_0_14_centos indeyo]$ grep -V
    grep (GNU grep) 2.20
    Copyright (C) 2014 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
    
  4. 为什么[^"]*可以代表任意字符串?

    这里的^表示非运算,非双引号的字符

  5. grep命令,如果不用-E,就不能用扩展正则表达式了吗?

    dei!

学习心得

作为小白来说,入门可能不那么容易。尤其是那些晦涩难记的语法格式,如果只是练习一次,随着时间递增,能记忆的部分只会线性递减。

刚学没多久的语法,上手做题的时候硬是想不出来,特别受挫。

感谢春节延长多出来的假期,有大段大段完整的时间把语法捡起来。命令多敲几遍,慢慢开始理解,之前的疑难杂症也解开了。

所以,除了多练习,没有太多的捷径。

当然,如果要记得更牢,还是尽量运用到工作中。

再啰嗦一点,要学会用man 命令名查看说明文档。英文看不懂,Google翻译安排一下。不过,求助翻译之前,最好自己先尝试翻译一下,毕竟不会看英文文档的测试人员不是好测开。

学习链接

推荐原因:awk语句执行顺序讲得很清楚,有较全的语法整理,方面查找

5 Likes

写的真全

可以放院内分享节点呀,要不然容易沉

关闭