PoshCode Archive  Hex Artifact Content

Artifact 806fca1c45e32086645d7563affac48b9890610803dcda178b7e4f7993993f62:

  • File finddupe.ps1 — part of check-in [e4e92dbecf] at 2018-06-10 13:13:13 on branch trunk — Find duplicates in <directories/files> comparing size then SHA-512 hash of files. Usage: finddupe.ps1 <directory/file #1> <directory/file #2> ... <directory/file #N> [-delete] [-noprompt] [-recurse] ; -delete will prompt to delete duplicates. -delete and -noprompt together will delete duplicates without prompting. -recurse looks in all subdirectories of all listed directories. The first file in a duplicate pair will not be deleted ever. (user: James Gentile size: 6121)

0000: ef bb bf 23 20 65 6e 63 6f 64 69 6e 67 3a 20 75  ...# encoding: u
0010: 74 66 2d 38 0d 0a 23 20 61 70 69 3a 20 70 6f 77  tf-8..# api: pow
0020: 65 72 73 68 65 6c 6c 0d 0a 23 20 74 69 74 6c 65  ershell..# title
0030: 3a 20 66 69 6e 64 64 75 70 65 2e 70 73 31 0d 0a  : finddupe.ps1..
0040: 23 20 64 65 73 63 72 69 70 74 69 6f 6e 3a 20 46  # description: F
0050: 69 6e 64 20 64 75 70 6c 69 63 61 74 65 73 20 69  ind duplicates i
0060: 6e 20 3c 64 69 72 65 63 74 6f 72 69 65 73 2f 66  n <directories/f
0070: 69 6c 65 73 3e 20 63 6f 6d 70 61 72 69 6e 67 20  iles> comparing 
0080: 73 69 7a 65 20 74 68 65 6e 20 53 48 41 2d 35 31  size then SHA-51
0090: 32 20 68 61 73 68 20 6f 66 20 66 69 6c 65 73 2e  2 hash of files.
00a0: 20 55 73 61 67 65 3a 20 66 69 6e 64 64 75 70 65   Usage: finddupe
00b0: 2e 70 73 31 20 3c 64 69 72 65 63 74 6f 72 79 2f  .ps1 <directory/
00c0: 66 69 6c 65 20 23 31 3e 20 3c 64 69 72 65 63 74  file #1> <direct
00d0: 6f 72 79 2f 66 69 6c 65 20 23 32 3e 20 2e 2e 2e  ory/file #2> ...
00e0: 20 3c 64 69 72 65 63 74 6f 72 79 2f 66 69 6c 65   <directory/file
00f0: 20 23 4e 3e 20 5b 2d 64 65 6c 65 74 65 5d 20 5b   #N> [-delete] [
0100: 2d 6e 6f 70 72 6f 6d 70 74 5d 20 5b 2d 72 65 63  -noprompt] [-rec
0110: 75 72 73 65 5d 20 3b 20 2d 64 65 6c 65 74 65 20  urse] ; -delete 
0120: 77 69 6c 6c 20 70 72 6f 6d 70 74 20 74 6f 20 64  will prompt to d
0130: 65 6c 65 74 65 20 64 75 70 6c 69 63 61 74 65 73  elete duplicates
0140: 2e 20 2d 64 65 6c 65 74 65 20 61 6e 64 20 2d 6e  . -delete and -n
0150: 6f 70 72 6f 6d 70 74 20 74 6f 67 65 74 68 65 72  oprompt together
0160: 20 77 69 6c 6c 20 64 65 6c 65 74 65 20 64 75 70   will delete dup
0170: 6c 69 63 61 74 65 73 20 77 69 74 68 6f 75 74 20  licates without 
0180: 70 72 6f 6d 70 74 69 6e 67 2e 20 2d 72 65 63 75  prompting. -recu
0190: 72 73 65 20 6c 6f 6f 6b 73 20 69 6e 20 61 6c 6c  rse looks in all
01a0: 20 73 75 62 64 69 72 65 63 74 6f 72 69 65 73 20   subdirectories 
01b0: 6f 66 20 61 6c 6c 20 6c 69 73 74 65 64 20 64 69  of all listed di
01c0: 72 65 63 74 6f 72 69 65 73 2e 20 54 68 65 20 66  rectories. The f
01d0: 69 72 73 74 20 66 69 6c 65 20 69 6e 20 61 20 64  irst file in a d
01e0: 75 70 6c 69 63 61 74 65 20 70 61 69 72 20 77 69  uplicate pair wi
01f0: 6c 6c 20 6e 6f 74 20 62 65 20 64 65 6c 65 74 65  ll not be delete
0200: 64 20 65 76 65 72 2e 0d 0a 23 20 76 65 72 73 69  d ever...# versi
0210: 6f 6e 3a 20 30 2e 31 0d 0a 23 20 74 79 70 65 3a  on: 0.1..# type:
0220: 20 66 75 6e 63 74 69 6f 6e 0d 0a 23 20 61 75 74   function..# aut
0230: 68 6f 72 3a 20 4a 61 6d 65 73 20 47 65 6e 74 69  hor: James Genti
0240: 6c 65 0d 0a 23 20 6c 69 63 65 6e 73 65 3a 20 43  le..# license: C
0250: 43 30 0d 0a 23 20 66 75 6e 63 74 69 6f 6e 3a 20  C0..# function: 
0260: 47 65 74 2d 53 48 41 35 31 32 0d 0a 23 20 78 2d  Get-SHA512..# x-
0270: 70 6f 73 68 63 6f 64 65 2d 69 64 3a 20 32 36 31  poshcode-id: 261
0280: 37 0d 0a 23 20 78 2d 64 65 72 69 76 65 64 2d 66  7..# x-derived-f
0290: 72 6f 6d 2d 69 64 3a 20 33 33 32 38 0d 0a 23 20  rom-id: 3328..# 
02a0: 78 2d 61 72 63 68 69 76 65 64 3a 20 32 30 31 37  x-archived: 2017
02b0: 2d 30 34 2d 33 30 54 31 32 3a 33 38 3a 30 34 0d  -04-30T12:38:04.
02c0: 0a 23 20 78 2d 70 75 62 6c 69 73 68 65 64 3a 20  .# x-published: 
02d0: 32 30 31 31 2d 30 34 2d 31 36 54 31 31 3a 35 35  2011-04-16T11:55
02e0: 3a 30 30 0d 0a 23 0d 0a 23 0d 0a 66 75 6e 63 74  :00..#..#..funct
02f0: 69 6f 6e 20 47 65 74 2d 53 48 41 35 31 32 28 5b  ion Get-SHA512([
0300: 53 79 73 74 65 6d 2e 49 4f 2e 46 69 6c 65 49 6e  System.IO.FileIn
0310: 66 6f 5d 20 24 66 69 6c 65 20 3d 20 24 28 74 68  fo] $file = $(th
0320: 72 6f 77 20 27 55 73 61 67 65 3a 20 47 65 74 2d  row 'Usage: Get-
0330: 4d 44 35 20 5b 53 79 73 74 65 6d 2e 49 4f 2e 46  MD5 [System.IO.F
0340: 69 6c 65 49 6e 66 6f 5d 27 29 29 0d 0a 7b 0d 0a  ileInfo]'))..{..
0350: 20 20 09 24 73 74 72 65 61 6d 20 3d 20 24 6e 75    .$stream = $nu
0360: 6c 6c 3b 0d 0a 20 20 09 24 63 72 79 70 74 6f 53  ll;..  .$cryptoS
0370: 65 72 76 69 63 65 50 72 6f 76 69 64 65 72 20 3d  erviceProvider =
0380: 20 5b 53 79 73 74 65 6d 2e 53 65 63 75 72 69 74   [System.Securit
0390: 79 2e 43 72 79 70 74 6f 67 72 61 70 68 79 2e 53  y.Cryptography.S
03a0: 48 41 35 31 32 43 72 79 70 74 6f 53 65 72 76 69  HA512CryptoServi
03b0: 63 65 50 72 6f 76 69 64 65 72 5d 3b 0d 0a 20 20  ceProvider];..  
03c0: 09 24 68 61 73 68 41 6c 67 6f 72 69 74 68 6d 20  .$hashAlgorithm 
03d0: 3d 20 6e 65 77 2d 6f 62 6a 65 63 74 20 24 63 72  = new-object $cr
03e0: 79 70 74 6f 53 65 72 76 69 63 65 50 72 6f 76 69  yptoServiceProvi
03f0: 64 65 72 0d 0a 20 20 09 24 73 74 72 65 61 6d 20  der..  .$stream 
0400: 3d 20 24 66 69 6c 65 2e 4f 70 65 6e 52 65 61 64  = $file.OpenRead
0410: 28 29 3b 0d 0a 20 20 09 24 68 61 73 68 42 79 74  ();..  .$hashByt
0420: 65 41 72 72 61 79 20 3d 20 24 68 61 73 68 41 6c  eArray = $hashAl
0430: 67 6f 72 69 74 68 6d 2e 43 6f 6d 70 75 74 65 48  gorithm.ComputeH
0440: 61 73 68 28 24 73 74 72 65 61 6d 29 3b 0d 0a 20  ash($stream);.. 
0450: 20 09 24 73 74 72 65 61 6d 2e 43 6c 6f 73 65 28   .$stream.Close(
0460: 29 3b 0d 0a 0d 0a 20 20 09 23 23 20 57 65 20 68  );....  .## We h
0470: 61 76 65 20 74 6f 20 62 65 20 73 75 72 65 20 74  ave to be sure t
0480: 68 61 74 20 77 65 20 63 6c 6f 73 65 20 74 68 65  hat we close the
0490: 20 66 69 6c 65 20 73 74 72 65 61 6d 20 69 66 20   file stream if 
04a0: 61 6e 79 20 65 78 63 65 70 74 69 6f 6e 73 20 61  any exceptions a
04b0: 72 65 20 74 68 72 6f 77 6e 2e 0d 0a 0d 0a 20 20  re thrown.....  
04c0: 09 74 72 61 70 0d 0a 20 20 09 7b 0d 0a 20 20 20  .trap..  .{..   
04d0: 09 09 69 66 20 28 24 73 74 72 65 61 6d 20 2d 6e  ..if ($stream -n
04e0: 65 20 24 6e 75 6c 6c 29 0d 0a 20 20 20 20 09 09  e $null)..    ..
04f0: 7b 0d 0a 09 09 09 24 73 74 72 65 61 6d 2e 43 6c  {.....$stream.Cl
0500: 6f 73 65 28 29 3b 0d 0a 09 09 7d 0d 0a 20 20 09  ose();....}..  .
0510: 09 62 72 65 61 6b 3b 0d 0a 09 7d 09 0d 0a 0d 0a  .break;...}.....
0520: 20 09 66 6f 72 65 61 63 68 20 28 24 62 79 74 65   .foreach ($byte
0530: 20 69 6e 20 24 68 61 73 68 42 79 74 65 41 72 72   in $hashByteArr
0540: 61 79 29 20 7b 20 69 66 20 28 24 62 79 74 65 20  ay) { if ($byte 
0550: 2d 6c 74 20 31 36 29 20 7b 24 72 65 73 75 6c 74  -lt 16) {$result
0560: 20 2b 3d 20 93 30 7b 30 3a 58 7d 94 20 2d 66 20   += .0{0:X}. -f 
0570: 24 62 79 74 65 20 7d 20 65 6c 73 65 20 7b 20 24  $byte } else { $
0580: 72 65 73 75 6c 74 20 2b 3d 20 93 7b 30 3a 58 7d  result += .{0:X}
0590: 94 20 2d 66 20 24 62 79 74 65 20 7d 7d 0d 0a 09  . -f $byte }}...
05a0: 72 65 74 75 72 6e 20 5b 73 74 72 69 6e 67 5d 24  return [string]$
05b0: 72 65 73 75 6c 74 3b 0d 0a 7d 0d 0a 0d 0a 24 73  result;..}....$s
05c0: 74 61 72 74 74 69 6d 65 3d 5b 64 61 74 65 74 69  tarttime=[dateti
05d0: 6d 65 5d 3a 3a 6e 6f 77 0d 0a 0d 0a 77 72 69 74  me]::now....writ
05e0: 65 2d 68 6f 73 74 20 22 46 69 6e 64 44 75 70 65  e-host "FindDupe
05f0: 2e 70 73 31 20 2d 20 66 69 6e 64 20 61 6e 64 20  .ps1 - find and 
0600: 6f 70 74 69 6f 6e 61 6c 6c 79 20 64 65 6c 65 74  optionally delet
0610: 65 20 64 75 70 6c 69 63 61 74 65 73 2e 20 46 69  e duplicates. Fi
0620: 6e 64 44 75 70 65 2e 70 73 31 20 2d 68 65 6c 70  ndDupe.ps1 -help
0630: 20 6f 72 20 46 69 6e 64 44 75 70 65 2e 70 73 31   or FindDupe.ps1
0640: 20 2d 68 20 66 6f 72 20 75 73 61 67 65 20 6f 70   -h for usage op
0650: 74 69 6f 6e 73 2e 22 0d 0a 0d 0a 24 6d 61 74 63  tions."....$matc
0660: 68 65 73 20 3d 20 30 20 20 20 20 20 09 23 20 69  hes = 0     .# i
0670: 6e 69 74 69 61 6c 69 7a 65 20 6e 75 6d 62 65 72  nitialize number
0680: 20 6f 66 20 6d 61 74 63 68 65 73 20 66 6f 72 20   of matches for 
0690: 73 75 6d 6d 61 72 79 2e 0d 0a 24 66 69 6c 65 73  summary...$files
06a0: 64 65 6c 65 74 65 64 20 3d 20 30 20 09 23 20 6e  deleted = 0 .# n
06b0: 75 6d 62 65 72 20 6f 66 20 66 69 6c 65 73 20 64  umber of files d
06c0: 65 6c 65 74 65 64 2e 0d 0a 24 62 79 74 65 73 72  eleted...$bytesr
06d0: 65 63 20 3d 20 30 20 09 09 23 20 4e 75 6d 62 65  ec = 0 ..# Numbe
06e0: 72 20 6f 66 20 62 79 74 65 73 20 72 65 63 6f 76  r of bytes recov
06f0: 65 72 65 64 2e 0d 0a 0d 0a 0d 0a 69 66 20 28 24  ered.......if ($
0700: 61 72 67 73 20 2d 65 71 20 22 2d 68 65 6c 70 22  args -eq "-help"
0710: 20 2d 6f 72 20 24 61 72 67 73 20 2d 65 71 20 22   -or $args -eq "
0720: 2d 68 22 29 20 23 20 63 68 65 63 6b 20 66 6f 72  -h") # check for
0730: 20 68 65 6c 70 20 72 65 71 75 65 73 74 2c 20 69   help request, i
0740: 66 20 66 6f 75 6e 64 20 64 69 73 70 6c 61 79 20  f found display 
0750: 75 73 61 67 65 20 6f 70 74 69 6f 6e 73 2e 2e 2e  usage options...
0760: 0d 0a 7b 0d 0a 09 22 22 0d 0a 09 22 55 73 61 67  ..{...""..."Usag
0770: 65 3a 22 0d 0a 09 22 20 20 20 20 20 20 20 50 53  e:"..."       PS
0780: 3e 2e 5c 46 69 6e 64 44 75 70 65 2e 70 73 31 20  >.\FindDupe.ps1 
0790: 3c 64 69 72 65 63 74 6f 72 79 2f 66 69 6c 65 20  <directory/file 
07a0: 23 31 3e 20 3c 64 69 72 65 63 74 6f 72 79 2f 66  #1> <directory/f
07b0: 69 6c 65 20 23 32 3e 20 2e 2e 2e 20 3c 64 69 72  ile #2> ... <dir
07c0: 65 63 74 6f 72 79 2f 66 69 6c 65 20 23 4e 3e 20  ectory/file #N> 
07d0: 5b 2d 64 65 6c 65 74 65 5d 20 5b 2d 6e 6f 70 72  [-delete] [-nopr
07e0: 6f 6d 70 74 5d 20 5b 2d 72 65 63 75 72 73 65 5d  ompt] [-recurse]
07f0: 20 5b 2d 68 65 6c 70 5d 22 0d 0a 09 22 4f 70 74   [-help]"..."Opt
0800: 69 6f 6e 73 3a 22 0d 0a 09 22 20 20 20 20 20 20  ions:"..."      
0810: 20 2d 72 65 63 75 72 73 65 20 72 65 63 75 72 73   -recurse recurs
0820: 65 73 20 74 68 72 6f 75 67 68 20 61 6c 6c 20 73  es through all s
0830: 75 62 64 69 72 65 63 74 6f 72 69 65 73 20 6f 66  ubdirectories of
0840: 20 61 6e 79 20 73 70 65 63 69 66 69 65 64 20 64   any specified d
0850: 69 72 65 63 74 6f 72 69 65 73 2e 22 0d 0a 09 22  irectories."..."
0860: 20 20 20 20 20 20 20 2d 64 65 6c 65 74 65 20 70         -delete p
0870: 72 6f 6d 70 74 73 20 74 6f 20 64 65 6c 65 74 65  rompts to delete
0880: 20 64 75 70 6c 69 63 61 74 65 73 20 28 62 75 74   duplicates (but
0890: 20 6e 6f 74 20 6f 72 69 67 69 6e 61 6c 73 2e 29   not originals.)
08a0: 22 0d 0a 09 22 20 20 20 20 20 20 20 2d 64 65 6c  "..."       -del
08b0: 65 74 65 20 77 69 74 68 20 2d 6e 6f 70 72 6f 6d  ete with -noprom
08c0: 70 74 20 64 65 6c 65 74 65 73 20 64 75 70 6c 69  pt deletes dupli
08d0: 63 61 74 65 73 20 77 69 74 68 6f 75 74 20 70 72  cates without pr
08e0: 6f 6d 70 74 73 20 28 62 75 74 20 61 67 61 69 6e  ompts (but again
08f0: 20 6e 6f 74 20 6f 72 69 67 69 6e 61 6c 73 2e 29   not originals.)
0900: 22 0d 0a 09 22 09 2d 68 69 64 64 65 6e 20 63 68  "...".-hidden ch
0910: 65 63 6b 73 20 68 69 64 64 65 6e 20 66 69 6c 65  ecks hidden file
0920: 73 2c 20 64 65 66 61 75 6c 74 20 69 73 20 74 6f  s, default is to
0930: 20 69 67 6e 6f 72 65 20 68 69 64 64 65 6e 20 66   ignore hidden f
0940: 69 6c 65 73 2e 22 0d 0a 09 22 09 2d 68 65 6c 70  iles."...".-help
0950: 20 64 69 73 70 6c 61 79 73 20 74 68 69 73 20 75   displays this u
0960: 73 61 67 65 20 6f 70 74 69 6f 6e 20 64 61 74 61  sage option data
0970: 2c 20 61 6e 64 20 69 67 6e 6f 72 65 73 20 61 6c  , and ignores al
0980: 6c 20 6f 74 68 65 72 20 61 72 67 75 6d 65 6e 74  l other argument
0990: 73 2e 22 0d 0a 09 22 22 0d 0a 09 22 45 78 61 6d  s."...""..."Exam
09a0: 70 6c 65 73 3a 22 0d 0a 09 22 20 20 20 20 20 20  ples:"..."      
09b0: 20 20 20 20 50 53 3e 2e 5c 66 69 6e 64 64 75 70      PS>.\finddup
09c0: 65 2e 70 73 31 20 63 3a 5c 64 61 74 61 20 64 3a  e.ps1 c:\data d:
09d0: 5c 66 69 6e 61 6e 63 65 20 2d 72 65 63 75 72 73  \finance -recurs
09e0: 65 22 0d 0a 09 22 20 20 20 20 20 20 20 20 20 20  e"..."          
09f0: 50 53 3e 2e 5c 66 69 6e 64 64 75 70 65 2e 70 73  PS>.\finddupe.ps
0a00: 31 20 64 3a 20 2d 72 65 63 75 72 73 65 20 2d 64  1 d: -recurse -d
0a10: 65 6c 65 74 65 20 2d 6e 6f 70 72 6f 6d 70 74 22  elete -noprompt"
0a20: 0d 0a 09 22 20 20 20 20 20 20 20 20 20 20 50 53  ..."          PS
0a30: 3e 2e 5c 66 69 6e 64 64 75 70 65 2e 70 73 31 20  >.\finddupe.ps1 
0a40: 63 3a 5c 75 73 65 72 73 5c 61 6c 69 63 65 5c 70  c:\users\alice\p
0a50: 69 63 74 75 72 65 73 5c 20 2d 72 65 63 75 72 73  ictures\ -recurs
0a60: 65 20 2d 64 65 6c 65 74 65 22 0d 0a 20 09 65 78  e -delete".. .ex
0a70: 69 74 0d 0a 7d 0d 0a 0d 0a 0d 0a 23 20 62 75 69  it..}......# bui
0a80: 6c 64 20 6c 69 73 74 20 6f 66 20 66 69 6c 65 73  ld list of files
0a90: 2c 20 62 79 20 72 75 6e 6e 69 6e 67 20 64 69 72  , by running dir
0aa0: 20 6f 6e 20 24 61 72 67 73 20 6d 69 6e 75 73 20   on $args minus 
0ab0: 65 6c 65 6d 65 6e 74 73 20 74 68 61 74 20 68 61  elements that ha
0ac0: 76 65 20 46 69 6e 64 44 75 70 65 2e 70 73 31 20  ve FindDupe.ps1 
0ad0: 73 77 69 74 63 68 65 73 2c 20 72 65 63 75 72 73  switches, recurs
0ae0: 69 76 65 6c 79 20 69 66 20 73 70 65 63 69 66 69  ively if specifi
0af0: 65 64 2e 0d 0a 0d 0a 24 66 69 6c 65 73 3d 28 64  ed.....$files=(d
0b00: 69 72 20 28 24 61 72 67 73 20 7c 20 3f 7b 24 5f  ir ($args | ?{$_
0b10: 20 2d 6e 65 20 22 2d 64 65 6c 65 74 65 22 20 2d   -ne "-delete" -
0b20: 61 6e 64 20 24 5f 20 2d 6e 65 20 22 2d 6e 6f 70  and $_ -ne "-nop
0b30: 72 6f 6d 70 74 22 20 2d 61 6e 64 20 24 5f 20 2d  rompt" -and $_ -
0b40: 6e 65 20 22 2d 72 65 63 75 72 73 65 22 20 2d 61  ne "-recurse" -a
0b50: 6e 64 20 24 5f 20 2d 6e 65 20 22 2d 68 69 64 64  nd $_ -ne "-hidd
0b60: 65 6e 22 7d 29 20 2d 72 65 63 75 72 73 65 3a 24  en"}) -recurse:$
0b70: 28 5b 62 6f 6f 6c 5d 28 24 61 72 67 73 20 2d 65  ([bool]($args -e
0b80: 71 20 22 2d 72 65 63 75 72 73 65 22 29 29 20 2d  q "-recurse")) -
0b90: 66 6f 72 63 65 3a 24 28 5b 62 6f 6f 6c 5d 28 24  force:$([bool]($
0ba0: 61 72 67 73 20 2d 65 71 20 22 2d 68 69 64 64 65  args -eq "-hidde
0bb0: 6e 22 29 29 20 7c 3f 7b 24 5f 2e 70 73 69 73 63  n")) |?{$_.psisc
0bc0: 6f 6e 74 61 69 6e 65 72 20 2d 65 71 20 24 66 61  ontainer -eq $fa
0bd0: 6c 73 65 7d 29 0d 0a 0d 0a 0d 0a 69 66 20 28 24  lse})......if ($
0be0: 66 69 6c 65 73 2e 63 6f 75 6e 74 20 2d 6c 74 20  files.count -lt 
0bf0: 32 29 20 20 23 20 69 66 20 74 68 65 20 6e 75 6d  2)  # if the num
0c00: 62 65 72 20 6f 66 20 66 69 6c 65 73 20 69 73 20  ber of files is 
0c10: 6c 65 73 73 20 74 68 61 6e 20 32 2c 20 74 68 65  less than 2, the
0c20: 6e 20 65 78 69 74 0d 0a 7b 0d 0a 09 22 4e 65 65  n exit..{..."Nee
0c30: 64 20 61 74 20 6c 65 61 73 74 20 74 77 6f 20 66  d at least two f
0c40: 69 6c 65 73 20 74 6f 20 63 68 65 63 6b 2e 60 61  iles to check.`a
0c50: 22 0d 0a 09 65 78 69 74 0d 0a 7d 0d 0a 0d 0a 66  "...exit..}....f
0c60: 6f 72 20 28 24 69 3d 30 3b 24 69 20 2d 6e 65 20  or ($i=0;$i -ne 
0c70: 24 66 69 6c 65 73 2e 63 6f 75 6e 74 3b 20 24 69  $files.count; $i
0c80: 2b 2b 29 20 20 23 20 43 79 63 6c 65 20 74 68 72  ++)  # Cycle thr
0c90: 75 20 61 6c 6c 20 66 69 6c 65 73 0d 0a 7b 0d 0a  u all files..{..
0ca0: 09 69 66 20 28 24 66 69 6c 65 73 5b 24 69 5d 20  .if ($files[$i] 
0cb0: 2d 65 71 20 24 6e 75 6c 6c 29 20 7b 63 6f 6e 74  -eq $null) {cont
0cc0: 69 6e 75 65 7d 20 20 23 20 66 69 6c 65 20 77 61  inue}  # file wa
0cd0: 73 20 61 6c 72 65 61 64 79 20 69 64 65 6e 74 69  s already identi
0ce0: 66 69 65 64 20 61 73 20 61 20 64 75 70 6c 69 63  fied as a duplic
0cf0: 61 74 65 20 69 66 20 24 6e 75 6c 6c 2c 20 73 6f  ate if $null, so
0d00: 20 64 6f 20 6e 65 78 74 20 66 69 6c 65 0d 0a 0d   do next file...
0d10: 0a 09 24 66 69 6c 65 63 68 65 63 6b 20 3d 20 24  ..$filecheck = $
0d20: 66 69 6c 65 73 5b 24 69 5d 20 20 09 20 20 20 20  files[$i]  .    
0d30: 20 20 23 20 62 61 63 6b 75 70 20 66 69 6c 65 20    # backup file 
0d40: 6f 62 6a 65 63 74 0d 0a 09 24 66 69 6c 65 73 5b  object...$files[
0d50: 24 69 5d 20 3d 20 24 6e 75 6c 6c 09 20 20 20 20  $i] = $null.    
0d60: 20 20 20 20 20 20 20 20 20 20 23 20 65 72 61 73            # eras
0d70: 65 20 66 69 6c 65 20 6f 62 6a 65 63 74 20 66 72  e file object fr
0d80: 6f 6d 20 6f 62 6a 65 63 74 20 64 61 74 61 62 61  om object databa
0d90: 73 65 2c 20 73 6f 20 69 74 20 69 73 20 6e 6f 74  se, so it is not
0da0: 20 6d 61 74 63 68 65 64 20 61 67 61 69 6e 73 74   matched against
0db0: 20 69 74 73 65 6c 66 0d 0a 0d 0a 09 66 6f 72 20   itself.....for 
0dc0: 28 24 63 3d 24 69 2b 31 3b 24 63 20 2d 6c 74 20  ($c=$i+1;$c -lt 
0dd0: 24 66 69 6c 65 73 2e 63 6f 75 6e 74 3b 20 24 63  $files.count; $c
0de0: 2b 2b 29 20 20 23 20 63 79 63 6c 65 20 74 68 72  ++)  # cycle thr
0df0: 6f 75 67 68 20 61 6c 6c 20 66 69 6c 65 73 20 61  ough all files a
0e00: 67 61 69 6e 0d 0a 09 7b 0d 0a 09 09 69 66 20 28  gain...{....if (
0e10: 24 66 69 6c 65 73 5b 24 63 5d 20 2d 65 71 20 24  $files[$c] -eq $
0e20: 6e 75 6c 6c 29 20 7b 63 6f 6e 74 69 6e 75 65 7d  null) {continue}
0e30: 20 20 23 20 24 6e 75 6c 6c 20 3d 20 66 69 6c 65    # $null = file
0e40: 20 77 61 73 20 61 6c 72 65 61 64 79 20 63 68 65   was already che
0e50: 63 6b 65 64 2f 6d 61 74 63 68 65 64 2e 0d 0a 09  cked/matched....
0e60: 0d 0a 09 09 69 66 20 28 24 66 69 6c 65 63 68 65  ....if ($fileche
0e70: 63 6b 2e 66 75 6c 6c 6e 61 6d 65 20 2d 65 71 20  ck.fullname -eq 
0e80: 24 66 69 6c 65 73 5b 24 63 5d 2e 66 75 6c 6c 6e  $files[$c].fulln
0e90: 61 6d 65 29 20 7b 24 66 69 6c 65 73 5b 24 63 5d  ame) {$files[$c]
0ea0: 3d 24 6e 75 6c 6c 3b 63 6f 6e 74 69 6e 75 65 7d  =$null;continue}
0eb0: 20 23 20 49 66 20 72 65 66 65 72 65 6e 63 69 6e   # If referencin
0ec0: 67 20 74 68 65 20 73 61 6d 65 20 66 69 6c 65 2c  g the same file,
0ed0: 20 73 6b 69 70 0d 0a 09 0d 0a 09 09 69 66 20 28   skip.......if (
0ee0: 24 66 69 6c 65 63 68 65 63 6b 2e 6c 65 6e 67 74  $filecheck.lengt
0ef0: 68 20 2d 65 71 20 24 66 69 6c 65 73 5b 24 63 5d  h -eq $files[$c]
0f00: 2e 6c 65 6e 67 74 68 29 20 20 23 20 69 66 20 66  .length)  # if f
0f10: 69 6c 65 73 20 6d 61 74 63 68 20 73 69 7a 65 20  iles match size 
0f20: 74 68 65 6e 20 63 68 65 63 6b 20 53 48 41 35 31  then check SHA51
0f30: 32 27 73 0d 0a 09 09 7b 0d 0a 09 09 09 69 66 20  2's....{.....if 
0f40: 28 24 66 69 6c 65 63 68 65 63 6b 2e 53 48 41 35  ($filecheck.SHA5
0f50: 31 32 20 2d 65 71 20 24 6e 75 6c 6c 29 20 20 20  12 -eq $null)   
0f60: 20 20 20 20 20 20 23 20 69 66 20 53 48 41 35 31        # if SHA51
0f70: 32 20 69 73 20 6e 6f 74 20 61 6c 72 65 61 64 79  2 is not already
0f80: 20 63 6f 6d 70 75 74 65 64 2c 20 63 6f 6d 70 75   computed, compu
0f90: 74 65 20 69 74 0d 0a 09 09 09 7b 20 0d 0a 09 09  te it.....{ ....
0fa0: 09 09 24 53 48 41 35 31 32 20 3d 20 28 67 65 74  ..$SHA512 = (get
0fb0: 2d 53 48 41 35 31 32 20 24 66 69 6c 65 63 68 65  -SHA512 $fileche
0fc0: 63 6b 2e 66 75 6c 6c 6e 61 6d 65 29 0d 0a 09 09  ck.fullname)....
0fd0: 09 09 24 66 69 6c 65 63 68 65 63 6b 20 3d 20 24  ..$filecheck = $
0fe0: 66 69 6c 65 63 68 65 63 6b 20 7c 20 25 7b 61 64  filecheck | %{ad
0ff0: 64 2d 6d 65 6d 62 65 72 20 2d 69 6e 70 75 74 6f  d-member -inputo
1000: 62 6a 65 63 74 20 24 5f 20 2d 6e 61 6d 65 20 53  bject $_ -name S
1010: 48 41 35 31 32 20 2d 6d 65 6d 62 65 72 74 79 70  HA512 -membertyp
1020: 65 20 6e 6f 74 65 70 72 6f 70 65 72 74 79 20 2d  e noteproperty -
1030: 76 61 6c 75 65 20 24 53 48 41 35 31 32 20 2d 70  value $SHA512 -p
1040: 61 73 73 74 68 72 75 7d 09 09 09 0d 0a 09 09 09  assthru}........
1050: 7d 0d 0a 09 09 09 69 66 20 28 24 66 69 6c 65 73  }.....if ($files
1060: 5b 24 63 5d 2e 53 48 41 35 31 32 20 2d 65 71 20  [$c].SHA512 -eq 
1070: 24 6e 75 6c 6c 29 20 20 20 20 20 20 20 20 20 23  $null)         #
1080: 20 72 65 73 75 6c 74 69 6e 67 20 69 6e 20 6e 6f   resulting in no
1090: 20 66 69 6c 65 20 62 65 69 6e 67 20 53 48 41 35   file being SHA5
10a0: 31 32 27 65 64 20 74 77 69 63 65 2e 0d 0a 09 09  12'ed twice.....
10b0: 09 7b 20 0d 0a 09 09 09 09 24 53 48 41 35 31 32  .{ ......$SHA512
10c0: 20 3d 20 28 67 65 74 2d 53 48 41 35 31 32 20 24   = (get-SHA512 $
10d0: 66 69 6c 65 73 5b 24 63 5d 2e 66 75 6c 6c 6e 61  files[$c].fullna
10e0: 6d 65 29 0d 0a 09 09 09 09 24 66 69 6c 65 73 5b  me)......$files[
10f0: 24 63 5d 20 3d 20 24 66 69 6c 65 73 5b 24 63 5d  $c] = $files[$c]
1100: 20 7c 20 25 7b 61 64 64 2d 6d 65 6d 62 65 72 20   | %{add-member 
1110: 2d 69 6e 70 75 74 6f 62 6a 65 63 74 20 24 5f 20  -inputobject $_ 
1120: 2d 6e 61 6d 65 20 53 48 41 35 31 32 20 2d 6d 65  -name SHA512 -me
1130: 6d 62 65 72 74 79 70 65 20 6e 6f 74 65 70 72 6f  mbertype notepro
1140: 70 65 72 74 79 20 2d 76 61 6c 75 65 20 24 53 48  perty -value $SH
1150: 41 35 31 32 20 2d 70 61 73 73 74 68 72 75 7d 09  A512 -passthru}.
1160: 09 09 09 0d 0a 09 09 09 7d 0d 0a 09 09 09 0d 0a  ........}.......
1170: 09 09 09 69 66 20 28 24 66 69 6c 65 63 68 65 63  ...if ($filechec
1180: 6b 2e 53 48 41 35 31 32 20 2d 65 71 20 24 66 69  k.SHA512 -eq $fi
1190: 6c 65 73 5b 24 63 5d 2e 53 48 41 35 31 32 29 20  les[$c].SHA512) 
11a0: 23 20 53 69 7a 65 20 61 6c 72 65 61 64 79 20 6d  # Size already m
11b0: 61 74 63 68 65 64 2c 20 69 66 20 53 48 41 35 31  atched, if SHA51
11c0: 32 20 6d 61 74 63 68 65 73 2c 20 74 68 65 6e 20  2 matches, then 
11d0: 69 74 27 73 20 61 20 64 75 70 6c 69 63 61 74 65  it's a duplicate
11e0: 2e 0d 0a 09 09 09 7b 0d 0a 09 09 09 09 0d 0a 09  ......{.........
11f0: 09 09 09 77 72 69 74 65 2d 68 6f 73 74 20 22 53  ...write-host "S
1200: 69 7a 65 20 61 6e 64 20 53 48 41 35 31 32 20 6d  ize and SHA512 m
1210: 61 74 63 68 3a 20 22 20 2d 66 6f 72 65 20 72 65  atch: " -fore re
1220: 64 20 2d 6e 6f 6e 65 77 6c 69 6e 65 0d 0a 09 09  d -nonewline....
1230: 09 09 77 72 69 74 65 2d 68 6f 73 74 20 22 60 22  ..write-host "`"
1240: 24 28 24 66 69 6c 65 63 68 65 63 6b 2e 66 75 6c  $($filecheck.ful
1250: 6c 6e 61 6d 65 29 60 22 20 61 6e 64 20 60 22 24  lname)`" and `"$
1260: 28 24 66 69 6c 65 73 5b 24 63 5d 2e 66 75 6c 6c  ($files[$c].full
1270: 6e 61 6d 65 29 60 22 22 0d 0a 0d 0a 09 09 09 09  name)`""........
1280: 24 6d 61 74 63 68 65 73 20 2b 3d 20 31 09 09 09  $matches += 1...
1290: 23 20 4e 75 6d 62 65 72 20 6f 66 20 6d 61 74 63  # Number of matc
12a0: 68 65 73 20 2b 2b 0d 0a 09 09 09 09 0d 0a 09 09  hes ++..........
12b0: 09 09 69 66 20 28 24 61 72 67 73 20 2d 65 71 20  ..if ($args -eq 
12c0: 22 2d 64 65 6c 65 74 65 22 29 20 20 20 20 20 20  "-delete")      
12d0: 20 20 23 20 63 68 65 63 6b 20 69 66 20 75 73 65    # check if use
12e0: 72 20 73 70 65 63 69 66 69 65 64 20 74 6f 20 64  r specified to d
12f0: 65 6c 65 74 65 20 74 68 65 20 64 75 70 6c 69 63  elete the duplic
1300: 61 74 65 0d 0a 09 09 09 09 7b 0d 0a 09 09 09 09  ate......{......
1310: 09 69 66 20 28 24 61 72 67 73 20 2d 65 71 20 22  .if ($args -eq "
1320: 2d 6e 6f 70 72 6f 6d 70 74 22 29 20 20 23 20 69  -noprompt")  # i
1330: 66 20 2d 64 65 6c 65 74 65 20 73 65 6c 65 63 74  f -delete select
1340: 2c 20 61 6e 64 20 2d 6e 6f 70 72 6f 6d 70 74 20  , and -noprompt 
1350: 73 65 6c 65 63 74 65 64 0d 0a 09 09 09 09 09 7b  selected.......{
1360: 0d 0a 09 09 09 09 09 09 64 65 6c 20 24 66 69 6c  ........del $fil
1370: 65 73 5b 24 63 5d 2e 66 75 6c 6c 6e 61 6d 65 20  es[$c].fullname 
1380: 20 23 20 74 68 65 6e 20 64 65 6c 65 74 65 20 74   # then delete t
1390: 68 65 20 66 69 6c 65 20 77 69 74 68 6f 75 74 20  he file without 
13a0: 70 72 6f 6d 70 74 69 6e 67 0d 0a 09 09 09 09 09  prompting.......
13b0: 09 77 72 69 74 65 2d 68 6f 73 74 20 22 44 65 6c  .write-host "Del
13c0: 65 74 65 64 20 64 75 70 6c 69 63 61 74 65 3a 20  eted duplicate: 
13d0: 22 20 2d 66 20 72 65 64 20 2d 6e 6f 6e 65 77 6c  " -f red -nonewl
13e0: 69 6e 65 0d 0a 09 09 09 09 09 09 77 72 69 74 65  ine........write
13f0: 2d 68 6f 73 74 20 22 60 22 24 28 24 66 69 6c 65  -host "`"$($file
1400: 73 5b 24 63 5d 2e 66 75 6c 6c 6e 61 6d 65 29 2e  s[$c].fullname).
1410: 60 22 22 0d 0a 09 09 09 09 09 7d 0d 0a 09 09 09  `"".......}.....
1420: 09 09 65 6c 73 65 0d 0a 09 09 09 09 09 7b 0d 0a  ..else.......{..
1430: 09 09 09 09 09 09 64 65 6c 20 24 66 69 6c 65 73  ......del $files
1440: 5b 24 63 5d 2e 66 75 6c 6c 6e 61 6d 65 20 2d 63  [$c].fullname -c
1450: 6f 6e 66 69 72 6d 20 23 20 6f 74 68 65 72 77 69  onfirm # otherwi
1460: 73 65 20 70 72 6f 6d 70 74 20 66 6f 72 20 63 6f  se prompt for co
1470: 6e 66 69 72 6d 61 74 69 6f 6e 20 74 6f 20 64 65  nfirmation to de
1480: 6c 65 74 65 0d 0a 09 09 09 09 09 7d 0d 0a 09 09  lete.......}....
1490: 09 09 09 69 66 20 28 28 67 65 74 2d 69 74 65 6d  ...if ((get-item
14a0: 20 2d 65 61 20 30 20 24 66 69 6c 65 73 5b 24 63   -ea 0 $files[$c
14b0: 5d 2e 66 75 6c 6c 6e 61 6d 65 29 20 2d 65 71 20  ].fullname) -eq 
14c0: 24 6e 75 6c 6c 29 20 23 20 63 68 65 63 6b 20 69  $null) # check i
14d0: 66 20 66 69 6c 65 20 77 61 73 20 64 65 6c 65 74  f file was delet
14e0: 65 64 2e 0d 0a 09 09 09 09 09 7b 0d 0a 09 09 09  ed........{.....
14f0: 09 09 09 24 66 69 6c 65 73 64 65 6c 65 74 65 64  ...$filesdeleted
1500: 20 2b 3d 20 31 09 09 23 20 75 70 64 61 74 65 20   += 1..# update 
1510: 72 65 63 6f 72 64 73 0d 0a 09 09 09 09 09 09 24  records........$
1520: 62 79 74 65 73 72 65 63 20 2b 3d 20 24 66 69 6c  bytesrec += $fil
1530: 65 73 5b 24 63 5d 2e 6c 65 6e 67 74 68 0d 0a 09  es[$c].length...
1540: 09 09 09 09 7d 0d 0a 0d 0a 09 09 09 09 7d 0d 0a  ....}........}..
1550: 09 0d 0a 09 09 09 09 24 66 69 6c 65 73 5b 24 63  .......$files[$c
1560: 5d 20 3d 20 24 6e 75 6c 6c 09 09 23 20 65 72 61  ] = $null..# era
1570: 73 65 20 66 69 6c 65 20 6f 62 6a 65 63 74 20 73  se file object s
1580: 6f 20 69 74 20 69 73 20 6e 6f 74 20 63 68 65 63  o it is not chec
1590: 6b 65 64 2f 6d 61 74 63 68 65 64 20 61 67 61 69  ked/matched agai
15a0: 6e 2e 0d 0a 09 09 09 7d 0d 0a 09 09 7d 09 0d 0a  n......}....}...
15b0: 09 7d 09 23 20 41 6e 64 20 6c 6f 6f 70 20 74 6f  .}.# And loop to
15c0: 20 6e 65 78 74 20 69 6e 6e 65 72 20 6c 6f 6f 70   next inner loop
15d0: 20 66 69 6c 65 0d 0a 7d 09 09 23 20 41 6e 64 20   file..}..# And 
15e0: 6c 6f 6f 70 20 74 6f 20 6e 65 78 74 20 66 69 6c  loop to next fil
15f0: 65 20 69 6e 20 6f 75 74 65 72 2f 6f 72 69 67 69  e in outer/origi
1600: 6e 61 6c 20 6c 6f 6f 70 0d 0a 77 72 69 74 65 2d  nal loop..write-
1610: 68 6f 73 74 20 22 22 0d 0a 77 72 69 74 65 2d 68  host ""..write-h
1620: 6f 73 74 20 22 4e 75 6d 62 65 72 20 6f 66 20 46  ost "Number of F
1630: 69 6c 65 73 20 63 68 65 63 6b 65 64 3a 20 24 28  iles checked: $(
1640: 24 66 69 6c 65 73 2e 63 6f 75 6e 74 29 2e 22 09  $files.count).".
1650: 23 20 44 69 73 70 6c 61 79 20 75 73 65 66 75 6c  # Display useful
1660: 20 69 6e 66 6f 3b 20 66 69 6c 65 73 20 63 68 65   info; files che
1670: 63 6b 65 64 20 61 6e 64 20 6d 61 74 63 68 65 73  cked and matches
1680: 20 66 6f 75 6e 64 2e 0d 0a 77 72 69 74 65 2d 68   found...write-h
1690: 6f 73 74 20 22 4e 75 6d 62 65 72 20 6f 66 20 64  ost "Number of d
16a0: 75 70 6c 69 63 61 74 65 73 20 66 6f 75 6e 64 3a  uplicates found:
16b0: 20 24 6d 61 74 63 68 65 73 2e 22 0d 0a 57 72 69   $matches."..Wri
16c0: 74 65 2d 68 6f 73 74 20 22 4e 75 6d 62 65 72 20  te-host "Number 
16d0: 6f 66 20 64 75 70 6c 69 63 61 74 65 73 20 64 65  of duplicates de
16e0: 6c 65 74 65 64 3a 20 24 66 69 6c 65 73 64 65 6c  leted: $filesdel
16f0: 65 74 65 64 2e 22 20 23 20 44 69 73 70 6c 61 79  eted." # Display
1700: 20 6e 75 6d 62 65 72 20 6f 66 20 64 75 70 6c 69   number of dupli
1710: 63 61 74 65 20 66 69 6c 65 73 20 64 65 6c 65 74  cate files delet
1720: 65 64 20 61 6e 64 20 62 79 74 65 73 20 72 65 63  ed and bytes rec
1730: 6f 76 65 72 65 64 2e 0d 0a 77 72 69 74 65 2d 68  overed...write-h
1740: 6f 73 74 20 22 24 62 79 74 65 73 72 65 63 20 62  ost "$bytesrec b
1750: 79 74 65 73 20 72 65 63 6f 76 65 72 65 64 2e 22  ytes recovered."
1760: 09 0d 0a 77 72 69 74 65 2d 68 6f 73 74 20 22 22  ...write-host ""
1770: 0d 0a 77 72 69 74 65 2d 68 6f 73 74 20 22 54 69  ..write-host "Ti
1780: 6d 65 20 74 6f 20 72 75 6e 3a 20 24 28 28 5b 64  me to run: $(([d
1790: 61 74 65 74 69 6d 65 5d 3a 3a 6e 6f 77 29 2d 24  atetime]::now)-$
17a0: 73 74 61 72 74 74 69 6d 65 7c 73 65 6c 65 63 74  starttime|select
17b0: 20 68 6f 75 72 73 2c 20 6d 69 6e 75 74 65 73 2c   hours, minutes,
17c0: 20 73 65 63 6f 6e 64 73 2c 20 6d 69 6c 6c 69 73   seconds, millis
17d0: 65 63 6f 6e 64 73 29 22 0d 0a 77 72 69 74 65 2d  econds)"..write-
17e0: 68 6f 73 74 20 22 22 0d 0a                       host ""..