| Barry MacDonnell's
Toolbox for WordPerfect
Macros, tips, and
templates for Corel® WordPerfect® for Windows®
| Page updated Jul 30, 2017
|RemDupes - "Remove duplicate items from a list of items"|
Related pages -
• Indexing documents with WordPerfect (manual and concordance methods)
• Index Tools - Tutorials, macros, and tips to help you index a document
This macro is designed to remove
duplicate items from a list of items.
You can also use it to
consolidate several concordance files into one concordance with no
See IndexDocs for more on creating an index in a WordPerfect document (manual and concordance methods).
Note that a companion macro, WordList, can
remove duplicates from the list it creates, so you won't need this
macro if you use WordList.
RemDupes can, of course, be used standalone on any list of words.
¤ The list should be in a document with no other text.
¤ Each item in the list must end in a hard return (that is, each item cannot span more than one line on the screen).
¤ The list does not need to be sorted first; the macro will sort the list alphabetically. (But the number of items in a list might be limited in sort routines. See Footnote 1 below.)
¤ You should not format anything before playing the macro. All words and phrases should be in plain text, and in one continuous column.
If you wish to combine several lists and remove duplicate entries from the final, consolidated list:
• open a new blank document;
• copy and paste (or use File, Insert) to place each list at the bottom of the new document; if you paste, you might get better results with some documents if you use Edit, Paste Special, Unformatted text to remove any formatting;
• save the newly combined list to disk for future use;
• if the combined list is not already onscreen, open it in the current window; and
• play the macro.
☼ To restore the newly combined list to the way it was before playing the macro, simply close the document without saving it (File, Close, No).
☼ (Note: Version 1.02 adds code to perform a multi-key sort on the list. If you are using an earlier version you can perform such a sort manually, as explained here.)
If you have lots of phrases
that do not seem to sort properly or that seem out of sequence, you can
sort the file on the first word on each line (the first "sort key"),
then, for each phrase that starts with the same word, sort on the
second word ("Key 2"), etc.
Use Tools, Sort, New and create a new user-defined sort with added keys (press the "Add Key..") button. The new sort keys are just like the first sort key, but with the Word field incremented for each new key in the sort. That is, Word 1, Word 2, Word 3. If you have 4-level deep words you want sorted by the fourth level, create 4 keys, etc.
☼ Words or phrases have to be exactly the same to be considered "duplicates" by the macro. Version 1.02 automatically removes any leading or trailing spaces (soft or hard) from each line in the list, because such spaces adjacent to an earlier word or phrase would cause the macro to treat the next item -- if it has no such spaces -- as not identical. But if "identical" words or phrases seem to appear next to each other, use Reveal Codes to verify that they are indeed duplicates.
[...Continued from above:]
Number of entries: The WordPerfect program (at least up to WPX4 (build 18.104.22.1686)) may limit the number of items it can sort to 32K of memory. This seems true of paragraph sorting, and may be true of line sorting -- the type that is used by the Unique (and RemDupes) macro. Until (and if) this issue is addressed by Corel, you should break apart large documents into, say, 20-25 pages per block and play the macro on each section separately. You can then manually combine the lists with Insert>File, but you will need to remove duplicates.