Monday, September 26, 2016

Hack 3.20 Saving Duplicate Observations Removed by PROC SORT

SAS Programming Professionals,

Did you know that you can have SAS store duplicate observations that are removed during a sort that uses the NODUPKEY option in a separate SAS data set?

The DUPOUT= option of PROC SORT specifies a SAS data set that is used to store observations eliminated from a sorted data set when the NODUPKEY option is employed.  SAS sorts the specified data set and each observation that has duplicate BY statement variable values is written to the DUPOUT= data set as it is eliminated from the sorted data set.  Here is an example:

proc sort nodupkey
      by sex age;

The SAS log for this example looks like this:

1    proc sort nodupkey data=sashelp.class out=sorted_class dupout=dupkeys;
2        by sex age;
3    run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.               
NOTE: SAS sort was used.
NOTE: 8 observations with duplicate key values were deleted.
NOTE: The data set WORK.SORTED_CLASS has 11 observations and 5 variables.
NOTE: The data set WORK.DUPKEYS has 8 observations and 5 variables.
NOTE: PROCEDURE SORT used (Total process time):
      real time           0.01 seconds
      cpu time            0.01 seconds

Note that 8 observations with duplicate values of SEX and AGE were written to the DUPKEYS SAS data set.

This can come in very handy for QC-ing programs where you are eliminating duplicate-keyed observations during a sort.  You can double-check the observations in the DUPOUT= SAS data set and verify that the sort really got rid of observations that are not needed in the original sorted data set.

Best of luck in all your SAS endeavors!

(aka Michael A. Raithel)

Excerpt from the book:  Did You Know That?  Essential Hacks for Clever SAS Programmers

I plan to post each and every one of the hacks in the book to social media on a weekly basis.  Please pass them along to colleagues who you know would benefit.

Monday, September 19, 2016

Hack 3.18 Moving Catalog Entries Between SAS Catalogs

SAS Programming Professionals,

Did you know that you can easily copy or move catalog entries between SAS catalogs?

The COPY statement of PROC CATALOG allows you to copy or to move catalog entries between SAS catalogs.  This can come in handy when you are working on creating lots of formats, and they end up in your Work library.  Or, when you receive a format catalog from a client and want to copy the entries to a catalog that is normally allocated in your SAS programs.  Here is an example:

libname cperm "c:\permanent_lib";

proc format;
      value $gender
      "M" = "Male"
      "F" = "Female"
      value agecat
       low - 12  = "Pre Teen"
       12 - 14   = "Young Teen"
       15 - high = "Older Teen"

data cperm.class;
set  sashelp.class;

format sex $gender.
       age agecat.;

/* Move catalog entries to permanent catalog*/
proc catalog catalog=work.formats;
      copy out=cperm.Myfmts move;

/*Verify the catalog entries were moved*/
proc catalog catalog=cperm.Myfmts;

In the example, the LIBNAME statement, FORMAT procedure, and DATA step simply set up our example situation.  We create two format entries ($GENDER and AGECAT) in the default Work library catalog.  Then, we create a permanent SAS data set and assigned those formats to it.  We will be in “trouble” the next time we access the permanent data set, because the $GENDER and AGECAT formats will have evaporated when our SAS program, above, ends.

So, we use PROC CATALOG to move the formats to a permanent format catalog (CPERM.MYFMTS) that is always allocated in our file when we run our SAS programs.  Note that the MOVE option specifies to move the catalog entries, not to simply copy them.  After that operation, WORK.FORMATS is going to be missing two format entries!

Here is the log from the final PROC CATALOG:

Contents of Catalog CPERM.MYFMTS

#    Name        Type               Create Date          Modified Date    Description
1    AGECAT      FORMAT     13Feb12:13:27:32       13Feb12:13:27:32
2    AGEGROUP    FORMAT     17Sep07:10:55:21       17Sep07:10:55:21
3    GENDER      FORMATC    13Feb12:13:27:32       13Feb12:13:27:32

…showing that all of our hard work paid off!

Best of luck in all your SAS endeavors!

(aka Michael A. Raithel)

Excerpt from the book:  Did You Know That?  Essential Hacks for Clever SAS Programmers

I plan to post each and every one of the hacks in the book to social media on a weekly basis.  Please pass them along to colleagues who you know would benefit.

Monday, September 12, 2016

SAS & Bugs & Rock & Roll

SAS Programming Professionals,
SAS & bugs & rock & roll? But, of course!
Because of its amazing versatility, SAS is indisputably the greatest software package currently in use anywhere within the Milky Way Galaxy.  Can SAS input every type of flat file imaginable?  Yes!  Can SAS read and write to relational database management systems?  Yea!  Can SAS perform predictive analysis?  Yup!  Can SAS facilitate end-user analysis of data via web-based GUI’s?  Uh-huh!  Can SAS be used to Extract?  Of course!  Can SAS be used to Transform?  Affirmative!  Can SAS be used to Load?  Well, duh!  Can SAS help you to lose weight?  Aw c’mon; let’s not be silly here! 

Given that SAS is so powerful and that you want to either learn it or to learn more about it, here are a few helpful resources:
  • Learning SAS – You can either learn SAS from scratch or learn something new about SAS from SAS Education.  They offer dozens of classes in various formats that range from classroom lectures/exercises at SAS training facilities to e-learning classes that you can take from the privacy of your own office.  There is likely a SAS class just waiting for your own particular needs at:
  • SAS Documentation – Documentation on every SAS product can be found on the web site at the following URL:  The documentation is entirely free and easily searchable via a number of facilities on the documentation web pages.
  • Using SAS – There is no substitute for experience; and you can learn how to use SAS from a legion of SAS programming professionals by reading the technical papers they have published as SAS conferences.  Two great links for this are:
 Now that you have some ideas about SAS information, you should:  Extract it from the classes, documentation and technical papers; Transform the way that you write your SAS programs; and Load those new ideas and techniques into your professional SAS programming assignments.
& Bugs
So, when was the last time that you wrote a SAS program from scratch and didn’t have a single NOTE:, WARNING:, or ERROR: message in the log?  Right; thought so; same here!  Bugs are as much a constant of SAS programming as they are of any other type of programming.  There are generally two types of bugs in SAS programs:  logic bugs and programming bugs.  Logic bugs are generally harder to catch because they are tied to the algorithms you are using to Extract, Transform, and Load your data in your SAS programs.  They may result from bad specs, an insufficient understanding of the programming assignment, unruly data, or other meta-programming circumstances.  Programming bugs result from the way that you code SAS programing language statements to Extract, Transform, and Load data.
Fortunately, the SAS log is a great source for identifying and correcting programming bugs in SAS programs.  But, don’t take my word for it; check out a few recent papers from the experts:
Of course, if your attitude is summed up by the infamous quote:
The SAS Log is basically a waste of time and effort.  I never look at it because it only prints bad news!
…then the aforementioned links are not going to be of any help.  None at all!
& Rock & Roll
Do you listen to music when you are writing SAS programs?  I do.  But, the type of music that I listen to is entirely dependent upon the type of SAS program that I am writing.  If I am writing a new program with complicated logic, then I like to have either Spa or Ambient music playing in the background.  If it is a run-of-the-mill SAS program, then it’s got to be rock!
I have eclectic tastes in rock music, but my go-to-bands for SAS programming are (in no particular order):  Led Zeppelin, Pink Floyd, Nirvana, the Who,  Nine Inch Nails, David Bowie, Depeche Mode, the Cure, and Green Day.
You can see that “classic rock” is well represented in my list of favorite groups to program by.  If you are a fan of classic rock and want to find a classic rock station in your own particular corner of the US of A, then check out SAS data visualization guru Robert Allison’s classic blog post:
When did ‘your music’ become ‘classic rock’
Robert’s informative post will also show you the top 20 rock artists overall in the 25 radio stations in the sample, so that you will know what to expect.  What to expect—that is—besides great music with which to write your serious SAS programs.
SAS & bugs & rock & roll?  Yep; just another great day in the office!
Best of luck in all your SAS endeavors!
(aka Michael A. Raithel)

Tuesday, September 6, 2016

Hack 3.19 Moving SAS Data Sets with PROC COPY

SAS Programming Professionals,

Did you know that you can physically move a SAS data sets from one directory to another using PROC COPY?

Using PROC COPY obviates your having to move SAS data sets manually via Windows Explorer or SAS Explorer.  For example, let’s move a couple of SAS data sets from a temporary directory to a permanent directory:

libname templib "c:\temp";

libname mylib "H:\My Documents\My SAS Files\9.4";

proc copy in=templib out=mylib move;
      select air buy;

In this example, we moved both the AIR and the BUY SAS data sets from the c:\temp directory to the H:\My Documents\My SAS Files\9.4 directory.  Thereafter, if we were to view the directories in Windows Explorer or SAS Explorer, we would see that they have indeed been moved.

There are a few other options that you may need to specify to successfully move SAS data sets that have additional features:

·        ALTER – Is used when you are using the MOVE option to move files that are have an ALTER password.

·        CLONE – Specifies whether to copy the following file characteristics to the file that is moved:
o   Input/output buffer size
o   Data set compression
o   Whether free space is reused in compressed data sets
o   Whether compressed data sets can be accessed by observation number
o   Data representation
o   Encoding value

·        CONSTRAINT = YES | NO – Copy existing integrity constraints when moving a data set.

·        FORCE – Must be used to MOVE a SAS data set that has an audit trail file.

·        INDEX = YES – Copy indexes for indexed data sets that are moved.

You would simply specify these options directly after the MOVE option as in the example, above.

The MOVE option provides a very handy way to have SAS programmatically move files between various data libraries.

Best of luck in all your SAS endeavors!

(aka Michael A. Raithel)

Excerpt from the book:  Did You Know That?  Essential Hacks for Clever SAS Programmers

I plan to post each and every one of the hacks in the book to social media on a weekly basis.  Please pass them along to colleagues who you know would benefit.