Phil Holland's New SAS Book is a Programming Tour de Force

SAS Programming Professionals,

The latest book in Phil Holland’s Power User’s Guide series, SAS Programming and Data Visualization Techniques, is an unmitigated tour de force.  It contains fourteen fact-filled chapters covering three very relevant main topic areas:  Programming and Efficiency Techniques, External Interfaces, and Data Visualization.  As with the other books in the Power User’s Guide series, this book is written in Phil’s authoritative, but conversation style and has lots and lots of code examples that illustrate real-world SAS programming solutions.

There are so many usable programming gems in this book—even for very experienced SAS programming professionals—that it is hard to call them all out in a brief review.  However, three specifically come to mind because they can save any organization much more than the price of the book. 

·        Part I: Programming Efficiency Techniques, Chapter 1: The Basics of Efficient SAS Coding.  In this chapter, Phil discusses speed versus low maintenance.  It is a very nuanced discussion that should make seasoned SAS programmers reconsider what they might have thought was the obvious choice.

 

·        Part II: External Interfaces, Chapter 4: SAS to R to SAS.  Because of its powerful statistical and graphic capabilities, its low price (it is actually free), and its popularity in academic institutions, R has gained entry into the programming infrastructure of many organizations—including my own.  So, sooner or later you are going to need to interface with R; and when you do, you will reach for the examples in this chapter.

·        Part II: External Interfaces; Chapter 8: Everyday Uses for SAS Output Delivery System (ODS).  The section titled Disguising a Web Page is absolutely brilliant and worth the price of the book.  I have bookmarked that page and already begun using this simple, but deeply clever technique in my own SAS programs.

My prediction is that when you purchase SAS Programming and Data Visualization Techniques:  A Power User’s Guide, it will not sit in your programming bookshelf between other SAS programming books.  Instead, it will be beside your keyboard on your desk with pages dog-eared and a dozen yellow sticky-notes protruding from various pages.  Mark my word!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

(aka Michael A. Raithel)

Amazon Author's Page:http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epw

Doctor, I Have This Friend Who Has A Pain Right About Here

SAS Programming Professionals,

We have all heard the old joke about the person who sidles up to a doctor at a party and describes in great detail a medical problem a "friend" is having in order to get free medical advice.  It could just as easily be a person buttonholing an accountant for free tax advice, a lawyer for free legal advice, a personal trainer for free fitness advice, a CEO for free business advice, a chef for free cooking advice, a financial planner for free investment advice, a dietitian for free dietary advice, a college professor for free educational advice, and so on.  But, what about free SAS programming advice?

The good news is that if you have a question about SAS programming, you don't have to sidle up to a SAS expert and ask it in a roundabout way. There are a number of great venues where you can ask SAS technical questions directly and have them answered by SAS experts from the US and from around the world.  And, the only cost to you is the several hundred keystrokes you expend writing a concise description of your problem with supporting information such as:

• The portion of your SAS program in question
• A snippet of the log showing an error, if applicable
• The version of SAS you are using
• The operating system you are running SAS on

Here are some of the facilities available to help SAS professionals with their programming questions:

• Support Communities on support.sas.com -- There are very active discussion groups with titles such as Base SAS Procedures, SAS Macro Language and Data Step, ODS and Base Reporting, and many more.  Users post their programming questions and then SAS experts from around the world and from SAS post possible solutions.  Questioners usually get many and varied answers to their questions.  This web-based community even has its own document titled "How to ask a Question in SAS Support Communities?" to help lead you through the process if you are new.  You can access the SAS Support Communities via this link:  https://communities.sas.com/community/support-communities 

• SAS-L Listserv.  This oldest medium for SAS Q&A is realized as an email listserv.  You subscribe to the SAS-L listserv, and then when people post questions and answers, all subscribers get the messages.  SAS-L has a very loyal base of SAS experts who usually jump on questions immediately with answers from many different angles.  The sasCommunity.org web site has a good write-up on how to subscribe to SAS-L:  http://www.sascommunity.org/wiki/SAS-L

• LinkedIn SAS Groups.  There are over a dozen SAS discussion groups on LinkedIn.  Some notable ones for Q&A are:  SAS Professional Forum, SAS Programming Tips, and SAS Author: Phillip R Holland.  These discussion groups are conducted much the same way as the previously mentioned Q&A sites, but also include articles of general interest to SAS programmers.  The LinkedIn home page is:  https://www.linkedin.com/

• Code Clinics.  Most of the SAS Local Users Groups (LUGs) and SAS Regional Users Groups (RUGs) feature a "code clinic" at their meetings.  The code clinics are staffed with SAS programming experts who can help you with programming problems one-on-one.  They are expecting drop-ins, so bring your SAS questions and sidle-away.

There are doubtlessly other web sites and other facilities out there in the big wide world where SAS users will be glad to pitch in and help answer your questions. But, the list above is a good starting point.

Hey, as long as I have you here; I have this friend who is interested in competing in his first triathlon; you wouldn't happen to know anything about training for one would you?

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

aka Michael A Raithel

Amazon Authors page:

http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Keep Your Hands Off of My SAS Data Sets!

SAS Programming Professionals,

I am all about sharing.  Knock on my door and I will gladly lend you a stick of margarine, a cup of sugar, an egg or two, some flour, a corkscrew, or a beer.  Not a problem to borrow a tie, one of my extra belts, a white shirt (if it fits), a scarf, or a pair of gloves.  Sure, I will gladly lend you one the books in my home library, a couple of my music Cd's, or one of my movie DVDs.  I am reasonably sure that most of these items will either be returned to me in due time, or reciprocated, or paid forward.  But, keep your hands off of my SAS data sets!

I would bet that final sentiment regarding SAS data sets is pretty prevalent in organizations where SAS programmers work with shared storage resources.  Whether you are storing your SAS data sets in server directories or on network directories, you do not want to have them deleted, resorted, updated, or otherwise overwritten by other programmers on your team.  At least, not without your permission.

Fortunately, most organizations create security permissions that largely safeguard against unauthorized access of data.  They implement security packages--either native to the OS or purchased from vendors--to ensure that data is accessed only by those with the need to know.  Groups of programmers are given access to specific directories and other staff, with the exception of systems administrators, cannot get into them or see them at all.  Despite all of this protection, data integrity issues can come from programmers within your own group; programmers who have access to the same directories that you do; programmers who have the same access rights that you do.

Unfortunately, there is no foolproof way for you to keep your teammates from crunching your SAS data sets.  But, here are a few protection measures that you can put into place to help safeguard your SAS data sets:

• SAS Data Set Passwords.  SAS allows you to specify ALTER, READ, and WRITE passwords.  Users must know the password in order to process password-protected data sets with SAS.  So, you can specify passwords for your do-not-disturb SAS data sets, wait for your co-workers to complain, and determine if they really do need access to those data sets.

• ACCESS=READONLY Libname Option.  This option specifies that no data sets in the library can be updated and no new files can be written to the library.  This option can be effectively deployed in a shared group AUTOEXEC.sas file.  If your colleagues grumble, tell them to copy the data sets in question to one of their libraries and process them there.  Caution them to be mindful of version control issues.

• SAS Views.  Create views of your SAS data sets via the DATA step or PROC SQL and allow your colleagues to use the views instead of the actual data sets.  The beauty of this approach is that you can have your valued data sets in one directory and the views in another.  Point your coworkers at the views directory and do not tell them the whereabouts of the permanent data sets.

• Generation Groups.  You can specify for SAS to keep several generations of your SAS data sets available.  Consequently, when one of them is modified, you will still have the older version available.

• SAS Audit Logs.  SAS Audit Logs can be used to determine who updated SAS data sets after the fact. They don't stop your colleagues from actually updating SAS data sets, but you can use them to determine the who, what, and when so that you can storm into the right office for an explanation.

• LOCK Statement.  You could use the LOCK statement to lock your SAS data sets so that no other SAS program can read or write to the file.  This is a bit extreme and would require that your SAS program with the LOCK statements be running for the duration of when you wanted to safeguard your SAS data sets.  This option can be effectively deployed in a shared group AUTOEXEC.sas file.

• Zip Files.  You could simply zip your SAS data sets up into zip files, delete the permanent SAS data sets, and restore the data sets from the zip files when you need them.  This is another extreme measure, but if your data sets get clobbered on a regular basis, you may find it more appealing than having them constantly restored.

Obviously, the best solution for keeping your important SAS data sets from being updated by your teammates is good communications between all involved, and a shared set of best practices for accessing and modifying SAS data sets.  But, when that is not available, you may have to reach for some of the ideas I have posted here.

Oh, and about that book that you borrowed from my SAS bookshelf library a couple of months ago... can I get it back?

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

aka Michael A. Raithel

Amazon Author's Page:  

http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0 

We Don't Need No Education!

SAS Programming Professionals,

So, how did you first learn SAS programming?

Originally, I was self taught.  Many years ago, I learned SAS on the job when a systems programmer quit and I took over supporting a mainframe performance software package that was written in SAS. I got a copy of the Base SAS users guide and the SAS Procedures guide and learned how to write my own SAS programs.  It was new and fun; and SAS was more powerful than any of the other programming languages I had been using.

After I had been programming with SAS for about three years, my then employer finally paid to send me to a SAS class.  I entered that class pretty confident of my SAS programming abilities, but was humbled within the first hour.  I had been running my SAS programs in batch on mainframe computers.  All of the students in the class ran their SAS programs interactively with SAS Display Manager, which I had never seen before.  So, I had to catch up to their level of expertise just to do the class exercises.

It was a great class!  I learned to use the SAS Display Manager; the proper ways to perform match merges; how the Program Data Vector works in the DATA step; the intricacies of PROC MEANS and its first cousin PROC SUMMARY; and a host of other very useful programming techniques. I learned the fundamentals of SAS that I had missed by simply jumping into using it and getting all of my information from the manuals.  That class made me a much stronger SAS programmer.

Today's programming professionals who are interested in taking formal SAS classes to increase their SAS programming acumen have a lot of choices.  SAS currently has the following training formats:

• Classroom.  You can take an instructor-led SAS class in one of SAS's state-of-the-art training facilities located throughout the US and in many other countries.

• Live Web Classroom.  Let an instructor-led SAS class come to you via the Internet.  This format saves on travel costs and time out of the office.

• E-Learning.  This option allows you to take SAS classes at your own pace via the Internet 24/7.  This is another way to save travel costs and time out of the office.

• On-Site Training.  Have qualified SAS instructors come on-site to teach you and your colleagues various SAS courses.

• Mentoring Services.  This option provides you with a SAS instructor who becomes your coach to help you learn how to write SAS programs that address your organization's unique data processing needs.

You can find out much more about available SAS classes being offered, their content, when they are scheduled, and the training formats in the Training section of the support.sas.com web site:

    http://support.sas.com/training/index.html

Check it out because there is likely a SAS class that would make you a stronger programmer in a format that fits with your own work schedule.

"We don't need no education"?  Nope, that couldn't be further from the truth!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

aka Michael A Raithel

Check out my SAS books:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Don't Call Me an April Fool!

SAS Programming Professionals,

With over 50,000 certification credentials awarded to people around the world, the SAS Global Certification program is obviously meeting the needs of a goodly number of SAS programming professionals.  Interested SAS software users can demonstrate an in-depth understanding of SAS by earning credentials in SAS Programming, SAS Analytics, SAS Administration, SAS Data Management, and SAS Business Intelligence.  They study the material, master it, take the test, and earn the credential.

You can learn more about the SAS Global Certification program by visiting this link: //support.sas.com/certify/

As it stands now, the two SAS Base Programmer certification tests require a person to answer multiple choice and short-answer questions.  Personally, I am locked-and-loaded for the day that one of those exams requires an essay.  I know that the secret to getting a good grade on an essay question is not to simply answer the prompt.  The secret is to add additional information about the topic which shows that you have a mastery of the overall subject matter.  Since I have been programming with SAS for a very long time, I have a lot of very sage and insightful comments that I would put into my essay.  So, I have no doubt that I would crush it!

Here are some of the clever things I have learned along the way that I would weave into my essay:

• The “DO WHILE” statement is known as the “Stalactite DO” since it is evaluated at the top.  The “DO UNTIL” statement is known as the “Stalagmite DO” since it is evaluated at the bottom.

• When I need to fix a SAS program with a hard-to-find error, I just put a %MEND at the bottom and rerun it.

• An example of a SAS regular expression is:  “Don’t forget the semicolon”!

• The SAS Log is basically a waste of time and effort.  I never use it because it only prints bad news!

• I sometimes code: PROC OPTIONS; ...without the “RUN;” statement, just to keep my OPTIONS open!

• A known SAS programming trick is: If you code a DROP statement and a KEEP statement for the same variable in a DATA step, you will see a bright flash on your monitor as they cancel each other out.

• It is difficult to determine if PROC MIXED has executed successfully because it prints MIXED messages.

• The SAS Display Manager is a long-time SAS Institute employee who is responsible for supervising the setup of SAS Institute booths at conferences and trade shows.

• Despite its strong odor, the RANK procedure is very popular among SAS programmers serving in the US armed forces.

• Due to heightened security concerns, SAS programs containing PROC EXPLODE have been banned from all government buildings, airports and rail stations.

• A recent survey of American SAS programmers found that most use PROC IMPORT far more than they use PROC EXPORT.  However, the same survey determined that this did not significantly add to the United States trade deficit.

 

Okay, so do I really have to say:  April fools!  Nah, didn't think so.

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
aka Michael A. Raithel
Check out my SAS books:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Raiders of the Lost Spreadsheet

SAS Programming Professionals,

Have you ever peered intently into an unfamiliar data delivery directory, realized what was in it, rolled over onto your side, stared blankly into the distance, and dejectedly uttered something akin to:

    "Spreadsheets!  Why did it have to be spreadsheets?"

If so, then we are definitely on the same page.  Why does it always have to be spreadsheets?

The answer to that question is actually pretty obvious when you think about it.  The popularity of Microsoft Office has made Excel one of the most popular mediums for storing data.  It is used extensively in grade schools, middle schools high schools, and colleges.  People with home businesses use it; office administrators use it; clerical staff use it; scientists use it; lawyers use it; hospital workers use it; Federal, state and local government workers use it; and programmers use it too.

An individual who needs to store data in electronic format and then process it may not have SAS, or C++, or JAVA, or C#, or PYTHON, or PHP, or R, or MATLAB, or ColdFusion, or FOCUS, or FORTRAN, or Groovy, or JavaScript, or MOBY, or MUMPS, or NATURAL, or Perl, or PHP, or PL/SQL, or PowerShell, or Python, or S-PLUS, or Visual Basic installed on his or her PC.  But, that person will undoubtedly have Microsoft Office and thus have Excel.  That is why it always has to be spreadsheets.

But, processing data stored in spreadsheets is not really a problem for intrepid SAS programmers.  When I go on a data exploration expedition where there is a good chance of encountering spreadsheets, I pack the usual:  my brown leather jacket, fedora, and bullwhip.  But, most importantly, I put SAS/Access Interface to PC Files into my backpack.

SAS/Access Interface to PC Files is a SAS for Windows product that allows you to read, write, and update data in Excel and Access.  As such, it is a must-have for your Windows SAS installation.

Here is an example of a program that I use to map out the contents of an unexplored spreadsheet:

   ods rtf file="G:\BigProject\Worksheets in NewDataSpreadsheet.rtf";

   libname xlslib "G:\NewProject\DeliveryDirectory\NewDataSpreadheet.xlsx" access=readonly;

   proc sql;
   create table WorkSheets as
   select distinct(compress(MEMNAME,"',$")) as WorkSheet_Name,
       name as ColumnName
   from dictionary.columns
   where libname = 'XLSLIB';
   quit;

   proc print noobs data=WorkSheets;
   var WorkSheet_Name ColumnName;
   title1 "Workseets in NewDataSpreadheet.xlsx";
   run;

   ods rtf close;

The ODS statement specifies that my report will be created as an RTF document.  Because I have SAS/Access Interface to PC Files, the LIBNAME statement allocates the NewDataSpreadheet.xlsx spreadsheet much the same way as it would for a SAS data set.  (Notice that I specified access=readonly so that I do not accidentally update the spreadsheet).  Since I have "LIBNAME-d" the spreadsheet, information about its worksheets and column names is now available in the SAS Dictionary Tables.

I use PROC SQL to extract the name of each worksheet (variable WorkSheet_Name) in the Excel file; and the names of the columns (variable ColumnName) within each worksheet and then plop them into a SAS data set for further exploration. The code snippet compress(MEMNAME,"',$") gets rid of the annoying quotes and dollar signs that are found in spreadsheet MEMNAMEs.  Then, I use the PRINT procedure to create a report.  A simple, neat, quick, and easily macro-tized piece of code.

Here are several good references that you can use to find out more about processing spreadsheets with SAS:

• SAS Access product information on the SAS website:  http://support.sas.com/software/products/access/index.html
• Exchanging Data between SAS and Microsoft Excel: Tips and Techniques to Transfer and Manage Data More Efficiently, by Bill Benjamin  http://support.sas.com/publishing/authors/benjamin.html
• Conference papers by Vince DelGobbo such as: Creating Multi-Sheet Microsoft Excel Workbooks with SAS®: The Basics and Beyond Part 1 http://support.sas.com/resources/papers/proceedings14/SAS050-2014.pdf 

Armed with those resources, some pluck, a sense of adventure, and with your own trusty copy of SAS/Access Interface to PC Files, you too can be a raider of the lost spreadsheet!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
aka Michael A. Raithel
Amazon Author's Page:
http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0 

What is a SAS Index Good for Anyhow?

 SAS Programming Professionals,

If there was a SAS performance tool that could drastically reduce your program’s I/O’s, lower its CPU time, and decrease its run time, would you use it?  Of course you would!  Such a performance tool exists; it is called a SAS index.  SAS Indexes can dramatically improve the performance of programs that access small subsets of observations from large SAS data sets.  They do this by only accessing and returning the observations that you specify in a WHERE expression, instead of reading the entire SAS data set.

It is easy to understand how a SAS index can help you to directly access the observations that you need in a particular SAS data set.  As an exercise, do the following:

• Open support.sas.com in a web browser
• Type "rtrace" in the search window at the top of the page
• Click on the Search button

The support.sas.com search function returns about seventy links.  When you click on any of those links, you get a page in the documentation that discusses the SAS RTRACE facility.  This saves you the tedious effort of going through the entire SAS Online documentation, page-by-page, looking for occurrences of the word “rtrace”.

A SAS index is analogous to the search function discussed above.  A good index allows your programs to quickly access the subset of SAS observations that you need from a large SAS data set when you specify a key variable value (or values) that must be matched.  This can dramatically improve the speed and efficiency of your SAS programs.

Conversely, badly conceived SAS indexes return far too many observations and are no better than reading the entire data set sequentially.  In the analogy, above, consider how many pages would be returned and how much longer it would take if you searched the SAS Online Documentation for the word “SAS”.  That is why it is important to know more about the selection criteria for index variables, as well as the actual creation and use of SAS indexes.

After deciding that an index is appropriate for your subsetting purposes, you have three tools to choose from to create one: 

1. The DATASETS procedure, 
2. The SQL procedure, 
3. The DATA option in the DATA step or in a Procedure.  

When you first use one of these tools to create an index, SAS creates a separate index file and associates it with your SAS data set.  The index file has the same name as the original SAS data set, but has a suffix of ".sas7bndx."  SAS stores additional indexes in that file and deletes the file when all indexes have been removed from the data set.

You can create a Simple index from a single variable, or a Composite index from two or more variables.  A SAS data set can have as many indexes as you think are necessary.

You can exploit indexes with the WHERE statement, the BY statement, or with the KEY statement used in conjunction with either a SET or MODIFY statement.  In doing so, you will be increasing the efficiency of your SAS programs that use the index.  That is what SAS indexes are good for!

There is enough information about SAS indexes to fill an entire book.  Here are a few resources to consider if you are interested in learning more about SAS indexes:

• The Complete Guide to SAS Indexes, book by Michael A. Raithel  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0
• The Basics of SAS Indexes, tech paper by Michael A. Raithel  http://www2.sas.com/proceedings/sugi30/247-30.pdf

• Understanding SAS Indexes, SAS Institute documentation  http://support.sas.com/documentation/cdl/en/lrcon/67885/HTML/default/viewer.htm#n06cy7dznbx6gen1q9mat8de6rdq.htm

If you find that your programs are consistently extracting very small subsets of observations from very large SAS data sets, then SAS indexes just might be the right tool for you.

Best of Luck in all your SAS endeavors!  

----MMMMIIIIKKKKEEEE 

aka Michael A. Raithel 

Amazon Author's Page: 

http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0 

I Know What You Did Last Summer!

SAS Programming Professional,

I know what you did last summer.

If it was unintentional, then you probably don't know what I am talking about.  If it was intentional, then you probably thought that I would never find out.  Either way, the damage is done.  The actions that you took on that warm summer evening are as clear to me now as they would have been if I had been watching over your shoulder while you did them.  I know what you did last summer: You updated one of my SAS data sets.

We work on the same project  and both have read, write, update, and delete rights to the project's directories.  The production SAS data set that I created for the spring data delivery was inexplicably updated in the summer.  And, you were the one who did it.  Because we have been teammates for a while, I am giving you the benefit of the doubt.  I bet that you made a copy of the production SAS program for a different use, updated it, but forgot to change the LIBREF to point to your test SAS data library.  So when you ran it, you accidentally deleted 400 observations and updated 273 observations in the production data set.

Oh, you want to know how I determined it was you and how I know exactly what changed.

Well, because that production data set is very important, I used PROC DATASETS to create a SAS audit trail file for it.  SAS audit trails record changes to SAS data sets.  They can record the before and after image of observations that were changed, the type of change, the date/time of the change, and the userid of the person who changed the SAS data set.  So, SAS audit trails can be very useful in a shared directory environment where many staff members have access to important SAS data sets.

Here is the code I used to create the audit trail for the production SAS data set:

proc datasets library=prodlib nolist;
        audit SpringDeliveryData;
        initiate;
        log admin_image=yes
              before_image=yes
              data_image=no
              error_image=yes;
        run;
quit;

When I executed that DATASETS procedure code, SAS created a file named SpringDeliveryData.sas7baud in the same directory as the SAS data set.  When an observation is updated, added, or deleted from SpringDeliveryData, SAS writes an observation to the audit trail data set containing the variables in the original SAS data set and six specific audit trail variables.  Of note are _ATDATETIME_ which specifies the date/time of the change; _ATOPCODE_ which specifies the type of change that took place--e.g. add, delete, modify; and _ATUSERID_ which specifies the userid of the person whose SAS program made the change.

When I noticed that SpringDeliveryData had been modified, I used a PROC PRINT to dump the audit trail file.  That is how I know that the data set was updated at 5:27 PM on August 5th by a program submitted under your userid.

You are interested in using SAS audit trails for your own production SAS data sets?  Great!  You can find a comprehensive write-up in the documentation on support.sas.com at this link:

http://support.sas.com/documentation/cdl/en/lrcon/67885/HTML/default/viewer.htm#n0ndg2uekz7qkbn1caoki2hzqx8l.htm

Don't fret about the updates to the SpringDeliveryData SAS data set.  I am going to request that our systems administrator restore the data set to the day before the summer update.  That way, we will have the original data set available in case our client has questions about it.

Good to know that I was right that you accidentally updated the production data set last summer.  Oh, don't go.  Unfortunately we have one more thing to talk about:

I know what you did last fall...

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
aka Michael A Raithel
Amazon Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Excuse Me; But, Is That a 32-Bit or a 64-Bit SAS Catalog?

SAS Programming Professionals,

I don’t know about you, but I get pretty determined to prove them wrong when people tell me that I cannot do something.  I am not talking about fantastical things such as flying through the heart of the sun and out the other side without getting burned.  Nor, am I talking about social things like becoming president of the United States or an author on the New York Times Bestseller list.  And, I am not talking about physical things such as swimming 2.4 miles, biking 112 miles, and running 26.2 miles back-to-back on the same day.  No, I am talking about being told that I cannot do something with SAS.

For example, I was once told:

• that you could not summarize impossibly large SAS data sets to load a data warehouse.  So, I figured out a way to do it:  http://www2.sas.com/proceedings/sugi22/DATAWARE/PAPER113.PDF 

• that you could not measure the performance of SAS/IntrNet application programs.  So, I figured out a way to do it: http://www2.sas.com/proceedings/sugi31/226-31.pdf 

• that you could not determine which SAS products individual staff members were using on shared servers.  So, I figured out a way to do it: http://www2.sas.com/proceedings/sugi29/215-29.pdf 

• that you could not create a chargeback system for UNIX and Linux systems without purchasing an accounting package.  So, I figured out a way to do it:  http://www2.sas.com/proceedings/forum2007/194-2007.pdf

      Consequently, when I was told that there was no SAS facility for programmatically determining whether a Windows SAS catalog was a 32-bit catalog or a 64-bit catalog, I resolved to figure out a way to do it.

The background is that my organization plans to migrate from 32-bit SAS to 64-bit SAS as part of a SAS 9.3 to SAS 9.4 upgrade.  SAS data sets are compatible between the two bitages, but SAS catalogs are not.  Stating the problem: you cannot open a 64-bit SAS catalog with 32-bit SAS.  So, it is advantageous to have a tool for determining which SAS catalog is which bitage as you move forward into a mixed-bit programming environment during the transition.

I did my due diligence and researched every place that I thought I might be able to find a way to differentiate the bitage.  An indicator in PROC CATALOG if I ran it with the STAT option enabled?  Nope.  Something in the directory portion of a PROC CONTENTS listing with the DETAILS option specified?  Nope.  A lesser-known option of PROC DATASETS?  Nope.  How about a flag in the Dictionary Tables CATALOGS table or in the SASHELP Views VCATALG view?  Nope.  A Usage Note on support.sas.com.  Nope.  A  SAS technical paper published at either SAS Global Forum or a Regional SAS Users Group?  Nope, not that either.

I figured that if you could not tell the difference within SAS, itself, how about if you looked at the catalogs as simply files.  So, I got a 32-bit SAS catalog and a 64-bit SAS catalog and opened them with WordPad to take a look inside.  Bingo!  There was enough information in the very first record of both catalog files to determine the difference.  So, I wrote a program that tested for the string of characters that told the tale.

Here is the SAS program that I wrote:

/*Macro to determine bitage of a SAS catalog */

%MACRO Test_Cat_Bitage(SASCatalog);

filename sascat "&SASCatalog";

data decompcat(keep=CAT_BITS SAS_Catalog);

length CAT_BITS $8

         SAS_Catalog $50;

infile sascat obs=1 truncover;

input bigline $charzb32767. ;

if index(bigline, "W32_7PRO") > 0 then CAT_BITS = "W32_7PRO";

      else if index(bigline, "X64_7PRO") > 0 then CAT_BITS = "X64_7PRO";

      else CAT_BITS = "Unknown ";

SAS_Catalog = strip("&SASCatalog");

label CAT_BITS    = "Bitage of SAS Catalog"

        SAS_Catalog = "Full Path SAS Catalog Name"

        ;

run;

proc append base=AllCatalogs

                  data=decompcat;

run;

%MEND Test_Cat_Bitage;

/* Example of executing the macro to read a catalog file */

%Test_Cat_Bitage(c:\temp\gender.sas7bcat);

As you can see, the program determines the bitage of a SAS catalog by treating the catalog as a file, not as a catalog.  It opens the catalog file and inspects the first line for a specific character string: W32_7PRO for 32-bit catalogs; X64_7PRO for 64-bit catalogs.  Once it determines the bitage, the program writes an observation to data set AllCatalogs in the WORK library.  Each observation in AllCatalogs has two variables:  CAT_BITS, which specifies whether the catalog is 32 or 64 bits, and SAS_Catalog, which is the full path name of the SAS catalog file.  

The object of this particular setup is to run the macro against several, a score, dozens, hundreds, or thousands of SAS catalogs and build a SAS data set which identifies their bitage.  After that, one may choose to copy AllCatalogs to a permanent SAS data set, or create a report from it.  Or both.

Being a talented SAS programmer yourself, I would bet that you also do not like it when people tell you that you cannot do something with SAS.  Right?  Yea, it goes with the territory.  How about posting a comment telling us about a particularly difficult SAS problem you encountered and the clever way that you resolved it?  Bet you can’t do that.

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
(aka Michael A Raithel)
Amazon Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

The 20 Most Promising SAS Solutions Providers

SAS Programming Professionals,

Without peeking, what do you think are the top best selling magazines in the United States?

Well, as of this writing, the top ten best selling magazines in the US are:

1. AARP The Magazine - 22,528,478 copies per issue
2. The Costco Connection – 8,631,275 copies per issue
3. Game Informer – 8,169,524 copies per issue
4. Better Homes and Gardens – 7,617,038 copies per issue
5. Reader’s Digest – 5,577,717 copies per issue
6. Good Housekeeping – 4,346,757 copies per issue
7. National Geographic – 4,232,205 copies per issue
8. Family Circle – 4,100,977 copies per issue
9. People – 3,563,035 copies per issue
10. Woman’s Day – 3,449,692 copies per issue

Probably no surprises there... unless, of course, your own favorite magazine did not make the top 10 list.

I would bet that among SAS programming professionals, the top magazine to read is going to be the current issue of CIO Review Magazine, which  spotlights the 20 Most Promising SAS Solutions Providers:

http://magazine.cioreview.com/Jan-2015/SAS/

That issue is packed with interesting descriptions of how a diverse group of organizations are harnessing the vast power of SAS to provide solutions to real-world business problems for both internal and external clients. So, what could be a more compelling read to a SAS programming professional such as yourself?  No magazine that I can think of!

I am happy to report that my own organization is listed among the 20 Most Promising SAS Solutions Providers.  You can find our profile on Page 45 in the above link, or skip directly to it with this link:

http://sas.cioreview.com/vendor/2015/westat

You can get a lot of good ideas as to how you might extend your own utilization of SAS and SAS's capabilities by reading this issue of CIO Review magazine.  And, unlike some of the other magazines in the US top ten, you can get the latest copy of this magazine with the click of a mouse!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

(aka Michael A. Raithel)

Amazon Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0 

The Lowdown on the Downloads

SAS Programming Professionals,

I would bet that the most popular four-letter-word in the English language; the one that gets the most attention; the one that everybody loves to see; the one that makes people the happiest is:  free.

Everybody likes to get something for free.  We love it when we purchase two garments at the regular price and get the third one for free.  We smile when after purchasing nine coffees with our customer loyalty card we get the tenth one for free.  We enjoy racking up those frequent flyer miles so that we can get a free flight or a free upgrade.  We happily grab that toothpaste bundle that has a free smaller tube or a free toothbrush attached.  In the store, we reach for that bag of chips that has 10% more included for a limited time for free.  And, so on.

The good news for SAS programming professionals is that SAS offers a number of value-added products that you can download from the support.sas.com web site for--you guessed it--free.

The main SAS downloads page can be found at this link:  http://support.sas.com/downloads/index.htm .  From that web page, you can explore downloads for the various SAS products that you have licensed to see what freebies you may be able to bring to bear on your own programming work.

Here are some of my favorite free downloads for Base SAS Software:

• The SAS Universal Viewer -- This software allows users to access SAS data sets without having SAS installed on their workstations.  Link: http://support.sas.com/downloads/package.htm?pid=1249
• The ODS Graphics Editor -- This is an interactive tool that allows users to edit and customize the ODS graphs that they create.  Link: http://support.sas.com/downloads/package.htm?pid=999 
• The SAS XML Mapper --  The XML Mapper analyzes the structure of an XML file, or schema, and generates metadata.  It basically provides a gateway for you to process XML files with SAS. Link:  http://support.sas.com/downloads/package.htm?pid=1002 
• The SAS Installation Qualification (IQ) Tool and the SAS Operational Qualification (OQ) Tool.  These two handy tools validate that SAS is installed and operating correctly. The IQ tool validates that you have SAS correctly installed on your computer.  The OQ tool validates that SAS is operating according to SAS's specifications.  Link:  http://support.sas.com/rnd/migration/planning/validation/sasoq.html
• SAS Data Sets for Practicing Statistical Analysis -- SAS provides about 50 SAS data sets that you can download to use tosharpen your SAS programming skills.  Link:  http://support.sas.com/learn/statlibrary/statlib_eg4.2/top_data.htm
• The SAS Installation Reporter -- This amazing program provides a fantastically detailed report of your SAS installation.  Link:  http://support.sas.com/kb/20/390.html

You can do a lot with just that handful of free downloads for Base SAS software.

Check out the free SAS downloads and see which ones may end up saving you time at work.  No need to hurry though, because unlike those deals at your supermarket, drugstore, local clothing store, or airline, the SAS downloads are not "free for a limited time only."  All that you need to exploit them is the knowledge that they are there, a couple of minutes to download them, and the smarts to use them to improve the utility of your SAS programs.

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
(aka Michael A Raithel)
Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Are You Doing Things SAS Backwards?

SAS Programming Professionals,

Are you doing things SAS backwards?

Yea; we have all been there.  A manager, or a user, or a client, asks you to perform an analysis or create a data set and gives you a sketchy description of what they want. Maybe they say "It's like that report you did last November", or they send a few descriptive lines in an email, or they verbally describe it in a meeting, or they sketch out some ideas on a notepad.  So, you dutifully write a program that processes what you think is the relevant data and creates the result set you believe they want. You QC the results, present it to the requester and... you are told that it is not right.  So, you modify your program according to their feedback and create the next result set.  And, it is still not right.  You find yourself repeating the process again and again until your manager or user or client is finally satisfied.

What is wrong with this process is that you are starting to write a SAS program when you do not have solid specifications.  Not understanding exactly what the requester wants invariably leads to wasted programming time.  So, you need to get your user to be specific about what, exactly, s/he wants before you start programming.  At a minimum, the requester should define:

Input

• The data sources you are to extract the data from
• The selection criteria you are to use to subset the data
• The variables they want in the final data set or report

Process

• What types of analysis they want performed on the data
• The formulas for any new variables that are to be created
• Any transformations that are to be made to existing variables
• What to do with missing values for important variables
• Whether any variables should be suppressed from the final result set

Output

• For data sets, the type of data set and the variable order
• For reports, the layout of the report, including variable order, headings, titles, footnotes, and summary columns
• For graphs, the type of graph, variables to be graphed, titles, legends, footnotes, and reference lines
• The type of additional documentation needed, such as a PROC CONTENTS or frequencies of the delivery data set

The best thing you can do is to have your user send you the specifications in writing.  That performs two functions:  It makes the person stop and really think out what s/he wants, and it gives you a solid starting point.  From there, you can ask questions to fill in any of the details that were not covered in the original, written request.  After a few iterations, you should have a solid spec to work from.

If you only receive verbal directions or sketchy notes via email and are not likely to get anything more solid, you may have to write the specs yourself.  In that case, you would write down the details of the Input, Process, and Output steps that you believe your client wants and send them to the client.  Let your client review the specs and verify that they are what s/he wants, or suggest changes to your proposal that better define what is needed.  Once the email exchange culminates in fully ratified specs, you are good-to-go with starting the SAS programming.

Almost invariably, any programming effort--no matter how well spec-ed out--will require an iteration or two to satisfy a user's requirements.  However, you can drastically reduce the time you spend programming by nailing-down your user's needs as much as possible before you begin to write your SAS program.  Few things in life are more satisfying than creating a tight, elegant, efficient SAS program that conforms to specifications and that produces the requested result set.

Remember; without specifications to refer to, you cannot judge whether or not the program you wrote and the result sets you created conform to the requirements.  And, as Lewis Carol wrote:  "If you do not know where you are going, any road will lead you there."

Best of Luck in all of your SAS endeavors!

----MMMMIIIIKKKKEEEE

(aka Michael A. Raithel)

Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0 

Did You Know That 15 Minutes Could Save You...? Yea, Everybody Knows That!

SAS Programming Professionals,


You would have to be a caveman to not be familiar with the "Did you know that 15 minutes could save you 15% or more on car insurance?" advertisement.  Multiple versions of the commercial are on television, on the radio, on the Internet, and likely on every other medium that has ever hosted that cute little gecko with the cheery British accent.


Well, the same sort of thing is true about SAS programming.  I would state it this way:

Did you know that studying SAS for 15 minutes a day could save you hours of programming time?

It is true.  Knowledge is power, and the more SAS knowledge you have the more powerful is your SAS programming prowess.

Some programmers are content to simply let their work tasks take them wherever they may lead, and to learn additional SAS programming hacks from solving the various software development problems that arise.  They hit a problem that cannot be solved with the SAS that they know, look up additional PROCS or functions, or options, or call routines, or SQL statements, etc. and use the new material to resolve the question at hand.  Fair enough!  On the job training is good, and it may make sense to let real-world problems dictate some component of your SAS acumen.

But, relying solely on a stimulus-response type of SAS learning is a bit short sighted.  A better way to increase your SAS knowledge is to be aggressive and actively study SAS.  That way, when you bump into a programming problem you have not previously experienced, you can reach into your in-memory SAS toolkit and pull out one of the SAS hacks that you have read about, but not yet had an opportunity to try.  You will already be familiar with the concept, so you will not need to spend much time coming up-to-speed on it.  Instead, you can customize the PROC, or ODS template, or hash, or... whatever to resolve the issue and go from there.

There are two things you need to implement this proactive approach to increasing your SAS knowledge:  fifteen minutes each day, and good material to study.

Fifteen minutes is only 1% of the 1,440 minutes available to you in a day.  So, it should be easy enough to find fifteen minutes to study SAS each day, right?  Well, that can be problematic for all of us.  We have so many work and personal commitments to attend to, that every moment of every day seems to already be scheduled with other things to do.  So, we need to get creative.  Perhaps you can take fifteen minutes on the subway or bus commute to work to read SAS material.  What about taking fifteen minutes the moment you get into your office before you begin reading emails?  How about using fifteen minutes of your lunchtime to read new SAS hacks?  Or, maybe fifteen minutes at bedtime.  Whatever time slot you decide upon, try to make it your own, inviolable, uninterruptable SAS study period each day of the week.  If becoming a stronger SAS programmer is important to you, you will find that elusive 1% of a day.

Regarding good material to study; there is a plethora of great SAS programming publications out there.  But, for "... studying SAS for 15 minutes a day ...", I recommend one of the SAS tips books.  Here are my recommendations:

• Did You Know That?  Essential Hacks for Clever SAS Programmers, by Michael A. Raithel
• Power User's Guide to SAS Programming, by Phil R Holland
• Carpenter's Guide to Innovative SAS Techniques, by Art Carpenter
• Cody's Collection of Popular SAS Programming Tasks and How to Tackle Them, by Ron Cody
• In the Know...SAS Tips & Techniques from Around the World, Second Edition, by Phil Mason
• The Little SAS Book; A Primer, Fifth Edition, by Lora D. Delwiche and Susan J. Slaughter

Of course the first recommendation is my favorite because I am the author.  However, you absolutely cannot go wrong with any or all of the publications in my list.

So, did you know that studying SAS for 15 minutes a day could save you hours of programming time?

Well yea; everybody who read this blog knows that!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE
(aka Michael A. Raithel)
Author page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90/ref=ntt_dp_epwbk_0

Meet the New Year; Same as the Old Year...NOT!

SAS Programming Professionals,

Happy New Year to you, your families, and your colleagues!

This is the time of year when people make all types of New Year's resolutions.  According to the USA.Gov web site, the most popular resolutions are:

1. Lose Weight
2. Volunteer to Help Others
3. Quit Smoking
4. Get a Better Education
5. Get a Better Job
6. Save Money
7. Get Fit
8. Eat Healthy Food
9. Manage Stress
10. Manage Debt
11. Take a Trip
12. Reduce, Reuse, and Recycle
13. Drink Less Alcohol

And, these resolutions all make sense because they promote personal growth.  It is a new year, so why not start making the changes that you want to carry forward throughout the year?

Along with personal changes, perhaps it is also time to consider what changes you could make to improve your SAS programming skills.  There is no "most popular list of SAS resolutions" for us to reference, so I will suggest some:

1. Lose unnecessary variables and observations when inputting data sets
2. Volunteer to help junior SAS programmers
3. Quit writing programs without comments in them
4. Get a better understanding of the intricacies of PROC SQL
5. Get more sophisticated in your SAS Macro programming
6. Save time by re-purposing your old programs
7. Purchase some interesting SAS Press books... and read them
8. Join a local SAS users group and attend meetings
9. Get better at using the Output Delivery System
10. Take a SAS class
11. Write a technical paper for SAS Global Forum
12. Reduce processing time by writing more efficient programs
13. Subscribe to the SAS-L listserv

Hardly a comprehensive list, but definitely a good start, no?  And, I would bet that one or more of these resolutions resonate with you; that they are things you have been wanting to do, but have not quite found the time to work on in 2014.  Well, there are 364 more days of 2015 just waiting for you to dig into your own SAS resolutions.

Of course, you could simply sit back and do the same old, same old.  But, how could you continue to achieve personal growth as a SAS programming professional if you did that?

Meet the new year; same as the old year--Oh, we really think not!

Best of luck in all your SAS endeavors!

----MMMMIIIIKKKKEEEE

(aka Michael A. Raithel)

Author's Page:  http://www.amazon.com/Michael-A.-Raithel/e/B001K8GG90