Observations on reading/writing SAS7BDAT and QVD

I had to process some large sas7bdat files today, and I thought someone might be interested in hearing how it worked.

I had a 10.9 GB SAS file that I loaded into EM. It took about 3 hours, and resulted in a 27 million row x 107 column table. It ended up taking about 100 GB of memory to run the process.

I then wrote it to a 3.8 GB QVD, which took quite a while to write (at least 30 min). Reloading that file took about 8 minutes. Once complete, memory usage was about 19 GB.

1 Like